function programming experimental lib

why another fp lib

I like fp style and I haven’t found a lib with these features:

streamingly, I can handle infinite data source such as go channel or a socket reader
lazy evaluation, well, huge list processing wouldn’t make me oom
generic, the interface{} type ocurrs in a map function sucks
chain calls, functions should be compositional
clean, I hope the core of the lib would be clean
performance, good performance would be a bonus

And when I decide to build a new fp lib, the theory of lisp come to my mind immediately.

If I can bring cons,car,cdr into golang, that would be cool and attractive for me.

So I spend couple of days make this, and I hope you like it. Any feedback is welcome.

Own to the poor performance of golang's closure and small objects gc, the lisp like version
runs a little slow. So I have to refact the whole project with iterator pattern, for now it runs
2xtimes than before and faster than go-linq at least, enjoy it.

goos: darwin
goarch: amd64
pkg: demo/fpdemo
cpu: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
BenchmarkFP-12             274879              3711 ns/op            1184 B/op         42 allocs/op
BenchmarkGoLinq-12         246768              4545 ns/op            1632 B/op         69 allocs/op

source

Stream is created from a source, source is a slice, a channel, or even a reader.

e.g. create stream from slice

StreamOf([]int{1, 2, 3})
StreamOf([]string{"a", "b", "c"})

e.g. create stream from channel

ch := make(chan string, 1)
StreamOf(ch)

e.g. create stream from iterator function

var i int
fn := func() (int, bool) {
	i++
	return i, i < 5
}
StreamOf(fn)

e.g. create stream from custom source

type Source interface {
	// source element type
	ElemType() reflect.Type
	// Next element
	Next() (reflect.Value, bool)
}

StreamOfSource(mySource)
// create a file source, read text line by line
file, _ := os.Open("example.txt")
defer file.Close()
source := NewLineSource(file)
StreamOfSource(source)

high order functions

Map

slice := []string{"a", "b", "c"}
var out []string
StreamOf(slice).Map(strings.ToUpper).ToSlice(&out)
suite.ElementsMatch(out, []string{"A", "B", "C"})

// map with selector
slice := []string{"a", "b", "c"}
var out []string
StreamOf(slice).Map(func(e string) (string, bool) {
	return strings.ToUpper(e), e == "b"
}).ToSlice(&out)
suite.ElementsMatch(out, []string{"B"})

// map with error
slice := []string{"a", "b", "c"}
var out []string
err := StreamOf(slice).Map(func(e string) (string, error) {
	return strings.ToUpper(e), genErr(e == "a" || e == "c")
}).ToSlice(&out)
suite.Len(out, 0)
suite.Error(err)

FlatMap

// flatmap sub collection
slice := []string{"abc", "de", "f"}
out := StreamOf(slice).FlatMap(func(s string) []byte {
	return []byte(s)
}).Bytes()
suite.Equal("abcdef", string(out))

// flatmap sub stream
databases := []string{"db1", "db2"}
tables := []string{"table1", "table2"}
fullnames := StreamOf(databases).FlatMap(func(db string) Stream {
	return StreamOf(tables).Map(func(table string) TupleString {
		return TupleStringOf(db, table)
	})
}).Map(func(t TupleString) string {
	return t.E1 + "." + t.E2
}).Strings()
suite.Equal([]string{"db1.table1", "db1.table2", "db2.table1", "db2.table2"}, fullnames)

Filter

slice := []string{"a", "b", "c"}
out := StreamOf(slice).Filter(func(s string) bool {
	return s == "b"
}).Strings()
suite.Equal([]string{"b"}, out)

// there're some helper partial functions
slice := []string{"a", "b", "c"}
out := StreamOf(slice).Filter(Equal("b")).Strings()
suite.Equal([]string{"b"}, out)

out := StreamOf(slice).Filter(EqualIgnoreCase("B")).Strings()
suite.Equal([]string{"b"}, out)

out := StreamOf([]string{"a",""}).Reject(EmptyString()).Strings()
suite.Equal([]string{"a"}, out)

Reject

slice := []string{"a", "b", "c"}
out := StreamOf(slice).Reject(func(s string) bool {
	return s == "b"
}).Strings()
suite.Equal([]string{"a", "c"}, out)

Foreach

var out string
slice := []string{"abc", "de", "f"}
out1 := StreamOf(slice).Foreach(func(s string) {
	out += s
}).Strings()
suite.Equal("abcdef", out)
suite.ElementsMatch(slice, out1)

Flatten

slice := []string{"abc", "de", "f"}
out := StreamOf(slice).Map(func(s string) []byte {
	return []byte(s)
}).Flatten().Bytes()
suite.Equal("abcdef", string(out))

deep flatten

databases := []string{"db1", "db2"}
tables := []string{"table1", "table2"}
fullnames := StreamOf(databases).FlatMap(func(db string) Stream {
	return StreamOf(tables).Map(func(table string) TupleString {
		return TupleStringOf(db, table)
	})
}).Map(func(t TupleString) string {
	return t.E1 + "." + t.E2
}).Strings()
suite.Equal([]string{"db1.table1", "db1.table2", "db2.table1", "db2.table2"}, fullnames)

slice := [][]string{
	{"abc", "de", "f"},
	{"g", "hi"},
}
var out [][]byte
StreamOf(slice).Map(func(s []string) [][]byte {
	return StreamOf(s).Map(func(st string) []byte {
		return []byte(st)
	}).ToSlice(&out)
}).Flatten().Flatten().Bytes()
suite.Equal("abcdefghi", string(out))

Partition/PartitionBy/LPartitionBy

source := []string{"a", "b", "c", "d"}

out := StreamOf(source).Partition(3).StringsList()
suite.Equal([][]string{
	{"a", "b", "c"},
	{"d"},
}, out)

slice := []string{"a", "b", "c", "d", "e", "c", "c"}
out := StreamOf(slice).PartitionBy(func(s string) bool {
	return s == "c"
}, true).StringsList()
suite.Equal([][]string{
	{"a", "b", "c"},
	{"d", "e", "c"},
	{"c"},
}, out)

slice := []string{"a", "b", "c", "d", "e", "c", "c"}
out := StreamOf(slice).LPartitionBy(func(s string) bool {
	return s == "c"
}, true).StringsList()
suite.Equal([][]string{
	{"a", "b"},
	{"c", "d", "e"},
	{"c"},
	{"c"},
}, out)

Reduce/Reduce0

source := []string{"a", "b", "c", "d", "a", "c"}

var out map[string]int
StreamOf(source).Reduce(map[string]int{}, func(memo map[string]int, s string) map[string]int {
	memo[s] += 1
	return memo
}).To(&out)
suite.Equal(map[string]int{
	"a": 2,
	"b": 1,
	"c": 2,
	"d": 1,
}, out)

max := func(i, j int) int {
	if i > j {
		return i
	}
	return j
}
min := func(i, j int) int {
	if i < j {
		return i
	}
	return j
}
sum := func(i, j int) int { return i + j }

source := []int{1, 2, 3, 4, 5, 6, 7}
ret := StreamOf(source).Reduce0(max).Int()
suite.Equal(int(7), ret)

ret = StreamOf(source).Reduce0(min).Int()
suite.Equal(int(1), ret)

ret = StreamOf(source).Reduce0(sum).Int()
suite.Equal(int(28), ret)

First

slice := []string{"abc", "de", "f"}
q := StreamOf(slice)
out := q.First()
suite.Equal("abc", out.String())

IsEmpty

slice := []string{"abc", "de", "f"}
q := StreamOf(slice)
suite.False(q.IsEmpty())
out := q.First()
suite.Equal("abc", out.String())

Take/TakeWhile

slice := []string{"abc", "de", "f"}
out := strings.Join(StreamOf(slice).Take(2).Strings(), "")
suite.Equal("abcde", out)

slice := []string{"a", "b", "c"}
out := StreamOf(slice).TakeWhile(func(v string) bool {
	return v < "c"
}).Strings()
suite.Equal([]string{"a", "b"}, out)

Skip/SkipWhile

slice := []string{"abc", "de", "f"}
out := strings.Join(StreamOf(slice).Skip(2).Strings(), "")
suite.Equal("f", out)

slice := []string{"a", "b", "c"}
out := StreamOf(slice).SkipWhile(func(v string) bool {
	return v < "c"
}).Strings()
suite.Equal([]string{"c"}, out)

Sort/SortBy

slice := []int{1, 3, 2}
out := StreamOf(slice).Sort().Ints()
suite.Equal([]int{1, 2, 3}, out)

slice := []string{"abc", "de", "f"}
out := StreamOf(slice).SortBy(func(a, b string) bool {
	return len(a) < len(b)
}).Strings()
suite.Equal([]string{"f", "de", "abc"}, out)

Uniq/UniqBy

slice := []int{1, 3, 2, 1, 2, 1, 3}
out := StreamOf(slice).Uniq().Ints()
suite.ElementsMatch([]int{1, 2, 3}, out)

slice := []int{1, 3, 2, 1, 2, 1, 3}
out := StreamOf(slice).UniqBy(func(i int) bool {
	return i%2 == 0
}).Ints()
suite.ElementsMatch([]int{1, 2}, out)

Size

out := StreamOf(slice).Size()
suite.Equal(2, out)

Contains/ContainsBy

slice := []string{"abc", "de", "f"}
q := StreamOf(slice)
suite.True(q.Contains("de"))

slice := []string{"abc", "de", "f"}
q := StreamOf(slice)
suite.True(q.ContainsBy(func(s string) bool { return strings.ToUpper(s) == "F" }))

GroupBy

slice1 := []string{"abc", "de", "f", "gh"}
var q map[int][]string
StreamOf(slice1).Map(strings.ToUpper).GroupBy(func(s string) int {
	return len(s)
}).To(&q)
suite.Equal(map[int][]string{
	1: {"F"},
	2: {"DE", "GH"},
	3: {"ABC"},
}, q)

Append/Prepend

slice := []string{"abc", "de"}
out := StreamOf(slice).Append("A").Strings()
suite.Equal([]string{"abc", "de", "A"}, out)

slice := []string{"abc", "de"}
out := StreamOf(slice).Prepend("A").Strings()
suite.Equal([]string{"A", "abc", "de"}, out)

Union/Sub/Interact

slice1 := []string{"abc", "de", "f"}
slice2 := []string{"g", "hi"}
q1 := StreamOf(slice1).Map(strings.ToUpper)
q2 := StreamOf(slice2).Map(strings.ToUpper)
out := q2.Union(q1).Strings()
suite.Equal([]string{"ABC", "DE", "F", "G", "HI"}, out)

slice1 := []int{1, 2, 3, 4}
slice2 := []int{2, 1}
out := StreamOf(slice1).Sub(StreamOf(slice2)).Ints()
suite.Equal([]int{3, 4}, out)

slice1 := []int{1, 2, 3, 4}
slice2 := []int{2, 1}
out := StreamOf(slice1).Interact(StreamOf(slice2)).Ints()
suite.ElementsMatch([]int{1, 2}, out)

Zip

slice1 := []int{1, 2, 3}
slice2 := []int{4, 5, 6, 7}
out := StreamOf(slice1).Zip(StreamOf(slice2), func(i, j int) string {
	return strconv.FormatInt(int64(i+j), 10)
}).Strings()
suite.ElementsMatch([]string{"5", "7", "9"}, out)

ZipN

slice1 := []int{1, 2, 3}
slice2 := []int{4, 5, 6, 7}
slice3 := []int{2, 3}
out := StreamOf(slice1).ZipN(func(i, j, k int) string {
	return strconv.FormatInt(int64(i+j+k), 10)
}, StreamOf(slice2), StreamOf(slice3)).Strings()
suite.ElementsMatch([]string{"7", "10"}, out)

Result

stream transform would not work unless Run/ToSlice is invoked.

Run

use Run if you just want stream flows but do not care about the result

// the numbers would not print without Run
StreamOf(source).Foreach(func(i int) {
	fmt.Println(i)
}).Run()

ToSlice

slice := []string{"a", "b", "c"}
var out []string
StreamOf(slice).Map(strings.ToUpper).ToSlice(&out)
suite.ElementsMatch(out, []string{"A", "B", "C"})

Monand

Error Monand

You can map value with error

	var v int64
	err := M("a").Map(func(s string) (int64, error) {
		return strconv.ParseInt(s, 10, 64)
	}).To(&v)
	suite.Zero(v)
	suite.Error(err)

Maybe Monad

You can ExpectPass/ExpectNoError on an Maybe monand

	var v int64
	err := M("2").Map(func(s string) (int64, error) {
		return strconv.ParseInt(s, 10, 64)
	}).ExpectPass(func(i int64) bool {
		return i > 0
	}).To(&v)
	suite.Equal(int64(2), v)
	suite.NoError(err)

	var v int64
	err := M("2").Map(func(s string) (int64, error) {
		return strconv.ParseInt(s, 10, 64)
	}).ExpectNoError(func(i int64) error {
		return errors.New("xerr")
	}).To(&v)
	suite.Equal(int64(0), v)
	suite.Error(err)

Transform to stream

	var out []int
	err := M("2").Map(func(s string) (int64, error) {
		return strconv.ParseInt(s, 10, 64)
	}).StreamOf(func(i int64) []int {
		return Times(int(i)).Ints()
	}).ToSlice(&out)
	suite.NoError(err)
	suite.Equal([]int{0, 1}, out)

Compose multiple Monand

	m1 := M("20").Map(func(s string) (int64, error) {
		return strconv.ParseInt(s, 10, 64)
	})

	var score int64
	err := M("10").Map(func(s string) (int64, error) {
		return strconv.ParseInt(s, 10, 64)
	}).Zip(func(a, b int64) int64 {
		return a + b
	}, m1).To(&score)
	suite.NoError(err)
	suite.Equal(int64(30), score)

Once Monand

	var cnt int
	var score int64
	m := M("10").Map(func(s string) (int64, error) {
		cnt++
		return strconv.ParseInt(s, 10, 64)
	})
	err := m.To(&score)
	suite.NoError(err)
	suite.Equal(int64(10), score)
	suite.Equal(1, cnt)
	err = m.To(&score)
	suite.NoError(err)
	suite.Equal(int64(10), score)
	suite.Equal(2, cnt)

Result

	var v int64
	err := M("a").Map(func(s string) (int64, error) {
		return strconv.ParseInt(s, 10, 64)
	}).To(&v)
	suite.Zero(v)
	suite.Error(err)

If you just want the error

	err := M("21a").Map(func(s string) (int64, error) {
		return strconv.ParseInt(s, 10, 64)
	}).Error()
	suite.Error(err)

KVStream

kvstream is merely a simple wrapper for golang’s map.

Foreach

iterater a map

	m := map[string]int{
		"a": 1,
		"b": 2,
	}
	var keys []string
	var vals []int
	KVStreamOf(m).Foreach(func(key string, val int) {
		keys = append(keys, key)
		vals = append(vals, val)
	}).Run()
	suite.ElementsMatch([]string{"a", "b"}, keys)
	suite.ElementsMatch([]int{1, 2}, vals)

Map

tranform a map to another map

	m := map[string]int{
		"a": 1,
		"b": 2,
	}
	var vk map[int]string
	KVStreamOf(m).Map(func(k string, v int) (int, string) {
		return v, k
	}).To(&vk)
	suite.Equal("a", vk[1])
	suite.Equal("b", vk[2])

Filter/Reject

filter a map

	m := map[string]int{
		"a": 1,
		"b": 2,
	}
	var b []int
	KVStreamOf(m).Filter(func(k string, v int) bool {
		return v == 1
	}).Values().ToSlice(&b)
	suite.ElementsMatch(
		[]int{1},
		b,
	)

	KVStreamOf(m).Reject(func(k string, v int) bool {
		return v == 2
	}).Values().ToSlice(&b)
	suite.ElementsMatch(
		[]int{1},
		b,
	)

Contains

predict key exist

	m := map[string]int{
		"a": 1,
		"b": 2,
	}
	var b []int
	KVStreamOf(m).Contains("a")  // true

Keys/Values

get map keys/values stream

	slice := []int{1, 2, 3, 2, 1}
	out := StreamOf(slice).ToSet().Keys().Ints()
	suite.ElementsMatch([]int{1, 2, 3}, out)

Size

get map size

	slice := []int{1, 2, 3, 2, 1}
	out := StreamOf(slice).ToSet().Size()
	suite.Equal(3, out)

Run/To

kvstream is also lazy evaluation, get will get the result until Run/To invoked

	m := map[string]int{
		"a": 1,
		"b": 2,
	}
	var vk map[int]string
	KVStreamOf(m).Map(func(k string, v int) (int, string) {
		return v, k
	}).To(&vk)
	suite.Equal("a", vk[1])
	suite.Equal("b", vk[2])

Files

README.org

Latest commit

History

README.org

File metadata and controls

function programming experimental lib

why another fp lib

source

high order functions

Map

FlatMap

Filter

Reject

Foreach

Flatten

Partition/PartitionBy/LPartitionBy

Reduce/Reduce0

First

IsEmpty

Take/TakeWhile

Skip/SkipWhile

Sort/SortBy

Uniq/UniqBy

Size

Contains/ContainsBy

GroupBy

Append/Prepend

Union/Sub/Interact

Zip

ZipN

Result

Run

ToSlice

Monand

Error Monand

Maybe Monad

Transform to stream

Compose multiple Monand

Once Monand

Result

KVStream

Foreach

Map

Filter/Reject

Contains

Keys/Values

Size

Run/To