Windowed operations (rolling average, etc.)

[cvb941]

Hello,

Kotlin offers the windowed function over iterables to create sub-lists of items along the data. This function can be used to calculate various metrics, such as moving averages and more.

Is this use case somehow supported in dataframe right now? Is it considered? It would be great to have a pretty way (even less tricky than using windowed), to quickly calculate moving averages, or any other more advanced functions which use rolling windows.

[Jolanrensen]

Might be a nice addition! However, it's a tiny bit difficult to implement since it can only work on áll columns (as the column gets shorter after a windowed function), unless we make it nullable and fill it in with nulls...
@koperagen what do you think?

Current workaround (with nulls):

val a by columnOf(*Array(10) { Random.nextDouble() })
val b by columnOf(*Array(10) { Random.nextDouble() })
val c by columnOf(*Array(10) { Random.nextDouble() })
val df = dataFrameOf(a, b, c)
df.print(title = true, borders = true, columnTypes = true)
//    ⌌--------------------------------⌍
//    |  | a:Double| b:Double| c:Double|
//    |--|---------|---------|---------|
//    | 0| 0.749965| 0.169327| 0.486369|
//    | 1| 0.636480| 0.293745| 0.872698|
//    | 2| 0.027803| 0.786161| 0.779551|
//    | 3| 0.594421| 0.589128| 0.249832|
//    | 4| 0.350703| 0.445626| 0.160247|
//    | 5| 0.382384| 0.136446| 0.805161|
//    | 6| 0.846572| 0.352280| 0.977022|
//    | 7| 0.587858| 0.021129| 0.357274|
//    | 8| 0.015700| 0.972652| 0.059243|
//    | 9| 0.610003| 0.223712| 0.793011|
//    ⌎--------------------------------⌏

val windowSize = 4
val dfRollingAverage = df.replace { a and b }.with {
    it.values
        .windowed(windowSize) { it.average() }
        .plus(List(windowSize - 1) { null })
        .toValueColumn(it.name())
}
dfRollingAverage.print(title = true, borders = true, columnTypes = true)
//    ⌌----------------------------------⌍
//    |  | a:Double?| b:Double?| c:Double|
//    |--|----------|----------|---------|
//    | 0|  0.502167|  0.459590| 0.486369|
//    | 1|  0.402352|  0.528665| 0.872698|
//    | 2|  0.338828|  0.489340| 0.779551|
//    | 3|  0.543520|  0.380870| 0.249832|
//    | 4|  0.541879|  0.238871| 0.160247|
//    | 5|  0.458128|  0.370627| 0.805161|
//    | 6|  0.515033|  0.392443| 0.977022|
//    | 7|      null|      null| 0.357274|
//    | 8|      null|      null| 0.059243|
//    | 9|      null|      null| 0.793011|
//    ⌎----------------------------------⌏

[koperagen]

I can say that we have a function relative
Look at the task number 30 here https://github.com/Kotlin/dataframe/blob/master/examples/jupyter-notebooks/puzzles/40%20puzzles.ipynb