Feature Request: Support Sparklyr

[harryprince]

Hi tidyverts team:
tsibble looks so great in handling time series data. In production, most of the common feature engineering is windowing, such as the sliding window, fixed window and so on.

I am manually make sliding window by pretty complicated Spark SQL instead of tidyverts style, because tsibble currently not support spark like dplyr.

Wish to discuss more tsibble on sparklyr in production.

The sparklyr struct streaming is also a good candidate to integrate with tsibble.

[earowang]

I'd actually like tsibble to support SQL databases #49 .

But the tsibble's APIs will take a while to get stable. I'd probably wait to implement other heavy features until the maturity of tsibble. It will be developed in a new package because tsibble is relatively a heavy package.

[harryprince]

@earowang I totally agree with you, wish to start the new package development in the near future.

When it comes to real-time time series problem, spark streaming is a candidate framework that we need to notice, and wish it could be a reference when we start next construction. Here are the common windowing models that we need to implement.

• fiexed window
• sliding window
• session window
• decay
• last value window

[ellisvalentiner]

@harryprince you might also be interested in the Flint issue over on the sparklyr repo. I haven't worked with Flint yet but it looks promising. Perhaps there is even a way of integrating tsibble with sparklyr/Flint down the road.

[earowang]

Closing this, since the window functions are deprecated in favour of the {slider} package.