Tracking issue of RFC-6297: Cache Layer

[koushiro]

proposal: #6297

[koushiro]

Some discussions can be found at #5678.

[koushiro]

BTW, if we adopt a design similar to the CacheLayer in the proposal , then I think foyer (#6366) should be implemented as a service rather than a layer.

[flaneur2020]

I recently begin to jump into this work to make it possible for SlateDB to adopt OpenDAL (which is mainly blocked by the lack of a cache layer (in slatedb/slatedb#1104)), as SlateDB's requirements mostly call for the cache to be chunked like Alluxio and JuiceFS to enable partial cached scans. 🤔

imho, i think it would be beneficial if the foyer layer and cache layer could develop in parallel, allowing us to iterate quickly in both directions and prioritize real use cases without getting blocked too long for reaching consensus on desinging abstractions. Later, we can make the foyer layer an alias of cache layer + foyer service after both become mature enough, and it should be a smooth transition for users.

I think it might be good to have a short zoom talk on to align on the direction we'd like to take. wdyt?

@koushiro @Xuanwo @MrCroxx

[Xuanwo]

Yes, I think we don't need to wait for the CacheLayer here. FoyerLayer can function as a separate feature. We can discuss how to integrate Foyer with the cache layer in the future.