[upstream] Initial framework for background item eviction/promotion

[igchor]

Implement stubs for periodic workers for item eviction/promotion. Actual implementation of eviction and promotion will be provided in the next patch.

The idea behing introducing those workers is the following:

• BG eviction: to keep certain amount of free memory and decreasing allocate latency
• BG promotion: to move hot items to memory decresing latency

---

This change is 

[igchor]

If and when this PR is merged into upstream we can followup with PRs that:

• implement params (watermarks, etc.) for BG workers and config support
• implement calculateBatchSize()
• implement eviction for single tier (to NVMe) + optionally promotion from NVMe to DRAM

[guptask]

cachelib/allocator/CacheAllocator.h line 1063 at r1 (raw file):

                      util::Throttler::Config reaperThrottleConfig);

  bool startNewBackgroundPromoter(

Is it better to set default values for interval and strategy ? I understand doing so will require a background thread attributes class since multiple default values won't be possible.
e.g. bool startNewBackgroundEvictor(BackgroundThreadOpts opts);

[guptask]

cachelib/allocator/CacheAllocatorConfig.h line 480 at r1 (raw file):

  std::shared_ptr<BackgroundMoverStrategy> backgroundPromoterStrategy;
  // time interval to sleep between runs of the background evictor
  std::chrono::milliseconds backgroundEvictorInterval{

I see now that default values are set in config. So maybe you can ignore my previous comment about background evictor/promoter thread creation api carrying default values for interval and strategy.

[guptask]

cachelib/allocator/CacheStats.h line 293 at r1 (raw file):

// Mover Stats
struct BackgroundMoverStats {

Should this class be renamed to BackgroundMoverLocalStats ? I think adding the term "local" adds clarity to the per-thread stats collection mode.

[guptask]

Overall the PR looks good. Please review the few cosmetic comments.

Reviewed 14 of 14 files at r1, all commit messages.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @igchor)

[igchor]

Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @guptask)

---

cachelib/allocator/CacheAllocator.h line 1063 at r1 (raw file):

Previously, guptask (Sounak Gupta) wrote…

Is it better to set default values for interval and strategy ? I understand doing so will require a background thread attributes class since multiple default values won't be possible.
e.g. bool startNewBackgroundEvictor(BackgroundThreadOpts opts);

We don't have too many params, so I think it's fine as it is for now. startNewPoolOptimizer and others also don't have default params.

---

cachelib/allocator/CacheAllocatorConfig.h line 480 at r1 (raw file):

Previously, guptask (Sounak Gupta) wrote…

I see now that default values are set in config. So maybe you can ignore my previous comment about background evictor/promoter thread creation api carrying default values for interval and strategy.

Yeah, I think it's probably better to have the default values here and perhaps in enableBackgroundEvictor function as well eventually.

---

cachelib/allocator/CacheStats.h line 293 at r1 (raw file):

Previously, guptask (Sounak Gupta) wrote…

Should this class be renamed to BackgroundMoverLocalStats ? I think adding the term "local" adds clarity to the per-thread stats collection mode.

Well, we're using this class to aggregate stats from all Bg workers at the end so I'm not sure. This is done inside getBackgroundMoverStats