Add AppendCacheCheckpoint large dataset usage warning to documentation

[JRAlexander]

https://github.com/dotnet/machinelearning-samples/blob/bfcd66167bb4518afb538661a97ad98c27685349/samples/csharp/getting-started/MulticlassClassification_Iris/IrisClassification/IrisClassificationConsoleApp/Program.cs#L50 has a warning:
// Use in-memory cache for small/medium datasets to lower training time.
// Do NOT use it (remove .AppendCacheCheckpoint()) when handling very large datasets.

This should be added to the documentation.

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 2a03f5f8-1ce2-ae98-0f03-3a0cdfd2a208
• Version Independent ID: 58d012b4-4275-8c95-1634-08f7f66cbafb
• Content: EstimatorChain<TLastTransformer>.AppendCacheCheckpoint(IHostEnvironment) Method (Microsoft.ML.Data)
• Content Source: [dotnet/xml/Microsoft.ML.Data/EstimatorChain1.xml](https://github.com/dotnet/ml-api-docs/blob/live/dotnet/xml/Microsoft.ML.Data/EstimatorChain1.xml)
• Product: dotnet-ml-api
• GitHub Login: @sfilipi
• Microsoft Alias: johalex

[PeterPann23]

Is this feature working as expected now? or is this still an issue with very large datasets, and what size is considered a very large dataset?

[justinormont]

@PeterPann23

Is this feature working as expected now? or is this still an issue with very large datasets

There is no automatic caching in ML.NET. I'm unsure if there's docs written to advise on its usage. Super high level.. caching is quite beneficial until you run out of RAM**.

... and what size is considered a very large dataset?

"Very large" depends on a few specifics of the dataset, machine, and the caching location(s).

Assuming you're adding a cache just before the final trainer (#1 below), it's the featurized dataset size which is important. You want this to be below the size of your RAM. Beyond RAM, you'll either hit an OOM, or begin disk swapping.

Text columns get much larger post featurization, categorical columns grow some, and numeric data is generally similar to the input size in the file.

---

**Generally useful caching locations:

1. Before a trainer which does many passes (only current style in AutoML)
2. Before feature engineering which does many passes
3. After feature engineering which is slow on its apply phase
4. At dataset loading if the source is slow (e.g. SQL)

[PeterPann23]

it would be nice to allow specifying the location for cashing as was available in the beginning, one could use some fast storage if swapping becomes necessary.

In the comments it is mentioned not to use cashing on very large data-sets, perhaps one should/ could mention in the documentation/ code comments that

Considerations that when using cashing that the combined size of the data-set together with the cached data could cause an system to run out of memory which may result in an OutOfMemory exception

Perhaps that's better than the statement we have now stating something like not to use it on very large datasets

I am correct to assume that cashing helps speeding up training? In the beginning it was also used to fix a bug as I recall.

[justinormont]

Correct, caching is for gains in training speed.

I might recommend a warning along the lines of:

"AppendCacheCheckpoint() should not be used on datasets where the featurized dataset is larger than available RAM otherwise disk thrashing or an OutOfMemory exception may occur."

Or: (stating in a reactive form, pushing for users to try it then fail back)

"AppendCacheCheckpoint() should be disabled if disk thrashing or OutOfMemory exceptions are seen, which can occur on when the featured dataset is larger than available RAM"

I think I prefer the reactive form, as it's unlikely the user will be able to estimate the size of a featurized dataset.

I think the bug you're referencing is #2099 (as referenced by @JRAlexander in dotnet/docs#10526); that bug is still open.

@JRAlexander / @luisquintanilla : Do we have a write-up on where/when to use/not-use caching?