Documentation of all local possibilities to shuffle and prefetch

[LukasDrude]

The current version is a bit opaque in the sense that I do not know how to use .shuffle() in combination with .prefetch() or .map(..., num_workers=10).

Please provide some documentation of best practices.

[boeddeker]

In general prefer .prefetch() over .map(..., num_workers=10), because it executes the complete input processing in parallel and not only one mapping function.

Since #2 is now merged you can .shuffle() in the beginning and .prefetch() at the end.

Example

ds = lazy_dataset.new(examples)
ds = ds.shuffle(reshuffle=True)
ds = ds.map(mapping_function)
ds = ds.batch(batch_size=4)
ds = ds.map(collate_function)
ds = ds.prefetch(num_workers=4, buffer_size=8)