Parallel POD

[shanemcq18]

Design paradigm for parallel POD (distributed OpInf):

• New ParallelPODBasis class (too different from serial POD to be in the same class)
• Constructor arguments (hyperparameters) for
  • specifying the number of processes and possibly the splitting scheme (num_processes=6). This means we have parallelism in the back, not MPI in the front.
  • criteria for the number of basis vectors to retain, probably num_vectors, cumulative_energy, svdval_threshold, and residual_energy, but not projection_error
  • whether to construct the actual basis vectors in the matrix V (record=False)
• Attribute for the actual basis vectors, None if record=False, an array of size $n_i \times r$ otherwise.
• Consider disabling fit() and compress(), force the user to use fit_compress()? Need to check with the ROM class to see if that messes something up.
• MUST have the data in a format that allows for distributed reading without loading the entire dataset. That means we either need to have a filename that has the entire dataset in a certain HDF5 format, or (probably more practical) a list of files to load, one for each process. Maybe we use this list instead of num_processes.

This last point is possibly the trickiest and may have consequences for the design of the rest of the package.

Ionut will take the lead.

[shanemcq18]

Need a utility for loading data, as well as instructions for valid formats for the data.

# utils._data.py

def load_data(
    filename: str | list,
    num_blocks: int | list = 1,
    block_index: int | list = 0,
):
    """Load a dataset or a portion of one dataset.

    """
    # ...

Need to think about the best way to do this for the whole preprocessing paradigm, including lifting, scaling, and compression.