Make the number of column partitions dependent on the number of row groups at .read_parquet()

[dchigarev]

At the current implementation, modin's parquet reader tries to create as many column partitions as possible (even if all of them will consist of only 1 column), not considering the amount of row partitions being generated naturally by parquet's row groups.

modin/modin/core/io/column_stores/column_store_dispatcher.py

Line 146 in abe20a5

def build_columns(cls, columns):

This leads to that .read_parquet() naturally produces square-frames (#5296) that performs poorly in modin.

import tempfile
import pandas
import modin.pandas as pd

import modin.config as cfg

cfg.NPartitions.put(16)

NROWS = 100
NCOLS = 16

with tempfile.TemporaryDirectory() as file:
    [
        pandas.DataFrame({f"col{j}": range(NROWS) for j in range(NCOLS)}).to_parquet(
            f"{file}/{i}.parquet"
        )
        for i in range(16)
    ]

    df = pd.read_parquet(file)
    print(df._query_compiler._modin_frame._partitions.shape)  # (16, 16)

We may want to change the logic of generating column partitions so if there are already enough row parts then it will only generate column partitions in accordance with the cfg.MinPartitionSize parameter, and not in 1 column per 1 partition style.