Skip to content

Refactor schema scanners #52356

Open
Open
@murphyatwork

Description

Refactoring request

Is your refactoring request related to a problem? Please describe.

SchemaScanner is the implementation at BE for the system tables in information_schema, it basically send an RPC request to FE to retrieve related information, then build the RPC result into a Chunk. But since each table can have different columns and individual rpc requests, the code is pretty verbose.

A good way to refactor these code is like be/src/exec/schema_scanner/schema_tasks_scanner.cpp:

  • return an DatumArray as a row of chunk
  • it doesn't need to handle the schema/slot anymore
  • in this way it's more extensible to add a column in the table
DatumArray SchemaTasksScanner::_build_row() {
    auto& task = _task_result.tasks.at(_task_index++);
    if (!task.__isset.catalog) {
        // Compatible for upgrades
        task.catalog = "default_catalog";
    }
    Datum expire_time = task.__isset.expire_time && task.expire_time > 0
                                ? TimestampValue::create_from_unixtime(task.expire_time, _runtime_state->timezone_obj())
                                : kNullDatum;
    Datum create_time = task.__isset.create_time && task.create_time > 0
                                ? TimestampValue::create_from_unixtime(task.create_time, _runtime_state->timezone_obj())
                                : kNullDatum;

    return {Slice(task.task_name),  create_time, Slice(task.schedule),   Slice(task.catalog), Slice(task.database),
            Slice(task.definition), expire_time, Slice(task.properties), Slice(task.creator)};
}

Since there're many SchemaScanner in the codebase, only few of them have been refactored. we'd like to refactor all of them in this way be/src/exec/schema_scanner/

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions