Skip to content

Remove the schema config from flags and use only config file #2208

Closed
@gouthamve

Description

@gouthamve

Right now for setting the schema config, there are a ton of flags like:

dynamodb.daily-buckets-from, dynamodb.base64-buckets-from, dynamodb.v4-schema-from,....... dynamodb.use-periodic-tables, dynamodb.periodic-table.from, etc.

This is confusing and can be better served by the config file which is cleaner and clearer:

      - from: "2019-02-13"
        chunks:
            period: 168h
            prefix: dev_chunks_
        index:
            period: 168h
            prefix: dev_index_
        object_store: gcs
        schema: v9
        store: gcp-columnkey
      - from: "2019-02-24"
        chunks:
            period: 168h
            prefix: dev_chunks_
        index:
            period: 168h
            prefix: dev_index_
        object_store: gcs
        schema: v9
        store: bigtable-hashed
      - from: "2019-03-05"
        chunks:
            period: 168h
            prefix: dev_chunks_
        index:
            period: 168h
            prefix: dev_index_
        object_store: gcs
        schema: v10
        store: bigtable-hashed

Further, you can set more things via the config file than flags. I initially set out to rename the dynamodb prefixes, but then realised its better to totally remove these flags and only allow it to be set via config file.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions