Skip to content

Batch import doesn't apply rate limit when using async indexing #1542

@rlmanrique

Description

@rlmanrique

Using the Python client in its version 4.10.4

The function rate_limit is not working if Weaviate is configured in async indexing mode:

    with open(dataset_path, "r") as f:
        collection.batch.rate_limit(100)
        with collection.batch.dynamic() as batch:
            for raw_json_email in f:
                email = json.loads(raw_json_email)
                batch.add_object(
                    properties=email["data"],
                    uuid=email["id"],
                )

The batches have 1000 objects, no mater the rate configured.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions