Skip to content

Add a Redis based worker to the tasking system #7210

@dkliban

Description

@dkliban

Is your feature request related to a problem? Please describe.
We would like to be able to run 500+ workers at the same time. The current worker implementation limits us to about 150 before the database starts slowing down.

Describe the solution you'd like
Use Redis to store resource locks. Stop running the unblocking algorithm.

The goal is to introduce a WORKER_TYPE setting which will default to pulpcore (current implementation). Users will be able to select the new worker type by selecting redis.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions