Skip to content

Dynamic task may not work as expected on BullMQ Driver #604

@caxerx

Description

@caxerx

Situation

I'm trying to add some delayed job to my Discord bot and I added task plugin for my bot.
I found that only the first job ever created will be executed. All the task created afterward was ignored.

Findings

I found that the problem may caused because of #584, it change the task creation from random uuid to a fixed jobId. The task will use the same jobId for each task runner.

However, according to BullMQ's documentation - JobIds
if you add a job with an existing id then that job will just be ignored and not added to the queue at all

Seems this PR is trying to add deduplication to tasks. Reference to BullMQ's documentation - Deduplication, to deduplicate the job, specifying jobId is not needed.

This change seems to used for solving an issue that Redis memory is using up due to the completed job.
However, with the document stated, The Simple Mode extends the deduplication duration until the job's completion or failure.
It may not work if the target of this change is solving the memory usage.

Moreover, it may break in some case when deduplication is used. For this example from CommandKit's documentation. This a reminder command and can be used by multiple user. It is possible to having multiple delayed task for same task runner at the same time. If deduplicate is enforced, only the one who run the task at last will be reminded.

Suggestion

If the purpose of #584 is to solve the memory issue of Redis, I would suggest using the Auto-removal option removeOnComplete and removeOnFail instead. It will remove the task from Redis after it run and free the memory up.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions