Sidekiq::Ultimate

Sidekiq ultimate experience.

---

WARNING

This ia an alpha/preview software. Lots of changes will be made and eventually it will overtake sidekiq-throttled and will become truly ultimate sidekiq extension one will need. :D

---

Installation

Add this line to your application's Gemfile:

gem "sidekiq-ultimate", ">= 0.0.1.alpha"

And then execute:

$ bundle

Or install it yourself as:

$ gem install sidekiq-ultimate

Usage

Add somewhere in your app's bootstrap (e.g. config/initializers/sidekiq.rb if you are using Rails):

require "sidekiq/ultimate"
Sidekiq::Ultimate.setup!

Configuration

Resurrection event handler

An event handler can be called when a job is resurrected.

Sidekiq::Ultimate.setup! do |config|
  config.on_resurrection = ->(queue_name, jobs_count) do
    puts "Resurrected #{jobs_count} jobs from #{queue_name}"
  end
end

Resurrection counter

A resurrection counter can be enabled to count how many times a job was resurrected. If enable_resurrection_counter setting is enabled, on each resurrection event, a counter is increased. Counter value is stored in redis and has expiration time 24 hours.

For example this can be used in the ServerMiddleware later on to early return resurrected jobs based on the counter value.

enable_resurrection_counter can be either a Proc or a constant.

Having a Proc is useful if you want to enable or disable resurrection counter in run time. It will be called on each resurrection event to decide whether to increase the counter or not.

Sidekiq::Ultimate.setup! do |config|
  config.enable_resurrection_counter = -> do
    DynamicSettings.get("enable_resurrection_counter")
  end
end

Sidekiq::Ultimate.setup! do |config|
  config.enable_resurrection_counter = true
end

Read the value

Resurrection counter value can be read using Sidekiq::Ultimate::Resurrector::Count.read method.

Sidekiq::Ultimate::Resurrector::Count.read(:job_id => "2647c4fe13acc692326bd4c2")
=> 1

Empty queues list refresh interval

Sidekiq::Ultimate.setup! do |config|
  config.empty_queues_refresh_interval = 42
end

It specifies how often the list of empty queues should be refreshed. In a nutshell, it specifies the maximum possible delay between a job was pushed to previously empty queue and the moment when that new job is picked up.

Note that every worker needs to maintain its own local list of empty queues. Setting this interval to a low values will increase the number of redis calls and will increase the load on redis.

---

NOTICE

Throttling is brought by sidekiq-throttled and it's automatically set up by the command above - don't run Sidekiq::Throttled.setup! yourself.

Thus look up it's README for throttling configuration details.

---

Supported Ruby Versions

This library aims to support and is tested against the following Ruby and Redis client versions:

• Ruby

  • 2.7.x

• redis-rb

  • 4.x

• redis-namespace

  • 1.6

If something doesn't work on one of these versions, it's a bug.

This library may inadvertently work (or seem to work) on other Ruby versions, however support will only be provided for the versions listed above.

If you would like this library to support another Ruby version or implementation, you may volunteer to be a maintainer. Being a maintainer entails making sure all tests run and pass on that implementation. When something breaks on your implementation, you will be responsible for providing patches in a timely fashion. If critical issues for a particular implementation exist at the time of a major release, support for that Ruby version may be dropped.

Development

After checking out the repo, run bundle install to install dependencies. Then, run bundle exec rake spec to run the tests with ruby-rb client.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Contributing

• Fork sidekiq-ultimate on GitHub
• Make your changes
• Ensure all tests pass (bundle exec rake)
• Send a pull request
• If we like them we'll merge them
• If we've accepted a patch, feel free to ask for commit access!

Copyright

Copyright (c) 2018-23 SensorTower Inc.
See LICENSE.md for further details.