Skip to content

This system should manage background jobs with worker processes, handle retries using exponential backoff, and maintain a Dead Letter Queue (DLQ) for permanently failed jobs

Notifications You must be signed in to change notification settings

adarsh2023calc/BackendAssignment

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

35 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

BackendAssignment

🧠 queuectl — CLI-based Background Job Queue System

queuectl is a lightweight, production-style background job queue system built in Python.
It supports enqueuing shell commands as jobs, processing them with worker processes, retrying failures using exponential backoff, and maintaining a Dead Letter Queue (DLQ) for permanently failed jobs.


✨ Features

  • 🧾 Job management — enqueue, list, and inspect jobs
  • ⚙️ Multiple workers — process jobs concurrently
  • 🔁 Automatic retries — exponential backoff on failure
  • 💀 Dead Letter Queue (DLQ) — permanently failed jobs are stored safely
  • 💾 Persistent storage — jobs stored in SQLite (queue.db)
  • 🧰 CLI interface — fully controllable from the terminal
  • Dashboard for viewing - Fully viewable dashboard

QueueCTL Logo


📦 Installation

git clone https://github.com/yourusername/queuectl.git
cd queuectl
chmod +x queuectl
sudo ln -s $(pwd)/queuectl /usr/local/bin/queuectl   # optional global usage

Job Format

{
  "id": "job-1",
  "command": "echo 'Hello World'",
  "state": "pending",
  "attempts": 0,
  "max_retries": 3,
  "created_at": "2025-11-04T10:30:00Z",
  "updated_at": "2025-11-04T10:30:00Z"
}

🚀 Usage

1️⃣ Enqueue a Job

./queuectl enqueue --json '{"id":"job3","command":"echo Hello","max_retries":2}'

QueueCTL Logo

or

./queuectl enqueue --command "ls-l"

QueueCTL Enqueue using command

Workers

./queuectl worker start --count 3

QueueCTL worker start

Status

    ./queuectl status 

QueueCTL job status

Pending

queuectl list --state running

QueueCTL list pending

DLQ

Showing the status of the DLQ

./queuectl dlq show

QueueCTL Showing  the status of the DLQ

Rescheduling the failed job

./queuectl dlq retry-job QueueCTL Rescheduling the failed job

Architecture Diagram

Architecture diagram

About

This system should manage background jobs with worker processes, handle retries using exponential backoff, and maintain a Dead Letter Queue (DLQ) for permanently failed jobs

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published