Tags: alob-mtc/runnerq
Tags
feat: Postgres scale improvements — native scheduling & activity type… … filtering (#71) * Support native scheduled activities in backends Add schedules_natively() to QueueStorage and ActivityQueueTrait (default false) so a storage/backend can declare it handles scheduled/retrying activities directly in dequeue(). WorkerEngine now skips the separate scheduled-activities processor when the activity queue reports native scheduling. BackendQueueAdapter forwards schedules_natively() to the underlying backend. PostgreSQL backend updated: dequeue query now considers 'scheduled' and 'retrying' rows with due scheduled_at, orders by retry_count and COALESCE(scheduled_at, created_at), process_scheduled() becomes a no-op and schedules_natively() returns true, and indexes were replaced with a new index optimized for the adjusted dequeue logic. Also add a runner::metrics module declaration. * feat: worker-level activity type filtering and Postgres scale improvements - Add activity_types option to WorkerEngineBuilder for workload isolation - Postgres dequeue() now filters by activity_type when configured - Add activity_type to idx_runnerq_dequeue_effective expression index - QueueStorage::dequeue() accepts activity_types parameter - BackendQueueAdapter owns the filter and injects it into backend calls - Panic at start() if activity_types lists types without registered handlers - Promote PostgreSQL backend from experimental to stable - Add metrics constants module (src/runner/metrics.rs) - Add activity_type_filtering example - Update README and lib.rs docs with workload isolation section Closes #70 (PostgreSQL portions) * fix: correct misleading log tag in SmsHandler example
feat: implement age-weighted fair scheduling and fix observability en… …dpoints (#66) * feat: implement age-weighted fair scheduling and fix observability endpoints ## Changes ### Fair Scheduling (Redis & Postgres) - Implement age-weighted fair scheduling algorithm that balances: - Responsiveness: new activities get processed promptly - Fairness: retrying activities accumulate priority to avoid starvation - Redis: priority score now considers age boost (~1pt/sec) and retry boost (retry² × 10k) - Postgres: ORDER BY now includes retry_count DESC for fair ordering ### Observability Fixes - Fix /activities/scheduled endpoint: correctly parse 'uuid:json' format - Fix /activities/completed endpoint: filter to only return Completed/Failed status ## Algorithm Details The new priority calculation balances competing concerns: - Base priority weight (Critical > High > Normal > Low) - Age boost: activities waiting longer accumulate priority - Retry boost: exponential boost for activities that keep failing - FIFO ordering within same tier Combined with exponential backoff, this ensures retrying activities eventually complete while new activities still get processed. * updated version and fmt * fix: cap retry_boost at 500k to prevent priority overwhelming The retry boost was unbounded (retry² × 10k), which could allow high-retry items to dominate priority. Now capped at 500k (reached at ~7 retries) so retries still get exponential benefit but cannot overwhelm base priority weights (Critical=4M, High=3M, Normal=2M, Low=1M).
feat: Thread-safe atomic idempotency with OnDuplicate behaviors (#26) * feat: implement thread-safe atomic idempotency with OnDuplicate behaviors - Add atomic Redis SET NX EX operation for thread-safe idempotency key management - Implement OnDuplicate enum behaviors: AllowReuse, ReturnExisting, AllowReuseOnFailure, NoReuse - Move idempotency evaluation early in execute_activity for atomicity - Remove idempotency checks from enqueue() and schedule_activity() - Tie idempotency TTL to IDEMPOTENCY_KEY_TTL_SECONDS (24 hours) - Check activity status directly from snapshot instead of storing in idempotency record * updated * bump version * removed deadcode
PreviousNext