Code

Release/1.23.0#143

Code

Merged

catileptic merged 122 commits intomainalephdata/servicelayer:mainfrom release/1.23.0alephdata/servicelayer:release/1.23.0Copy head branch name to clipboardOct 8, 2024

+1,142-182

Commits

Commits on Sep 29, 2023

• Add priorities to tasks published using RabbitMQ

  catileptic

  committed
  556cd28

• Paint it black

  catileptic

  committed
  369f8b8

• Paint it black again

  catileptic

  committed
  a308782

Commits on Oct 2, 2023

• Add priority to Task body

  catileptic

  committed
  e53f8fb

Commits on Oct 10, 2023

• Temporary debug commit

  catileptic

  committed
  2042dc6

• Temporary debug commit

  catileptic

  committed
  0839455

Commits on Nov 8, 2023

• Fix logging message formatting

  catileptic

  committed
  82daf4e

Commits on Nov 20, 2023

• Make RabbitMQ qork without threads

  catileptic

  committed
  3d70937

• Fix linting errors

  catileptic

  committed
  f0622a3

• Fix tests for priority RabbitMQ implementation

  catileptic

  committed
  3ddfebe

• Add one thread

  catileptic

  committed
  655706b

• Add python3-setuptools

  catileptic

  committed
  59ae9db

• Update pip, setuptools and wheel before installing packages

  catileptic

  committed
  fca3d4f

Commits on Nov 21, 2023

• Install setuptools in Makefile, rm from Dockerfile

  catileptic

  committed
  f56e36a

Commits on Dec 14, 2023

• last_updated timestamp for the status API (RabbitMQ impl)

  stchris

  committed
  1301028

Commits on Dec 20, 2023

• Merge pull request #112 from alephdata/feature/rabbitmq-priorities

  Show description for 728dbfa
  stchris

  authored
  728dbfa

• Merge pull request #137 from alephdata/feature/last-updated-rabbitmq

  Show description for 0333891
  stchris

  authored
  0333891

Commits on Dec 21, 2023

• Add rc support to bumpversion cfg

  stchris

  committed
  a3f2fd0

• Bump version: 1.22.1 → 1.23.0-rc1

  stchris

  committed
  69dc59b

Commits on Jan 23, 2024

• Refactor the creation of the rabbitMQ connection to ensure it is always open (#148)

  Show description for e757cc1
  catileptic

  authored
  e757cc1

• Bump version: 1.23.0-rc1 → 1.23.0-rc2

  catileptic

  committed
  5229f3f

Commits on Feb 6, 2024

• Add Redis keys for tracking task stages

  catileptic

  committed
  f6756bf

• Fix incorrect call to pipe.incr

  catileptic

  committed
  378f8dd

• Update call to moto

  catileptic

  committed
  4b6a9ae

Commits on Feb 8, 2024

• Ensure status have a valid minimum value

  catileptic

  committed
  8184e3e

• Make job_id value an empty string in order to not break the schema validation

  catileptic

  committed
  15ebc8d

• Bump version: 1.23.0-rc2 → 1.23.0-rc3

  catileptic

  committed
  2cd1938

Commits on Mar 14, 2024

• Merge branch 'main' into release/1.23.0

  catileptic

  committed
  f0fc18a

Commits on Mar 18, 2024

• Fix call to moto to be compatible with versions <5

  catileptic

  committed
  569729f

• Remove jobs from status API

  catileptic

  committed
  cf2ccf1

Commits on Mar 21, 2024

• Bump version: 1.23.0-rc3 → 1.23.0-rc4

  catileptic

  committed
  863cacf

Commits on Apr 22, 2024

• Merge branch 'main' into release/1.23.0

  catileptic

  committed
  bdc70d5

Commits on Apr 23, 2024

• Will revert: limit queue size to 10 msgs & drop newer msgs first

  catileptic

  committed
  a34f38b

• Set queues to delete newer msgs if the queue fills up

  catileptic

  committed
  093b3fc

Commits on May 3, 2024

• Merge branch 'main' into release/1.23.0

  stchris

  committed
  d8d8213

• Bump version: 1.23.0-rc4 → 1.23.0-rc5

  stchris

  committed
  b7df50c

Commits on May 8, 2024

• Log worker hostname when dispatching task

  catileptic

  committed
  4de7e00

• Fix linter error

  catileptic

  committed
  e2fb94a

• Fix other linter

  catileptic

  committed
  d7a0d6f

• Bump version: 1.23.0-rc5 → 1.23.0-rc6

  catileptic

  committed
  838964c

Commits on May 14, 2024

• Catch pika.exceptions.StreamLostError

  catileptic

  committed
  cbf0ba6

• Fix linter error

  catileptic

  committed
  5f32bf1

• Bump version: 1.23.0-rc6 → 1.23.0-rc7

  catileptic

  committed
  2ba898e

• chore: friendlier rabbitmq connection names

  stchris

  committed
  0d6328f

• Bump version: 1.23.0-rc7 → 1.23.0-rc8

  stchris

  committed
  431fb72

Commits on May 22, 2024

• Remove periodic invocation

  Show description for 2b058fa
  stchris

  committed
  2b058fa

• Bump version: 1.23.0-rc8 → 1.23.0-rc9

  stchris

  committed
  41d70a4

• Allow workers to define queues and QoS settings dynamically (#171)

  catileptic

  authored
  7981c16

• Bump version: 1.23.0-rc9 → 1.23.0-rc10

  catileptic

  committed
  2e9247e

Commits on May 23, 2024

• Sanity check for connection establishment

  stchris

  committed
  5ef6c25

• Expose management UI port for RabbitMQ

  stchris

  committed
  968c366

• Always re-establish connection after a failed attempt

  stchris

  committed
  75f4665

• Sanity check connection is alive

  stchris

  committed
  29e1daa

• Use the newly assigned connection

  stchris

  committed
  b235092

• Add a note about the unrelated SQLAlchemy warning

  stchris

  committed
  8a57068

• Bump version: 1.23.0-rc10 → 1.23.0-rc11

  stchris

  committed
  671570e

Commits on May 24, 2024

• Don't log first failed rmq connectiona attempt

  stchris

  committed
  2946dc7

• Bump version: 1.23.0-rc11 → 1.23.0-rc12

  stchris

  committed
  acb734b

• Update ruff.toml settings

  stchris

  committed
  b2cfe45

• Fix formatting

  stchris

  committed
  7e4ad36

• Bump version: 1.23.0-rc12 → 1.23.0-rc13

  stchris

  committed
  5838b6d

• More accurate logging of attempt retries

  stchris

  committed
  614a28e

• Revert "Don't log first failed rmq connectiona attempt"

  Show description for 71812e1
  stchris

  committed
  71812e1

• Bump version: 1.23.0-rc13 → 1.23.0-rc14

  stchris

  committed
  ef5bbdb

• Run periodic tasks after handling tasks :/

  stchris

  committed
  fa51c23

• Bump version: 1.23.0-rc14 → 1.23.0-rc15

  stchris

  committed
  799c0b8

• Run periodic tasks before handling tasks

  stchris

  committed
  e2cd6fb

• Bump version: 1.23.0-rc15 → 1.23.0-rc16

  stchris

  committed
  03a9b0f

Commits on May 27, 2024

• Add a sentry tag for rmq conn attempt count

  stchris

  committed
  3c597d5

• Don't log an error for the first attempt

  stchris

  committed
  7179ea0

• Bump version: 1.23.0-rc16 → 1.23.0-rc17

  stchris

  committed
  8e0ef2b

• Revert "Add a sentry tag for rmq conn attempt count"

  Show description for 0bd2b41
  stchris

  committed
  0bd2b41

• Bump version: 1.23.0-rc17 → 1.23.0-rc18

  stchris

  committed
  35a7983

Commits on May 30, 2024

• Try 30 times to establish whether a task should be executed (#176)

  Show description for 79db6a5
  catileptic

  authored
  79db6a5

• Bump version: 1.23.0-rc18 → 1.23.0-rc19

  catileptic

  committed
  397acbb

Commits on Jun 5, 2024

• Enable Sentry in the RMQ Worker (#182)

  catileptic

  authored
  6dc7d4e

• Bump version: 1.23.0-rc19 → 1.23.0-rc20

  catileptic

  committed
  8c8e531

• Enable Prometheus metrics on the RabbitMQ Worker (#181)

  Show description for d6d6129
  catileptic

  authored
  d6d6129

• Modify the on_message function to be usable from AlephWorker

  catileptic

  committed
  8bd4082

• Bump version: 1.23.0-rc20 → 1.23.0-rc21

  catileptic

  committed
  8dcfab3

Commits on Jun 6, 2024

• call periodic tasks after handling tasks

  stchris

  committed
  2e826c4

• Bump version: 1.23.0-rc21 → 1.23.0-rc22

  stchris

  committed
  87ffe7e

Commits on Jun 12, 2024

• un-skip test for tasks which shouldn't execute

  stchris

  committed
  7192299

• remove unused import

  stchris

  committed
  551e742

Commits on Jun 18, 2024

• Properly NACK failed tasks

  stchris

  committed
  608b5f8

• Make sure we don't requeue tasks with excessive retries

  stchris

  committed
  695b569

• Bump version: 1.23.0-rc22 → 1.23.0-rc23

  stchris

  committed
  9eab35a

Commits on Jun 19, 2024

• Add a task to Redis if it is no longer tracked after NACKing

  

  catileptic

  authored and

  stchris

  committed
  c8fb3a6

• Removed unused code

  stchris

  committed
  53ef724

• Bump version: 1.23.0-rc23 → 1.23.0-rc24

  stchris

  committed
  f34fe01

• Hotfix: call add_task with correct parameters

  catileptic

  committed
  b7b1730

• Bump version: 1.23.0-rc24 → 1.23.0-rc25

  catileptic

  committed
  8fa15bc

Commits on Jun 25, 2024

• Generic queue_task implementation with priority buckets

  stchris

  committed
  21f1c8c

• Add flush_queues

  stchris

  committed
  386fa09

• Refactor queue_task to take a collection_id

  stchris

  committed
  c1b24f5

• Bump version: 1.23.0-rc25 → 1.23.0-rc26

  stchris

  committed
  981dff7

• Add high and low prio lists

  stchris

  committed
  46e29db

• Bump version: 1.23.0-rc26 → 1.23.0-rc27

  stchris

  committed
  2d279bb

Commits on Jun 28, 2024

• Fix rmq channel leak (#189)

  Show description for 2fe5598
  stchris

  authored
  2fe5598

• Bump version: 1.23.0-rc27 → 1.23.0-rc28

  stchris

  committed
  8b522b7

Commits on Jul 3, 2024

• Call confirm_delivery only once per channel

  stchris

  committed
  6ab363a

• Bump version: 1.23.0-rc28 → 1.23.0-rc29

  stchris

  committed
  e0f19ed

• Clear inactive dataset if the last task was executed

  catileptic

  committed
  ef315d0

• Fix formatting

  stchris

  committed
  264f878

Commits on Jul 4, 2024

• Do not add failed tasks to redis

  catileptic

  committed
  f12e13e

• Fix tests now that we clean up finished tasks

  stchris

  committed
  a92c592

• Bump version: 1.23.0-rc29 → 1.23.0-rc30

  stchris

  committed
  6200817

Commits on Jul 8, 2024

• Fix Prometheus metrics (#199)

  Show description for decef6d
  tillprochaska

  authored
  decef6d

• Bump version: 1.23.0-rc30 → 1.23.0-rc31

  stchris

  committed
  a7ef3aa

• Remove task from Redis if NACK-ing with retry=False

  catileptic

  committed
  8f69579

Commits on Jul 9, 2024

• Bump version: 1.23.0-rc31 → 1.23.0-rc32

  stchris

  committed
  3c1c690

Commits on Jul 19, 2024

• Refactor Redis keys in order to ensure status cleanup (#202)

  catileptic

  authored
  ef5027e

• Fix dataset timestamps (#190)

  Show description for eedfb94
  

  tillprochaska

  and

  catileptic

  authored
  eedfb94

• Bump version: 1.23.0-rc32 → 1.23.0-rc33

  catileptic

  committed
  995743f

Commits on Sep 3, 2024

• Don't show retry flag when a task succeeded

  stchris

  committed
  f97f086

• s/docker-compose/docker compose

  stchris

  committed
  8a4f717

Commits on Sep 11, 2024

• Add _channel attr in test runs, too

  catileptic

  committed
  85d8544

Commits on Sep 24, 2024

• Revert "Add _channel attr in test runs, too"

  Show description for 24c4b7f
  stchris

  committed
  24c4b7f

• Add _channel attr to task obj in tests

  catileptic

  committed
  273496b

• Bump version: 1.23.0-rc33 → 1.23.0-rc34

  catileptic

  committed
  83c62cb

Commits on Oct 8, 2024

• Fix Redis keys (#201)

  Show description for 59629f1
  catileptic

  authored
  59629f1

• Merge branch 'main' into release/1.23.0

  catileptic

  authored
  2c32068