Skip to content

Comments

Fix LISTEN/NOTIFY connection self-healing#420

Merged
azygis merged 2 commits intohangfire-postgres:masterfrom
nicolaj-hartmann:fix/listen-connection-self-healing
Feb 7, 2026
Merged

Fix LISTEN/NOTIFY connection self-healing#420
azygis merged 2 commits intohangfire-postgres:masterfrom
nicolaj-hartmann:fix/listen-connection-self-healing

Conversation

@nicolaj-hartmann
Copy link
Contributor

Summary

When EnableLongPolling is enabled, the LISTEN connection could die from network issues without recovery, causing workers to stop processing jobs while the server heartbeat continued.

This adds automatic reconnection when the LISTEN connection fails.

Changes

  • Added inner try-catch in LISTEN loop to handle connection failures
  • Reconnects automatically after 1 second backoff
  • Swallows cleanup exceptions in Dequeue finally block

Test

Added test that kills LISTEN connection via pg_terminate_backend and verifies worker continues processing.

Fixes #410

When EnableLongPolling is enabled, the ListenForNotificationsAsync method
only caught TaskCanceledException, causing all other exceptions (network
timeouts, connection drops) to propagate up and kill the worker thread.

This change adds self-healing reconnection logic that catches connection
failures, reconnects after a 1 second backoff, and continues the loop.

Fixes hangfire-postgres#410
@nicolaj-hartmann nicolaj-hartmann force-pushed the fix/listen-connection-self-healing branch from c62568d to ca736ae Compare December 29, 2025 15:20
@nicolaj-hartmann
Copy link
Contributor Author

@azygis ? :)

@nicolaj-hartmann
Copy link
Contributor Author

@frankhommers ?:)

@azygis
Copy link
Collaborator

azygis commented Feb 5, 2026

Will try to merge and publish this week.

@azygis azygis closed this Feb 7, 2026
@azygis azygis reopened this Feb 7, 2026
@azygis azygis merged commit 31e2c8e into hangfire-postgres:master Feb 7, 2026
2 checks passed
github-merge-queue bot pushed a commit to DFE-Digital/teaching-record-system that referenced this pull request Feb 24, 2026
Pinned [Hangfire.AspNetCore](https://github.com/HangfireIO/Hangfire) at
1.8.23.

<details>
<summary>Release notes</summary>

_Sourced from [Hangfire.AspNetCore's
releases](https://github.com/HangfireIO/Hangfire/releases)._

## 1.8.23

### Release Notes

#### Hangfire.Core

* **Changed** – Use stable sorting algorithm for background job filters
again (by @​jirikanda).
* **Fixed** – Custom `AutomaticRetryAttribute` is ignored under certain
conditions regression from 1.8.14 (by @​jirikanda).
* **Fixed** – Add missing keys for Swedish translation (by
@​karl-sjogren).
* **Project** – Use `TypeNameAssemblyFormatHandling` in tests with .NET
6 (by @​viktor-vintertass).

#### Hangfire.AspNetCore

* **Fixed** – `InvalidOperationException`: The request reached the end
of the pipeline without executing the endpoint.

Commits viewable in [compare
view](HangfireIO/Hangfire@v1.8.22...v1.8.23).
</details>

Pinned [Hangfire.Core](https://github.com/HangfireIO/Hangfire) at
1.8.23.

<details>
<summary>Release notes</summary>

_Sourced from [Hangfire.Core's
releases](https://github.com/HangfireIO/Hangfire/releases)._

## 1.8.23

### Release Notes

#### Hangfire.Core

* **Changed** – Use stable sorting algorithm for background job filters
again (by @​jirikanda).
* **Fixed** – Custom `AutomaticRetryAttribute` is ignored under certain
conditions regression from 1.8.14 (by @​jirikanda).
* **Fixed** – Add missing keys for Swedish translation (by
@​karl-sjogren).
* **Project** – Use `TypeNameAssemblyFormatHandling` in tests with .NET
6 (by @​viktor-vintertass).

#### Hangfire.AspNetCore

* **Fixed** – `InvalidOperationException`: The request reached the end
of the pipeline without executing the endpoint.

Commits viewable in [compare
view](HangfireIO/Hangfire@v1.8.22...v1.8.23).
</details>

Pinned
[Hangfire.PostgreSql](https://github.com/frankhommers/Hangfire.PostgreSql)
at 1.21.1.

<details>
<summary>Release notes</summary>

_Sourced from [Hangfire.PostgreSql's
releases](https://github.com/frankhommers/Hangfire.PostgreSql/releases)._

## 1.21.1

## What's Changed
* Use PrivateAssets for Dapper.Aot package reference by @​latop2604 in
hangfire-postgres/Hangfire.PostgreSql#422


**Full Changelog**:
hangfire-postgres/Hangfire.PostgreSql@1.21.0...1.21.1

## 1.21.0

## What's Changed
* Fix duplicate key error in FetchedJobs when job has multiple queue
entries by @​nicolaj-hartmann in
hangfire-postgres/Hangfire.PostgreSql#419
* Fix LISTEN/NOTIFY connection self-healing by @​nicolaj-hartmann in
hangfire-postgres/Hangfire.PostgreSql#420
* Dapper.AOT by @​latop2604 in
hangfire-postgres/Hangfire.PostgreSql#417

## New Contributors
* @​nicolaj-hartmann made their first contribution in
hangfire-postgres/Hangfire.PostgreSql#419
* @​latop2604 made their first contribution in
hangfire-postgres/Hangfire.PostgreSql#417

**Full Changelog**:
hangfire-postgres/Hangfire.PostgreSql@1.20.13...1.21.0

Commits viewable in [compare
view](hangfire-postgres/Hangfire.PostgreSql@1.20.13...1.21.1).
</details>

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: James Gunn <james@gunn.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Jobs stuck in Enqueued state when Postgres Listener connection is disrupted using EnableLongPolling

2 participants