Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Dec 8, 2025

Proposed changes

Thread pool starvation occurred when publishing to ~50 MQTT topics. Each WriterGroup spawned an IntervalRunner that blocked thread pool threads on a lock in MqttPubSubConnection.PublishNetworkMessage, then blocked again on GetAwaiter().GetResult().

Changes:

  • Converted PublishNetworkMessage to PublishNetworkMessageAsync (async Task<bool>) across all PubSub connections following .NET async naming conventions
  • Updated IntervalRunner to accept Func<Task> instead of Action, eliminating thread pool blocking
  • Removed all GetAwaiter().GetResult() patterns in publish paths
  • Updated test methods to properly await async operations
  • Applied "Async" suffix to all methods returning Task per .NET conventions

Key files:

  • IUaPubSubConnection.cs - Interface signature change to PublishNetworkMessageAsync
  • MqttPubSubConnection.cs - Removed blocking on async MQTT publish
  • UdpPubSubConnection.cs - Async signature for consistency
  • IntervalRunner.cs - Support async actions
  • UaPublisher.cs, MqttMetadataPublisher.cs, UdpDiscoverySubscriber.cs - Async publish methods
  • UdpDiscoveryPublisher.cs - Updated to call PublishNetworkMessageAsync

Related Issues

Addresses thread pool exhaustion reported in issue with ~50 writer groups.

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • Enhancement (non-breaking change which adds functionality)
  • Test enhancement (non-breaking change to increase test coverage)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected, requires version increase of Nuget packages)
  • Documentation Update (if none of the other choices apply)

Checklist

  • I have read the CONTRIBUTING doc.
  • I have signed the CLA.
  • I ran tests locally with my changes, all passed.
  • I fixed all failing tests in the CI pipelines.
  • I fixed all introduced issues with CodeQL and LGTM.
  • I have added tests that prove my fix is effective or that my feature works and increased code coverage.
  • I have added necessary documentation (if appropriate).
  • Any dependent changes have been merged and published in downstream modules.

Further comments

The async conversion follows .NET naming conventions with "Async" suffix on all methods returning Task. Callers can await the Task or continue calling synchronously. UDP transport also converted to async for consistency, though it doesn't suffer from the same blocking issue.

Original prompt

This section details on the original issue you should resolve

<issue_title>[PubSub] Thread pool exhaustion issues for many writer groups</issue_title>
<issue_description>### Type of issue

  • Bug
  • Enhancement
  • Compliance
  • Question
  • Help wanted

Current Behavior

I am running into thread pool starvation issues when publishing to a couple (~50) of MQTT topics.

Expected Behavior

Publishing to a larger amount of topics works without thread pool starvation / hiccups.

Steps To Reproduce

Here is a fork of the starter kit that reproduces the issue:
https://github.com/jnsjll/UA-IIoT-StarterKit

Environment

- OS: Windows 10
- Environment:
- Runtime: .NET 6
- Nuget Version:
- Component: PubSub
- Server:
- Client:

Anything else?

The problem seems to be that the IntervalRunner used for each WriterGroupDataType / UaPublisher calls the PublishMessages method on a thread pool thread which will block on a lock in MqttPubSubConnection.PublishNetworkMessage:
268619809-b9334de6-8c82-47db-abbb-f838f6a07d4d

Making the methods async and awaiting rather than blocking can resolve the issue.</issue_description>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copilot AI and others added 2 commits December 8, 2025 22:31
…exhaustion

Co-authored-by: romanett <7413710+romanett@users.noreply.github.com>
Co-authored-by: romanett <7413710+romanett@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix thread pool exhaustion issues for multiple writer groups Fix thread pool exhaustion with many MQTT writer groups Dec 8, 2025
Copilot AI requested a review from romanett December 8, 2025 22:42
Co-authored-by: marcschier <11168470+marcschier@users.noreply.github.com>
@marcschier marcschier marked this pull request as ready for review December 9, 2025 07:52
Copilot AI requested a review from marcschier December 9, 2025 07:52
@marcschier marcschier merged commit e032dae into master Dec 10, 2025
129 of 130 checks passed
@romanett romanett deleted the copilot/fix-thread-pool-exhaustion-issue branch December 20, 2025 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[PubSub] Thread pool exhaustion issues for many writer groups

4 participants