Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

StatsD TCP connection does not recover #1212

Closed
benzaita opened this issue Feb 5, 2019 · 2 comments · Fixed by #1782
Closed

StatsD TCP connection does not recover #1212

benzaita opened this issue Feb 5, 2019 · 2 comments · Fixed by #1782
Labels
enhancement A general enhancement registry: statsd A StatsD Registry related issue
Milestone
1.4.0

Comments

@benzaita
Copy link

benzaita commented Feb 5, 2019

Version 1.2.0

When the TCP connection (e.g. to Telegraf) drops the Netty TCP client does not recover it.

See this for a possible solution.
izeye added a commit to izeye/micrometer that referenced this issue Feb 26, 2019
@izeye
Restore TCP connection when disconnected in StatsD meter registry
13183a1 
Closes micrometer-metricsgh-1212
@izeye izeye mentioned this issue Feb 26, 2019
Closed
@izeye
Copy link
Contributor

izeye commented Feb 26, 2019

@benzaita Thanks for the report! I created #1251 to try to resolve this.

@shakuzen shakuzen added enhancement A general enhancement registry: statsd A StatsD Registry related issue labels Sep 6, 2019
@shakuzen shakuzen added this to the 1.3.0 milestone Sep 6, 2019
@shakuzen shakuzen modified the milestones: 1.3.0, 1.4.0 Sep 29, 2019
@shakuzen shakuzen mentioned this issue Oct 30, 2019
Closed
@shakuzen shakuzen mentioned this issue Jan 20, 2020
Merged
shakuzen added a commit that referenced this issue Jan 27, 2020
@shakuzen @izeye
Rewrite Statsd publishing implementation (#1782)
ae531f2 
Upgrades to the latest version of reactor-core and reactor-netty. Uses a `DirectProcessor` instead of a `UnicastProcessor` that can only be subscribed to once. We need to be able to subscribe multiple times for reconnecting to a server with a new `TcpClient`. For multi-threaded publishing to the `Processor`, we use a `FluxSink` rather than calling `onNext` directly. Starting and stopping the registry should now work as expected and is tested.

Unlike before, there is no buffering done in the `Processor`. If the UDP/TCP client or `lineSink` cannot keep up with the rate metrics are produced at, metric lines will be dropped rather than buffered indefinitely (potentially until the application runs out of memory). A mitigation when using the UDP/TCP client is to use the `BufferingFlux` which is enabled by default - it buffers metric lines in memory up to a configurable size/time before emitting to the UDP/TCP client. As a consequence, the queue size/capacity methods and metrics (`StatsdMetrics`) are no longer available.

Resolves #1212
Resolves #1591
Resolves #1676
Resolves #1741
Supersedes #1251

Co-authored-by: Johnny Lim <izeye@naver.com>
@shakuzen
Copy link
Member

@benzaita could you try with the latest 1.4.0-SNAPSHOT version to confirm this is working as expected now?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement A general enhancement registry: statsd A StatsD Registry related issue
Projects
None yet
Milestone
1.4.0
3 participants
@benzaita @izeye @shakuzen