GraphiteReporter should reconnect when connection fails temporarily #694
Comments
|
Related code is below. Seems that close() on socket fails, and then the reconnect logic doesn't notice that reconnection should be made. GraphiteReporter.java: @Override
public void report(SortedMap<String, Gauge> gauges,
SortedMap<String, Counter> counters,
SortedMap<String, Histogram> histograms,
SortedMap<String, Meter> meters,
SortedMap<String, Timer> timers) {
final long timestamp = clock.getTime() / 1000;
// oh it'd be lovely to use Java 7 here
try {
if (!graphite.isConnected()) {
graphite.connect();
}
for (Map.Entry<String, Gauge> entry : gauges.entrySet()) {
reportGauge(entry.getKey(), entry.getValue(), timestamp);
}
for (Map.Entry<String, Counter> entry : counters.entrySet()) {
reportCounter(entry.getKey(), entry.getValue(), timestamp);
}
for (Map.Entry<String, Histogram> entry : histograms.entrySet()) {
reportHistogram(entry.getKey(), entry.getValue(), timestamp);
}
for (Map.Entry<String, Meter> entry : meters.entrySet()) {
reportMetered(entry.getKey(), entry.getValue(), timestamp);
}
for (Map.Entry<String, Timer> entry : timers.entrySet()) {
reportTimer(entry.getKey(), entry.getValue(), timestamp);
}
graphite.flush();
} catch (IOException e) {
LOGGER.warn("Unable to report to Graphite", graphite, e);
try {
graphite.close();
} catch (IOException e1) {
LOGGER.warn("Error closing Graphite", graphite, e);
}
}
}
`` |
|
I'm also experiencing this same error. |
|
I can verify that dropping back to 3.0.2 fixes this issue, so the bug appears to be around that |
|
True, hope to test this out sometime. On Fri, Oct 24, 2014 at 1:51 PM, Juha Syrjälä notifications@github.com
The fruit of silence is prayer; |
|
Indeed, this is the same issue we experienced and reported in #669. Until 3.1.1 is released a possible workaround is to use a modified Graphite class which contains the fix like we did. |
|
Closed as fixed in #669, and 3.1.1 will be out very soon. |
|
@ryantenney, any plans to release 3.1.1 ? I'd love to upgrade from 3.0 to 3.1, without the risk of loosing metric data :) |
|
+1 on a release of this fix. Is there any ETA? In the meantime, is it feasible to roll back to 3.0.2 as suggested by @velvia without losing important changes? I don't see a changelog anywhere. |
|
+1 on 3.1.1 release. Any timeline? This is causing us issues in production... |
|
+1. When it will be fixed??? Guys do this simple fix - make us happier |
|
@ryantenney |
|
@ryantenney Can you re-open this issue? As it is, I can not add dependencies to this project/version from mine. It is just broken for graphite use. 3.1.1 has not seen the light of day, and 4.0 may not be here for awhile yet. |
|
@jshook I just pushed 3.1.1 to Maven Central, check for it an hour or two. I'm sorry for the delay, a lot of things have been getting in the way. |
With Metrics 3.1.0 GraphiteReporter we had a problem where data was no longer sent to graphite server. This happened after some network component was rebooted. Same server is sending data also with Metrics 2.1.3 GraphiteReporter. Older graphite reporters kept on sending data just fine.
Sounds like this could be related to #607.
Stacktrace from 3.1.0:
The text was updated successfully, but these errors were encountered: