Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do simple logging for failed export. #1480

Merged
merged 3 commits into from
Jul 31, 2020

Conversation

carlosalberto
Copy link
Contributor

We already log other situations (such as failing to do a clean shutdown), so it makes sense to add logging for failed export.

@iNikem
Copy link
Contributor

iNikem commented Jul 29, 2020

Will that solve #1480 ?

@carlosalberto
Copy link
Contributor Author

@iNikem You referenced this same PR ;)

@iNikem
Copy link
Contributor

iNikem commented Jul 29, 2020

Bah! I mean #1418

@carlosalberto
Copy link
Contributor Author

I think this should fix #1418 yes, mostly because we are already logging the other important failure case (failure to properly shutdown).

The rest of the errors do not seem unrecoverable (they are all parameters-handling related), so I think this simple PR is enough.

Copy link
Contributor

@anuraaga anuraaga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of logging in each exporter how about checking the result code in the processor? We already log for throwable there, this seems to be very similar logging.

logger.log(Level.WARNING, "Exception thrown by the export.", t);

@@ -126,6 +126,7 @@ public ResultCode export(Collection<SpanData> spans) {
stub.postSpans(request);
return ResultCode.SUCCESS;
} catch (Throwable e) {
logger.log(Level.SEVERE, "Failed to export spans", e);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't seem to use SEVERE anywhere else yet - spans can fail to export for transient reasons (backend is restarting), and observability is not critical to the app anyways, so I'd go with WARNING if not INFO.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please at least WARN :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARN works for me. Will update.

@iNikem
Copy link
Contributor

iNikem commented Jul 30, 2020

@anuraaga Taking into account open-telemetry/opentelemetry-specification#716, I would say logging in each exporter makes more sense :)

@anuraaga
Copy link
Contributor

@iNikem Well I'm definitely ok with removing resultcode instead :) But it seems like one or the other makes sense, either exporter is left to their own ways or are controlled by the span processor.

@carlosalberto
Copy link
Contributor Author

On top of the possibility of having the export result code being removed, there's the case of using a MultiSpanProcessor and having a specific backend fail (imagine that Jaeger exported its data fine, but Zipkin didn't). Also, in each processor's export step you have access to the actual Exception, which could be useful.

@anuraaga
Copy link
Contributor

MultiSpanProcessor is also a SpanProcessor, if we think that's the right layer to log exporter failures it could take care of it too (sorry wasn't clear with my reference, I linked to batch but simple, multi would also probably have a similar change if going with it).

Also, in each processor's export step you have access to the actual Exception, which could be useful.

The exception could be propagated up too - it seems weird to return a failure status without the reason. Shouldn't we either leave everything to the exporter, or otherwise have it propagate everything? I personally think either is fine, but a mix seems strange, it's hard to know who has responsibility when it comes to exporting spans.

@carlosalberto
Copy link
Contributor Author

Shouldn't we either leave everything to the exporter

Going forward, we should 'normalize' things with Exception/Exporter Code being returned or being totally gone. For the time being, I suggest we keep these warnings and tune this part once we decide how to proceed (most likely after we come to an agreement on whether we want #716 or not).

@codecov
Copy link

codecov bot commented Jul 30, 2020

Codecov Report

Merging #1480 into master will increase coverage by 0.05%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #1480      +/-   ##
============================================
+ Coverage     92.50%   92.56%   +0.05%     
  Complexity      939      939              
============================================
  Files           117      117              
  Lines          3348     3348              
  Branches        270      270              
============================================
+ Hits           3097     3099       +2     
+ Misses          169      168       -1     
+ Partials         82       81       -1     
Impacted Files Coverage Δ Complexity Δ
...telemetry/sdk/trace/export/BatchSpanProcessor.java 95.94% <0.00%> (+1.35%) 8.00% <0.00%> (ø%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 24ab6c3...bea929d. Read the comment docs.

@jkwatson jkwatson merged commit 199df3a into open-telemetry:master Jul 31, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants