Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the random trace id flag #474

Merged
merged 13 commits into from
Apr 13, 2022
Merged

Conversation

dyladan
Copy link
Member

@dyladan dyladan commented Nov 9, 2021

Resolves #467

As discussed in the previous working group meeting, this PR adds a trace flag to signal to downstream services that part of the trace id is random. Randomness in the trace ID is desirable because it may be used for things like consistent trace sampling, sharding, and more.

/cc @bogdandrutu author of #467
/cc @jmacd @oertl


Preview | Diff


Preview | Diff

@yurishkuro
Copy link
Member

If this resolves #467, please state this in the description.

Does this not require a bump to the header's version number?

spec/20-http_request_header_format.md Outdated Show resolved Hide resolved
spec/20-http_request_header_format.md Outdated Show resolved Hide resolved
@bogdandrutu
Copy link
Contributor

Does this not require a bump to the header's version number?

No, since this adds just a flag to the trace-flags and is backwards compatible.

@dyladan
Copy link
Member Author

dyladan commented Nov 9, 2021

Does this not require a bump to the header's version number?

No, since this adds just a flag to the trace-flags and is backwards compatible.

I agree with Bogdan here.

@dyladan dyladan force-pushed the random-flag branch 2 times, most recently from ed11809 to 65b2552 Compare November 9, 2021 20:00
spec/60-trace-id-format.md Outdated Show resolved Hide resolved
Co-authored-by: Yuri Shkuro <yurishkuro@users.noreply.github.com>
Copy link
Contributor

@basti1302 basti1302 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I left one nitpick/phrasing suggestion.

(And I assume the two open TODOs need to be decided and removed before merging).

spec/20-http_request_header_format.md Outdated Show resolved Hide resolved
@yurishkuro
Copy link
Member

Note that while this resolves #467, it does not address #463 because it makes the operator choose between using externally generated ids (sometimes needed for legacy/compatibility reasons) and having randomness in the context.

@dyladan
Copy link
Member Author

dyladan commented Nov 16, 2021

It addresses part of #463 which is that the trace id can be used to ensure sampling is consistent between services. It does not address the part about propagating the probability itself which is intentional.

@yurishkuro
Copy link
Member

It addresses part of #463 which is that the trace id can be used to ensure sampling is consistent between services.

It "addresses #463" only by introducing another limitation.

@dyladan
Copy link
Member Author

dyladan commented Nov 16, 2021

I'm afraid we might be talking past each other. I'm not entirely sure I understand what you're arguing.

@yurishkuro
Copy link
Member

Sorry, let me try again. #463 presents a high level problem - we want to have access to a single random number across all nodes in the trace. This PR says you can have it but ONLY if your trace-id is random(-ish). So this PR only solves the high level problem for a subset of systems that can also meet the additional restriction of random IDs. In contrast, the proposal in #463 solves the problem for everyone.

@dyladan
Copy link
Member Author

dyladan commented Nov 16, 2021

Sorry, let me try again. #463 presents a high level problem - we want to have access to a single random number across all nodes in the trace. This PR says you can have it but ONLY if your trace-id is random(-ish). So this PR only solves the high level problem for a subset of systems that can also meet the additional restriction of random IDs. In contrast, the proposal in #463 solves the problem for everyone.

OK I see what you're saying, but in order to implement #463 you would still need a random number at the head of the trace. If the restriction is because of the random requirement, why does it not affect the other proposal? Sorry if this is obvious I just want to make sure I fully understand.

@yurishkuro
Copy link
Member

I don't think the need to generate a random number is a blocker, having a random trace-id is. There are situations where traces must reuse a 3rd-party identifier as trace-id on which no randomness guarantees can be made (same reason why traceparent spec does not require random trace-id, we only use SHOULD).

@dyladan
Copy link
Member Author

dyladan commented Nov 16, 2021

Thanks for the explanation. Would you rather have the proposal in #463 (or something like it)? We discussed it at a meeting and even started making a draft prototype #468 but some people thought it was too breaking of a change.

edit: it's obviously possible to have both, but with #463/#468 there is less need for this

@yurishkuro
Copy link
Member

I am not opposed to the new bit proposal, I like it, we just need to be aware that it does not solve the problem for everyone.

@dyladan
Copy link
Member Author

dyladan commented Apr 12, 2022

I have added the SHOULD wording and some wording about why users should be motivated to use the flag. Please take a look and review when you can.

spec/20-http_request_header_format.md Show resolved Hide resolved
spec/20-http_request_header_format.md Outdated Show resolved Hide resolved
spec/20-http_request_header_format.md Outdated Show resolved Hide resolved
spec/60-trace-id-format.md Outdated Show resolved Hide resolved
@@ -116,6 +120,8 @@ Vendors MUST ignore the `traceparent` when the `parent-id` is invalid (for examp

#### trace-flags

The current version of this specification (`00`) supports only two flags: `sampled` and `random-trace-id`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we not need a new version if we're adding a new flag?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is backwards compatible so it is not required. We can decide to bump the version if we feel it is different enough.

Copy link
Member

@SergeyKanzhelev SergeyKanzhelev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few suggestions added

@kalyanaj kalyanaj merged commit cd1c099 into w3c:main Apr 13, 2022
dyladan added a commit to dynatrace-oss-contrib/trace-context that referenced this pull request Apr 26, 2022
* Add the random trace id flag

* Wording

* Wording

Co-authored-by: Yuri Shkuro <yurishkuro@users.noreply.github.com>

* Add SHOULD wording to trace id randomness

* Specify that implementers SHOULD set random flag when appropriate

* Random flag means at least 7 bytes

* Flag wording

* Only 2 flags are specified

* Remove redundant wording

* At least 7 bytes

* Review comments

* Remove RECOMMENDED wording

Co-authored-by: Yuri Shkuro <yurishkuro@users.noreply.github.com>
kalyanaj pushed a commit that referenced this pull request Apr 29, 2022
* Add the random trace id flag

* Wording

* Wording

Co-authored-by: Yuri Shkuro <yurishkuro@users.noreply.github.com>

* Add SHOULD wording to trace id randomness

* Specify that implementers SHOULD set random flag when appropriate

* Random flag means at least 7 bytes

* Flag wording

* Only 2 flags are specified

* Remove redundant wording

* At least 7 bytes

* Review comments

* Remove RECOMMENDED wording

Co-authored-by: Yuri Shkuro <yurishkuro@users.noreply.github.com>

Co-authored-by: Yuri Shkuro <yurishkuro@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Randomness flag bit
6 participants