-
Notifications
You must be signed in to change notification settings - Fork 105
feat: Add an API to send multiple samples at once #235
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
vickenty
merged 5 commits into
DataDog:master
from
blemale:bastien.lemale/feat_send_multiple_samples
Feb 13, 2024
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
bd9d85f
feat: Add an API to send multiple samples at once
blemale 55ebdc5
feat: Handle gracefully samples overflow
blemale d14c446
fix: Handle non-ascii character in metadata in multi valued messages
blemale 82eea75
test: Use a different port for statsd server
blemale d4f3cf8
fix: Correct splitting over more than 2 messages
blemale File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
package com.timgroup.statsd; | ||
|
||
/** | ||
* DirectStatsDClient is an experimental extension of {@link StatsDClient} that allows for direct access to some | ||
* dogstatsd features. | ||
* | ||
* <p>It is not recommended to use this client in production. This client might allow you to take advantage of | ||
* new features in the agent before they are released, but it might also break your application. | ||
*/ | ||
public interface DirectStatsDClient extends StatsDClient { | ||
|
||
/** | ||
* Records values for the specified named distribution. | ||
* | ||
* <p>The method doesn't take care of breaking down the values array if it is too large. It's up to the caller to | ||
* make sure the size is kept reasonable.</p> | ||
* | ||
* <p>This method is a DataDog extension, and may not work with other servers.</p> | ||
* | ||
* <p>This method is non-blocking and is guaranteed not to throw an exception.</p> | ||
* | ||
* @param aspect the name of the distribution | ||
* @param values the values to be incorporated in the distribution | ||
* @param sampleRate percentage of time metric to be sent | ||
* @param tags array of tags to be added to the data | ||
*/ | ||
void recordDistributionValues(String aspect, double[] values, double sampleRate, String... tags); | ||
|
||
|
||
/** | ||
* Records values for the specified named distribution. | ||
* | ||
* <p>The method doesn't take care of breaking down the values array if it is too large. It's up to the caller to | ||
* make sure the size is kept reasonable.</p> | ||
* | ||
* <p>This method is a DataDog extension, and may not work with other servers.</p> | ||
* | ||
* <p>This method is non-blocking and is guaranteed not to throw an exception.</p> | ||
* | ||
* @param aspect the name of the distribution | ||
* @param values the values to be incorporated in the distribution | ||
* @param sampleRate percentage of time metric to be sent | ||
* @param tags array of tags to be added to the data | ||
*/ | ||
void recordDistributionValues(String aspect, long[] values, double sampleRate, String... tags); | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
11 changes: 11 additions & 0 deletions
11
src/main/java/com/timgroup/statsd/NoOpDirectStatsDClient.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
package com.timgroup.statsd; | ||
|
||
/** | ||
* A No-Op {@link NonBlockingDirectStatsDClient}, which can be substituted in when metrics are not | ||
* required. | ||
*/ | ||
public final class NoOpDirectStatsDClient extends NoOpStatsDClient implements DirectStatsDClient { | ||
@Override public void recordDistributionValues(String aspect, double[] values, double sampleRate, String... tags) { } | ||
|
||
@Override public void recordDistributionValues(String aspect, long[] values, double sampleRate, String... tags) { } | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
166 changes: 166 additions & 0 deletions
166
src/main/java/com/timgroup/statsd/NonBlockingDirectStatsDClient.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,166 @@ | ||
package com.timgroup.statsd; | ||
|
||
import static java.nio.charset.StandardCharsets.UTF_8; | ||
|
||
final class NonBlockingDirectStatsDClient extends NonBlockingStatsDClient implements DirectStatsDClient { | ||
|
||
public NonBlockingDirectStatsDClient(final NonBlockingStatsDClientBuilder builder) throws StatsDClientException { | ||
super(builder); | ||
} | ||
|
||
@Override | ||
public void recordDistributionValues(String aspect, double[] values, double sampleRate, String... tags) { | ||
if ((Double.isNaN(sampleRate) || !isInvalidSample(sampleRate)) && values != null && values.length > 0) { | ||
if (values.length == 1) { | ||
recordDistributionValue(aspect, values[0], sampleRate, tags); | ||
} else { | ||
sendMetric(new DoublesStatsDMessage(aspect, Message.Type.DISTRIBUTION, values, sampleRate, 0, tags)); | ||
} | ||
} | ||
} | ||
|
||
@Override | ||
public void recordDistributionValues(String aspect, long[] values, double sampleRate, String... tags) { | ||
if ((Double.isNaN(sampleRate) || !isInvalidSample(sampleRate)) && values != null && values.length > 0) { | ||
if (values.length == 1) { | ||
recordDistributionValue(aspect, values[0], sampleRate, tags); | ||
} else { | ||
sendMetric(new LongsStatsDMessage(aspect, Message.Type.DISTRIBUTION, values, sampleRate, 0, tags)); | ||
} | ||
} | ||
} | ||
|
||
abstract class MultiValuedStatsDMessage extends Message { | ||
private final double sampleRate; // NaN for none | ||
private final long timestamp; // zero for none | ||
private int metadataSize = -1; // Cache the size of the metadata, -1 means not calculated yet | ||
private int offset = 0; // The index of the first value that has not been written | ||
|
||
MultiValuedStatsDMessage(String aspect, Message.Type type, String[] tags, double sampleRate, long timestamp) { | ||
super(aspect, type, tags); | ||
this.sampleRate = sampleRate; | ||
this.timestamp = timestamp; | ||
} | ||
|
||
@Override | ||
public final boolean canAggregate() { | ||
return false; | ||
} | ||
|
||
@Override | ||
public final void aggregate(Message message) { | ||
} | ||
|
||
@Override | ||
public final boolean writeTo(StringBuilder builder, int capacity, String containerID) { | ||
int metadataSize = metadataSize(builder, containerID); | ||
writeHeadMetadata(builder); | ||
boolean partialWrite = writeValuesTo(builder, capacity - metadataSize); | ||
writeTailMetadata(builder, containerID); | ||
return partialWrite; | ||
|
||
} | ||
|
||
private int metadataSize(StringBuilder builder, String containerID) { | ||
if (metadataSize == -1) { | ||
final int previousLength = builder.length(); | ||
final int previousEncodedLength = builder.toString().getBytes(UTF_8).length; | ||
writeHeadMetadata(builder); | ||
writeTailMetadata(builder, containerID); | ||
metadataSize = builder.toString().getBytes(UTF_8).length - previousEncodedLength; | ||
builder.setLength(previousLength); | ||
} | ||
return metadataSize; | ||
} | ||
|
||
private void writeHeadMetadata(StringBuilder builder) { | ||
builder.append(prefix).append(aspect); | ||
} | ||
|
||
private void writeTailMetadata(StringBuilder builder, String containerID) { | ||
builder.append('|').append(type); | ||
if (!Double.isNaN(sampleRate)) { | ||
builder.append('|').append('@').append(format(SAMPLE_RATE_FORMATTER, sampleRate)); | ||
} | ||
if (timestamp != 0) { | ||
builder.append("|T").append(timestamp); | ||
} | ||
tagString(tags, builder); | ||
if (containerID != null && !containerID.isEmpty()) { | ||
builder.append("|c:").append(containerID); | ||
} | ||
|
||
builder.append('\n'); | ||
} | ||
|
||
private boolean writeValuesTo(StringBuilder builder, int remainingCapacity) { | ||
if (offset >= lengthOfValues()) { | ||
return false; | ||
} | ||
|
||
int maxLength = builder.length() + remainingCapacity; | ||
|
||
// Add at least one value | ||
builder.append(':'); | ||
writeValueTo(builder, offset); | ||
int previousLength = builder.length(); | ||
|
||
// Add remaining values up to the max length | ||
for (int i = offset + 1; i < lengthOfValues(); i++) { | ||
builder.append(':'); | ||
writeValueTo(builder, i); | ||
if (builder.length() > maxLength) { | ||
builder.setLength(previousLength); | ||
offset = i; | ||
return true; | ||
} | ||
previousLength = builder.length(); | ||
} | ||
offset = lengthOfValues(); | ||
return false; | ||
} | ||
|
||
protected abstract int lengthOfValues(); | ||
|
||
protected abstract void writeValueTo(StringBuilder buffer, int index); | ||
} | ||
|
||
final class LongsStatsDMessage extends MultiValuedStatsDMessage { | ||
private final long[] values; | ||
|
||
LongsStatsDMessage(String aspect, Message.Type type, long[] values, double sampleRate, long timestamp, String[] tags) { | ||
super(aspect, type, tags, sampleRate, timestamp); | ||
this.values = values; | ||
} | ||
|
||
@Override | ||
protected int lengthOfValues() { | ||
return values.length; | ||
} | ||
|
||
@Override | ||
protected void writeValueTo(StringBuilder buffer, int index) { | ||
buffer.append(values[index]); | ||
} | ||
} | ||
|
||
final class DoublesStatsDMessage extends MultiValuedStatsDMessage { | ||
private final double[] values; | ||
|
||
DoublesStatsDMessage(String aspect, Message.Type type, double[] values, double sampleRate, long timestamp, | ||
String[] tags) { | ||
super(aspect, type, tags, sampleRate, timestamp); | ||
this.values = values; | ||
} | ||
|
||
@Override | ||
protected int lengthOfValues() { | ||
return values.length; | ||
} | ||
|
||
@Override | ||
protected void writeValueTo(StringBuilder buffer, int index) { | ||
buffer.append(values[index]); | ||
} | ||
} | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.