Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Emqtt batch output #4094

Merged
merged 46 commits into from
May 19, 2018
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
c8b3f7a
Adds code to batch messages sent on a topic in a flush interval
jvrahav May 3, 2018
3cd8a34
Adds code to batch messages sent on a topic in a flush interval
jvrahav May 3, 2018
df6fea7
modified telegraf configuration file entry
jvrahav May 3, 2018
821f5c6
formatted code
jvrahav May 3, 2018
ea98635
Ignore UTF8 BOM in JSON parser (#4099)
dmeiners88 May 3, 2018
468ff6f
Update changelog
danielnelson May 3, 2018
f957ca6
Update gopsutil version
danielnelson May 3, 2018
96146a7
Clarify max_retry option in kafka output
danielnelson May 4, 2018
7724344
Fix grammar
danielnelson May 4, 2018
7b14329
Remove -i flag from `make telegraf`
danielnelson May 4, 2018
7e2f219
Move usage string to internal to fix `go run`
danielnelson May 4, 2018
8df34fe
Fix platform not supported error in build.py (#4102)
otherpirate May 4, 2018
a529da0
Fix name_override example in mysql rreadme (#4100)
nsteinmetz May 4, 2018
7845d10
Only lowercase mysql slave metrics with metric_version = 2
danielnelson May 4, 2018
a863b3e
Update kafka readme
danielnelson May 4, 2018
5bb797e
Simplify testing with TLS (#4095)
danielnelson May 4, 2018
403b515
Add SerializeBatch method to the Serializer interface (#4107)
danielnelson May 5, 2018
6875f34
Update changelog
danielnelson May 5, 2018
77ae07b
Add topk processor plugin (#4096)
mirath May 5, 2018
3f09c69
Update changelog
danielnelson May 5, 2018
5f1fafd
Update issue templates (#4116)
danielnelson May 7, 2018
e2e6fe0
Remove combined issue template
danielnelson May 7, 2018
eb15ece
Add cursor metrics to mongodb input (#4114)
grubernaut May 7, 2018
e5216d3
Update changelog
danielnelson May 7, 2018
5c9c40f
Run apt-get update in release.sh
danielnelson May 7, 2018
9611788
Don't report 0ms on timeout in dns_query (#4118)
danielnelson May 8, 2018
d43acc1
Update changelog
danielnelson May 8, 2018
ad9dfdf
Add instructions on how to repair windows performance counters
danielnelson May 8, 2018
9b0e011
Add uint/bool support to cratedb output (#4117)
danielnelson May 8, 2018
814bf15
Update changelog
danielnelson May 8, 2018
e3cabe3
Skip fields that report "not supported" in nvidia-smi (#4123)
danielnelson May 8, 2018
c3bad9d
Add tag/integer pair for result to net_response (#3455)
morfien101 May 8, 2018
9a0fc78
Use result and result_code in net_response
danielnelson May 8, 2018
7ec960e
Update changelog
danielnelson May 8, 2018
ad40c58
Clarify max_retry option in kafka output
danielnelson May 4, 2018
0b0d466
Fix grammar
danielnelson May 4, 2018
10128c5
Update kafka readme
danielnelson May 4, 2018
b1db50f
Add tag/integer pair for result to net_response (#3455)
morfien101 May 8, 2018
322dfa2
Use result and result_code in net_response
danielnelson May 8, 2018
509d5e1
Merge branch 'master' into emqtt-batch-output
jvrahav May 9, 2018
64ad442
Modifies emqtt to use SerializeBatch for batch mode of operation
jvrahav May 9, 2018
150b344
removed serializeBatchMetric as we need to process each metric at a time
jvrahav May 10, 2018
95ac23e
removed serializeBatchMetric as we need to process each metric at a time
jvrahav May 10, 2018
266cdad
removed serializeBatchMetric as we need to process each metric at a time
jvrahav May 10, 2018
8e2a408
updated Readme
jvrahav May 11, 2018
f6d067f
modified code to use serializeBatch method for all metrics
jvrahav May 18, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add uint/bool support to cratedb output (#4117)
  • Loading branch information
danielnelson authored and jvrahav committed May 9, 2018
commit 9b0e0110a20b14613ebb808c6648b240a2e0ad34
24 changes: 12 additions & 12 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -79,15 +79,15 @@ services:
- "389:389"
- "636:636"
crate:
image: crate/crate
ports:
- "4200:4200"
- "4230:4230"
command:
- crate
- -Cnetwork.host=0.0.0.0
- -Ctransport.host=localhost
- -Clicense.enterprise=false
environment:
- CRATE_HEAP_SIZE=128m
- JAVA_OPTS='-Xms256m -Xmx256m'
image: crate/crate
ports:
- "4200:4200"
- "4230:4230"
- "5432:5432"
command:
- crate
- -Cnetwork.host=0.0.0.0
- -Ctransport.host=localhost
- -Clicense.enterprise=false
environment:
- CRATE_HEAP_SIZE=128m
19 changes: 15 additions & 4 deletions plugins/outputs/cratedb/cratedb.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"encoding/binary"
"fmt"
"sort"
"strconv"
"strings"
"time"

Expand All @@ -16,6 +17,8 @@ import (
_ "github.com/jackc/pgx/stdlib"
)

const MaxInt64 = int64(^uint64(0) >> 1)

type CrateDB struct {
URL string
Timeout internal.Duration
Expand Down Expand Up @@ -115,11 +118,19 @@ func escapeValue(val interface{}) (string, error) {
switch t := val.(type) {
case string:
return escapeString(t, `'`), nil
// We don't handle uint, uint32 and uint64 here because CrateDB doesn't
// seem to support unsigned types. But it seems like input plugins don't
// produce those types, so it's hopefully ok.
case int, int32, int64, float32, float64:
case int64, float64:
return fmt.Sprint(t), nil
case uint64:
// The long type is the largest integer type in CrateDB and is the
// size of a signed int64. If our value is too large send the largest
// possible value.
if t <= uint64(MaxInt64) {
return strconv.FormatInt(int64(t), 10), nil
} else {
return strconv.FormatInt(MaxInt64, 10), nil
}
case bool:
return strconv.FormatBool(t), nil
case time.Time:
// see https://crate.io/docs/crate/reference/sql/data_types.html#timestamp
return escapeValue(t.Format("2006-01-02T15:04:05.999-0700"))
Expand Down
8 changes: 4 additions & 4 deletions plugins/outputs/cratedb/cratedb_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -111,12 +111,12 @@ func Test_escapeValue(t *testing.T) {
{`foo`, `'foo'`},
{`foo'bar 'yeah`, `'foo''bar ''yeah'`},
// int types
{123, `123`}, // int
{int64(123), `123`},
{int32(123), `123`},
{uint64(123), `123`},
{uint64(MaxInt64) + 1, `9223372036854775807`},
{true, `true`},
{false, `false`},
// float types
{123.456, `123.456`},
{float32(123.456), `123.456`}, // floating point SNAFU
{float64(123.456), `123.456`},
// time.Time
{time.Date(2017, 8, 7, 16, 44, 52, 123*1000*1000, time.FixedZone("Dreamland", 5400)), `'2017-08-07T16:44:52.123+0130'`},
Expand Down