Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix issue with mqtt concurrent map write #8562

Merged
merged 2 commits into from
Dec 16, 2020
Merged

fix issue with mqtt concurrent map write #8562

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a3e646d
fix issue with mqtt concurrent map write
ssoroka Dec 14, 2020
bf39b7f
clean up locks
ssoroka Dec 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion plugins/inputs/mqtt_consumer/mqtt_consumer.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,10 @@ import (
"errors"
"fmt"
"strings"
"sync"
"time"

"github.com/eclipse/paho.mqtt.golang"
mqtt "github.com/eclipse/paho.mqtt.golang"
"github.com/influxdata/telegraf"
"github.com/influxdata/telegraf/internal"
"github.com/influxdata/telegraf/plugins/common/tls"
Expand Down Expand Up @@ -69,6 +70,7 @@ type MQTTConsumer struct {
state ConnectionState
sem semaphore
messages map[telegraf.TrackingID]bool
messagesMutex sync.Mutex
topicTag string

ctx context.Context
Expand Down Expand Up @@ -219,7 +221,9 @@ func (m *MQTTConsumer) connect() error {

m.Log.Infof("Connected %v", m.Servers)
m.state = Connected
m.messagesMutex.Lock()
m.messages = make(map[telegraf.TrackingID]bool)
m.messagesMutex.Unlock()

// Persistent sessions should skip subscription if a session is present, as
// the subscriptions are stored by the server.
Expand Down Expand Up @@ -258,13 +262,15 @@ func (m *MQTTConsumer) recvMessage(c mqtt.Client, msg mqtt.Message) {
select {
case track := <-m.acc.Delivered():
<-m.sem
m.messagesMutex.Lock()
_, ok := m.messages[track.ID()]
if !ok {
// Added by a previous connection
continue
}
// No ack, MQTT does not support durable handling
delete(m.messages, track.ID())
m.messagesMutex.Unlock()
case m.sem <- empty{}:
err := m.onMessage(m.acc, msg)
if err != nil {
Expand All @@ -290,7 +296,9 @@ func (m *MQTTConsumer) onMessage(acc telegraf.TrackingAccumulator, msg mqtt.Mess
}

id := acc.AddTrackingMetricGroup(metrics)
m.messagesMutex.Lock()
m.messages[id] = true
m.messagesMutex.Unlock()
return nil
}

Expand Down
2 changes: 1 addition & 1 deletion plugins/inputs/mqtt_consumer/mqtt_consumer_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import (
"testing"
"time"

"github.com/eclipse/paho.mqtt.golang"
mqtt "github.com/eclipse/paho.mqtt.golang"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please elaborate on why this is necessary!?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was automatically added by the formatting library, (I think goreturns, in this case). It doesn't like the implicit import of the mqtt package via a path that doesn't match the package name.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok so this was my "TIL". :-) Thanks for the explanation!

"github.com/influxdata/telegraf"
"github.com/influxdata/telegraf/plugins/parsers"
"github.com/influxdata/telegraf/testutil"
Expand Down