[inputs.solr] Changed response format in Solr7 leaves plugin returning 0 for most values

[kingcrunch]

Bug report

Relevant telegraf.conf:

[[inputs.solr]]
  servers = ["http://localhost:8983"]

System info:

Telegraf: 1.5.1
Solr: 7.1 and Solr 7.2

Steps to reproduce:

1. Setup Solr >=7.0 with Core
2. Configure telegraf to use inputs.solr
3. telegraf -config telegraf.conf -input-filter solr -test

Expected behavior:

Reasonable values are retrieved

Actual behavior:

Most values are simply 0

Additional info:

See https://gist.github.com/KingCrunch/37187347ddfb2ac4267cb3166c8b2680

This are outputs from /solr/a_core/admin/mbeans?... from one Solr 6.6 and one Solr 7.2. That are two different instances, so don't expect, that both has the same values.
I had to remove some values.

[ljagiello]

Solr 7 was released after the telegraf plugin was created :(

[mkboudreau]

I'd be interested in working on this as I'm looking at adding in Solr 3 compatibility (#3681) and adding in dataimporthandler data (#3682). I would love some discussion around what makes sense for implementation (separate plugin or version config flag or ???)

[danielnelson]

We need to investigate all the details to decide, but I imagine we can do this in the same plugin. Ideally we can just handle both cases without a server_version = 7 style option, but that's an option too.

[mkboudreau]

I've been looking through the solr docs as well as looking through the API request/responses, including http headers. I cannot find anything to give a good indication of server version in order to allow detection of the server version.

Here are the approaches I currently see (and maybe you see something I don't), but only two that appear to be feasible:

1. Detect version from API somehow (does not appear to be feasible from what I can find)
2. Use a config parameter, such as server_version
3. Rework parsing to be more generic and do more type detection, where applicable. The drawback here is the code is fairly clean when it can make assumptions about the json and use json tags for unmarshalling.
4. Use a separate plugin altogether (not desirable)

Thoughts?

[danielnelson]

It depends on how different the versions end up being. I think we start with 3, and fallback to 2 if we see something that is radically different and can't coexist.

[mkboudreau]

Yeah, that makes sense. I was kinda thinking the same thing, but I wanted to have the discussion first. My definition of radically different might be different than other people. ;)

[thePanz]

@mkboudreau http://localhost:8983/solr/admin/info/system?wt=json looks like it contains the solr version. (tested on Solr 6.6.0)

Still unsure if it is available by default and how it differs on different Solr versions

[danielnelson]

Closed by #4271