[GNMI Plugin] Support for processing Base64 Encoded Binary Data #15046
Assignees
Labels
Comments
rc-networks
added
the
feature request
|
@rc-networks could you please create a configuration containing [agent]
debug = true
[[inputs.gnmi]]
...
dump_responses = true
[[inputs.gnmi.subscription]]
name = "ps"
origin = "openconfig"
path = "/components/component/power-supply/state"
subscription_mode = "sample"
sample_interval = "60s"
[[outputs.file]]
files = ["stdout"]with the adapted Please post a couple of those lines as I need this for adding test cases! |
|
Sorry about that, here is the complete (redacted) config file: And here is the complete CLI debug output: Please let me know if you need other info |
|
@rc-networks for my understanding, the field values are encoded as a JSON IETF field which itself is in Base64 format. When decoding for example the first data-sample we see {
"openconfig-platform-psu:enabled": true,
"openconfig-platform-psu:capacity": "RDLAAA==",
"openconfig-platform-psu:input-current": "PvEm6Q==",
"openconfig-platform-psu:input-voltage": "Q1CAAA==",
"openconfig-platform-psu:output-current": "P5n75w==",
"openconfig-platform-psu:output-voltage": "QmF3zw==",
"openconfig-platform-psu:output-power": "QolAAA=="
}representing the fields. Now from my understanding decoding those values require the corresponding YANG model (or at least the knowledge encoded in there) as we otherwise cannot know if the field is Is this understanding correct? If so, would it be feasible to provide a complete YANG model hierarchy, i.e. do you usually have all required model files available locally? |
|
You are correct, in my experience everything are sent as a string and as you said. That As for the model references, I usually use openconfig and YangModels/yang. I believe the best thing we can do is to be able to decode |
|
I read RFC 7951 which is used for JSON-IETF fields according to the GNMI spec and that JSON representation indeed contains data serialized to the corresponding YANG model. So for decoding we really need to read the YANG model to know how to interpret the data. In our case, the YANG model for the PSU encodes the data as Playing around with existing libraries, I do have a PoC to get the YANG model information but there currently is no upstream way to convert the actual data. Will probably dig deeper but this might take some time... |
1 task
|
@rc-networks please test the binary in PR #15201, available once CI finished the tests! You need to specify a directory containing the YANG-model(s) to decode the platform-psu information. I tested with the models from https://github.com/openconfig/public and successfully decoded the values. |
telegraf-tiger
bot
removed
the
waiting for response
|
Yes! I'm still interested, I just got too busy with other things at the moment. |
telegraf-tiger
bot
removed
the
waiting for response
|
@srebhan I'm sorry for being late, it looks like the artifacts are no longer available for download. |
|
@rc-networks I pushed the code again, a binary should be available once CI finished the tests. It might take ~30min... |
|
@srebhan , I tested with the problematic one that is sending binary string data I tested with another one just to be sure ( Some fields worked, but others are not so lucky. Additional note, here is the setting for the yang directories: basically just cloned the gh repo and pointed to the yang model folders. |
|
@rc-networks could you please dump the received data so I can debug what is going on!? I need the |
|
About that, I've been running it for some time now, but I never got the any lines with I only got these: |
|
@rc-networks did you set |
|
I totally forgot about that dump under the gnmi plugin. Here is output of the dump and here is the entire log if needed: |
|
@rc-networks thanks for the data. I pushed a change to the PR, could you please test again? |
|
@srebhan Maybe I should test every openconfig yang model that I use before we close it just to be sure |
Would love the extra testing! How much time would this take? |
|
@powersj But here's my observation for the openconfig models so far:
output: fields are returning as stringed integers
output openconfig datatype: decimal64 The one that was initially reported (openconfig-platform-psu) was working as expected. output: if the goal was to only fix openconfig-platform-psu datatype handling, then the goal for this change is already achieved. |
|
@rc-networks thanks for your report and the data. Decoding the "temperature" fields is kind of another feature as here, the JSON field not explicitly states the YANG model to use (contrary to the PSU ones) but it requires to lookup the provided path in the YANG model. Anyway, I extended the PR to handle the above case. Please note that it can only handle Let me know if this helps... |
|
This change helps a lot. |
|
So you can confirm that the PR works as intended? If so, I can mark it as ready-for-review... |
|
@srebhan Yes, as far as I can see, it's working as expected. |
|
Thank you very much for confirming! I'm going to reopen, just until we review and land the PR. |
|
@rc-networks I improved the code a bit further (faster lookup)... Can you please test the updated version again as soon as the binaries are available!? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Use Case
Opening this Feature Request based on the discussion over #14950 .
GNMI uses YANG models to get standardized data.
Some models return data in binary format which is then encoded as Base64 following the IETF standard defined on this link, saying:
Example GNMI subscription setting:
CLI Logs with debug option:
Sample output:
For this example, we are using the Openconfig yang model openconfig-platform-psu, is returning
ieeefloat32inbinary formatwhich is then encoded into base64 as stated in the IETF standard defined on this link.Here is a description on one of the model leafs:
(image taken from YANGSUITE app)
Expected behavior
The ability of GNMI plugin to decode Base64 binary string and convert that decoded data to the required format before passing it to the output plugin.
Final expected output:
Values are in human readable float/int format
Actual behavior
Values are Base64 encoded binary strings.
Additional info
No response
The text was updated successfully, but these errors were encountered: