Implement native histograms

[SuperQ]

Prometheus now supports a new "Native Histograms" datatype. We should implement this here.

[weyert]

@SuperQ Do you know if the text version of "native histograms" has been finalised? I would like to implement in the pull exporter for opentelemetry-js.

[kcy1019]

Sadly, native histograms can only be exposed through protobuf for now, which is not supported in the Python client. It might be better to wait until the feature gets stabilized[source].

[csmarchbanks]

Hello any following along here. I am working on a design doc for text based support for native histograms: https://docs.google.com/document/d/1qoHf24cKMpa1QHskIjgzyf3oFhIPvacyJj8Tbe6fIrY/edit. It is still preliminary, and I plan to share with the broader prometheus-dev (and OpenMetrics) communities soon. Feel free to comment now though!

[Xaelias]

Hi,

Since this (plain text native histograms) is not finalized or implemented yet, I'm looking at trying to implement protobuf support in the python client.

Anyway, there are two ways of handling proto types in the client, and I figured this was as good a place to ask as any:

• keep all the internal structure as is, and just do an edge conversion from Sample to protobuf in a generate_latest
• do what golang does, and store all samples in the registry as protobuf structs, and only convert them to plain text at the edge

The first one is the least amount of changes, but also has to deal with re-aggregating samples that have been "split" for plain text reasons (things like _created).

The second is much more involved, but has the benefit of being consistent with the golang client. I would also assume it's overall more performant but I might be wrong there.

Thoughts?

PS: Happy to move this into it's own issue or discussion thread (or wherever you want to point me at)

[csmarchbanks]

I think we would want to keep the internal structure and do an edge conversion. The second option would require anyone using this library to also import a protobuf library which we want to avoid. I am ok with an optional dependency on protobuf.

[SuperQ]

I'm OK with requiring a proto library dependency by default. I think it depends on the performance impact and how much complexity it adds/removes.

[Xaelias]

also import a protobuf library which we want to avoid

What's the reasoning behind this?

I also started a thread on the mailing list but haven't heard back yet.

I'm happy to try and compare performances if we already have a way to do that though.

[csmarchbanks]

The problem with required dependencies is that, for example, protobuf requires numpy which can end up with version conflicts for many users. It is also more things for small environments to install, but mainly the risk of version conflicts is not something I want in a core library.

[D1CED]

Coincidentally I have also been working on implementing text based native histograms in OTel format D1CED@4665eed.

I am happy to collaborate. Feel free to take some test cases you see fit.