Add json_metadata property to BaseDistribution

[sbidoul]

As suggested by @pfmoore in #10771 (comment), this copies the msg_to_json function from his pkg_metadata project.

This function will be used in pip install --report (and possibly pip list --format=json).

Towards #53 and #10771.

[pfmoore]

How important is it that msg_to_json takes a Message argument? I'm considering changing it in pkg_metadata to take a bytes object (and hence hiding the dependency on email.message) as that makes the normal usage (read a metadata file) marginally simpler. But if conversion from Message format is useful, I'll leave that available as well.

[sbidoul]

@pfmoore In this case, it was convenient to have it since BaseDistribution exposes metadata as an email.message.Message. If pkg_metada would not have had that capability I'd have needed to refactor a bit.

So from this unique sample, I say this is useful. In other places where I've had to handle package metadata, email.message.Message is also involved, so it is certainly a convenient intermediate representation.

That said, exposing functions that handle metadata file encodings properly in pkg_metadata would certainly be useful too as I remember it is a topic that is not obvious in the PEP texts.

[pfmoore]

Sounds good. I can expose msg_to_json and a thin wrapper for bytes_to_json easily enough.

[uranusjr]

This should probably be msg_to_dict instead since the output is not actually JSON (which should be a str), but a dict decoded from JSON metadata.

[uranusjr]

Let’s make this private so we don’t need to worry about json_name being much too vague…

[pfmoore]

It's not in __all__ in the original implementation, but agreed a better name might be good.

Would you care to review https://github.com/pfmoore/pkg_metadata? I'd be happy to incorporate any other feedback you might have in the original library.

[uranusjr]

Same for this property name; json feels misleading 😟

[sbidoul]

Hi @uranusjr, thanks for the review!

Actually this json conversion function is copied verbatim from @pfmoore 's pkg_metadata, as the goal is to vendor it when Paul says it is good enough for that. That is also the reason why there are no tests for it here. See also comment

The BaseDistribution.json_metadata property is the way I chose to expose it to pip internals and the rest is an implementation detail that will change as soon as we vendor pkg_metadata. What I could do did is rename json.py to _json.py to make it more explicit that it is "private".

I felt a property was appropriate since there was already a metadata property returning a Message.

I agree json_metadata is slightly misleading as the value is a dict and not a json string or bytes. But TBH I have no better idea that would also express the fact that this dictionary is json-serializable according to the transformation specified in PEP 566. Perhaps json_metadata_dict would be better ?

[sbidoul]

@uranusjr @pfmoore how do you think we should move forward with this one ?

[pfmoore]

I’ll probably change the name to msg_to_dict in pkg_metadata. I don’t really have a strong opinion on the name to use here. I’m fine with keeping json_metadata, but I’d also be ok with something like metadata_dict. Whatever we choose, we can easily change it later, after all… 🤷

[pfmoore]

This could be cached, too. There's no real reason to recalculate it every time. Although it's probably not a performance bottleneck, so it's not a big deal either way.

[sbidoul]

@uranusjr I think all your concerns have been addressed. If not please do not hesitate to say so and I'll be happy to address them in a followup.

Thanks for the review and thanks @pfmoore for pkg_metadata. When you think it is ready for prime time, ping me and I'll do the vendoring.