Convert voluptuous schema to msgspec

[abhishekmadan30]

No description provided.

[ahal]

Thanks, this is shaping up nicely!

I don't think I have any other major concerns, it's mostly all nits and a minor changes. So I think you can go ahead and start converting Gecko without fear of a major new request (from me at least :p)

[ahal]

Nice, thanks for the updates! I don't see any fundamental problems or anything, though I'll reserve the right to request future changes as we test this out in Gecko ;)

I'll avoid approving this for now because I want to hold off landing until we have a working patch for Gecko, but I think you can go ahead and get started on that! Hopefully claude or some clever macros can help with the conversion, because there's going to be a lot of schemas :)

[ahal]

I didn't get a chance to review the whole thing, but I'll leave the comments I had so far for now and resume tomorrow. Figured you might want to get a head start

[ahal]

A few more comments, still not all the way through though.

[ahal]

You should import the schema from gecko_taskgraph.transforms.task and reference the index key there when we come across these "cross references".

[ahal]

Thanks, looks like you have some merge conflicts too. Please rebase and push your latest iteration and hopefully I can do one final review (of just the interdiff this time)

[jcristau]

Is there any chance we can do this in a backwards-compatible way, e.g. by leaving existing util/schema.py APIs alone and adding the msgspec schemas as an alternative, so we can convert things piecemeal and avoid massive changes and likely regressions/back and forth? Or is that impossible for some reason?

[abhishekmadan30]

Is there any chance we can do this in a backwards-compatible way, e.g. by leaving existing util/schema.py APIs alone and adding the msgspec schemas as an alternative, so we can convert things piecemeal and avoid massive changes and likely regressions/back and forth? Or is that impossible for some reason?

Yep, I am currently working on it, I think we will go with just a different name for each of the schemas. eg volutpous is still called Schema while msgspec is called Struct and for shared functions, Ill add use_msgspec flag which by default is to false to limit future compatibility issues

[ahal]

Oh, I see what you meant now by having separate functions for this.. For some reason I was thinking there'd be an object passed in here we could test, but that obviously isn't the case. I like this approach though.

[ahal]

Do you like Struct as a name? Do you have any other ideas? I'm not sure if I like it, but I can't really think of anything better.

[ahal]

Would you mind splitting all the actual schema conversion work into a separate commit from supporting msgspec in validate_schema? This way it will be easy to prove that the generation works before and after the change.