Generate Rust type from JSON

[yoshuawuyts]

As suggested on Twitter, it'd be amazing if Rust-Analyzer could support generating Rust structs from JSON output directly, rather than needing to go through a manual converter.

Prior Art

Jetbrain's GoLand has a section titled "working with JSON" where they show off a copy-paste workflow to generate Go structures directly from JSON. Perhaps something similar could be supported in Rust-Analyzer too?

[yoshuawuyts]

Follow-up: @evestera shared they have working JSON -> Rust struct logic here: https://github.com/evestera/json_typegen

We could likely either import this code or use it as the starting point for an integration into Rust-Analyzer.

[Afourcat]

Hello, If nobody is working on this issue, I think I could help.
Is someone able to help me figure out where to start? I'm not yet familiar with the code-base it's only my second time contributing 😅
Thank you very much,

[evestera]

To add some context that may be useful here:

The core of json_typegen, json_typegen_shared, which is probably what you want to use if you want to use that code, may be a bit more heavy in terms of dependencies than you may want to include in rust-analyzer if you don't have any use for the configurability of json_typegen (e.g. if you just want to create something like the "create-type-on-paste"-interaction shown above).

If you want to use json_typegen_shared I could do some things to make it less heavy (like making option parsing an optional feature).

If you want to write new code for this to keep it light and understandable, and you already use serde + serde_json, there is a "hello world"-version of the shape inference algorithm and code generation you can look at here: https://github.com/evestera/thesis/tree/master/code/shape_inference

I also have a simpler no-dependency prototype rewrite in a currently private repo where the parsing is done on streaming JSON tokens rather than serde_json::Value, both to improve memory use for very large files and to optimize the crate for WASM use. If that is of interest I guess I can open that up as well.

[Afourcat]

Thank you for answering quickly,

...maybe a bit heavier in terms of dependencies than you may want to include in rust-analyzer.

Indeed, if we could avoid adding dependencies to the project, it would be great.

I also have a simpler no-dependency prototype rewrite in a currently private repo.

If the use of this lib does not add a lot of complexity to your legacy version, why not work with it (If you are ok with it). However, in this case, we want to parse relatively small JSONs, I think that memory and WASM compatibility are not the main goals here 😃

How would you want to proceed?
According to you, which one of your propositions is a good compromise between simplicity, dependencies, and dev time?
Is your private prototype ready for production use?

Thank you @evestera for helping me out on this one! 👍

[flodiebold]

We should also use our own code generation facilities.

[evestera]

To be clear, I know nothing about rust-analyzer, which is why I'm only giving context on json_typegen.

What is actually the best way to do this depends on what you want to implement, and details of how rust-analyzer works. E.g. if you want to have some configurability like the options you can see at https://typegen.vestera.as/ , depending on json_typegen_shared or bundling it as a WASM module may be a good solution. (Bundling it as a WASM module is what https://transform.tools/json-to-rust-serde – the manual tool referred to in the initial comment – does).

The plan for the private prototype was for it to replace the relevant parts of json_typegen, so it is not intended to ever become a published project. It currently only has JSON parsing and shape inference, since those are the parts that were under consideration to be replaced in json_typegen. So if you want to use that code you have to combine it with either code from json_typegen or your own code generation code. I've made the repo visible now, so you can have a look if it is of interest: https://github.com/evestera/typegen2

If you want an easy solution with regards to dev time, I think the solution is to depend on json_typegen, either as a dependency or as a WASM module. If I make the mentioned change of making option parsing optional (edit: Done. Published as 0.7.0), using json_typegen as a normal dependency would bring in the following transitive dependencies:

serde = "1.0"
serde_derive = "1.0"
serde_json = { version = "1.0", features = ["preserve_order"] }
error-chain = "0.12.4"
lazy_static = "1.2"
linked-hash-map = "0.5.3"
Inflector = "0.11"
regex = "1.1"
unindent = "0.1.2"

Most of those are very common or small enough to not really matter, but of course, you need to evaluate what you are fine with and not.

[Afourcat]

Ok, thank you for all those details,

We may need the opinion of a maintainer, I don't think I'm able to choose between these options even If I try as much as I can 😢
In the meantime, I will look at the rust-analyzer code-base, and play with your libraries to have a better understanding of where and how to plug your work in.

Thanks again,

😄

[Afourcat]

From what I may have understood,
I need to add an ide_assist for example: crates/ide_assists/handlers/convert_json_to_struct.

And I should write a function that may look like this:

pub(crate) fn convert_json_to_struct(acc: &mut Assists, ctx: &AssistContext) -> Option<()> {
    let json_expr = ctx.find_json_node(); // (1)

    let target = // Target is the full JSON expression.

    acc.add(
        AssistId("convert_json_to_struct", AssistKind::RefactorRewrite), // (2)
        "Convert a valid JSON expression to a rust struct.",
        target,
        |edit| {
            // (3)
            generate_rust_type_from_json(edit, json_expr);
        },
    )
}

However, I have some questions,

• (1) I don't know how to properly find/parse JSON from the context, because it is not a node of Rust AST. Is there a proper way to get this or should I add some logic to handle it?
• (2) Is the AssistKind::RefactorRewrite right?
• (3) Is it possible to ask the user for the name of the struct, or is it more user-friendly to put his cursor on between struct and {.

Thank you,