Deserializing borrowed string when it contains \n escape

[terhechte]

Hey!

I have this very simple example:

extern crate serde;
extern crate serde_json;
#[macro_use]
extern crate serde_derive;

use std::fs::File;
use std::collections::HashMap;
use std::io::prelude::*;

#[derive(Deserialize)]
struct User<'a> {
    username: &'a str
}

#[derive(Deserialize)]
struct Comment<'a> {
    author: User<'a>,
    text: &'a str,
    likes: Vec<User<'a>>
}

#[derive(Deserialize)]
struct Media<'a> {
    author: User<'a>,
    likes: Vec<User<'a>>,
    comments: Vec<Comment<'a>>,
    images: HashMap<String, String>,
    description: &'a str
}

fn conv<'a>(string: &'a str) -> Vec<&'a str> {
    let media: Vec<Media> = serde_json::from_str(&string).unwrap();
    media.into_iter().map(|x|x.author.username).collect::<Vec<&str>>()
}

fn main() {
    let mut f = File::open("./test.json").expect("file not found");
    let mut contents = String::new();
    f.read_to_string(&mut contents).expect("something went wrong reading the file");
    println!("{:?}", &conv(&contents));
}

This is the input json:

[{"images": {"small": "http://appventure.me/small.png", "medium": "http://appventure.me/medium.png"}, "description": {"text": "Duis quam lorem, lacinia nec tempus non, Sondra Swigart tristique", "likes": [{"username": "Loan"}, {"username": "Shaneka"}], "author": {"username": "Hattie"}}, "likes": [{"username": "Les"}, {"username": "Vance"}, {"username": "Marilee"}], "comments": [{"text": "dictum feugiat. Ut blandit volutpat Kandis Ammerman ante in commodo.", "likes": [{"username": "Les"}], "author": {"username": "Les"}}, {"text": "luctus, enim lacus sagittis arcu, Clement Cassity at mollis tellus", "likes": [{"username": "Marisha"}], "author": {"username": "Liliana"}}], "author": {"username": "Debbie"}}]

I'm getting the error:

invalid type: string [...] expected a borrowed string

What did I do wrong? I did read the docs but couldn't figure it out.

[dtolnay]

That is not the error I get when running the program you gave.

Your input file contains "description": { ... } where the description value is a map, but the Media struct contains description: &'a str where the value is a string.

[terhechte]

Hey! Sorry for the confusion. I had edited the JSON because the actual file is 5mb big. I botched it while doing so. I've created a playground that exhibits the error:

https://play.rust-lang.org/?version=stable&mode=debug&edition=2015&gist=ae491238f0e4da1c555e7b26180ad2f6

[dtolnay]

Here is a minimized example (same as your code but with extraneous fields removed):

https://play.rust-lang.org/?edition=2015&gist=378fc363d8ec9752d4e19f4500c07851

Here you need to consider what you expect 'a to be, i.e. what variable owns the data from which the &'a str is borrowed.

[terhechte]

Doesn't (in your example) j own the data as 'static? And even if not, everything is dropped at the end of main. How could I change it, so that it works? I have to admit I don't get what's wrong here. Changing it to:

struct Media {
    description: String
}

would obviously work, but I wanted to play around the serde zero-copy deserialization feature.

[dtolnay]

Well the length of the string in description needs to be 3 and it needs to contain the byte 'A' followed by a newline (byte '\x0A') followed by the byte 'B', because this is the JSON string in the input. There is no 'static data (in j or anywhere else) containing 'A' followed by a newline followed by 'B'.

[michael-grunder]

For anyone landing on this issue (like I did), I think a bit more context may be useful.

The reason serde can't deserialize into a borrowed string is that it needs to modify (unescape) the encoded json, which requires ownership.

That said, it appears you can get around this by using a Cow<'a, str> allowing deserialization to borrowed or owned memory depending on the string itself.

Link to playground

If this is incorrect please let me know (I'm still somewhat new to Rust).

[Xaeroxe]

This seems like a case where the error messaging describing the problem could be improved. I can accept that the lib needs to own the data once the "escape" dilemma is explained to me (thanks @michael-grunder ) however when I first saw this error message in my own code I was baffled as to why a borrowed string posed a problem here. Perhaps serde's error message could better explain why the distinction matters.

[ytrezq]

For anyone landing on this issue (like I did), I think a bit more context may be useful.

The reason serde can't deserialize into a borrowed string is that it needs to modify (unescape) the encoded json, which requires ownership.

That said, it appears you can get around this by using a Cow<'a, str> allowing deserialization to borrowed or owned memory depending on the string itself.

Link to playground

If this is incorrect please let me know (I'm still somewhat new to Rust).

@michael-grunder as an alternative, is there a way to specify a borrowed string from the json?