Flatten

[arnm]

I am interacting with an API which returns pagination information on most of their requests. Here is an example:

{
    "limit": 25,
    "offset": 0,
    "total": 1,
    "users": [
        {...}
    ]
}

limit, offset, and total are present for many of the responses received. Here is the struct I deserialize the JSON response into:

#[derive(Debug, Serialize, Deserialize)]
pub struct Users {
    limit: i32,
    offset: i32,
    total: i32,
    users: Vec<User>
}

The problem that I'm having is that I don't know how I can avoid repeating those same 3 fields in other structs.

I would like to have something similar to this:

#[derive(Debug, Serialize, Deserialize)]
pub struct Pagination {
    offset: i32,
    limit: i32,
    total: i32
}

#[derive(Debug, Serialize, Deserialize)]
pub struct Users {
    pagination: Pagination,
    users: Vec<User>
}

Here serde would see that no field pagination was present but the type of the struct field is Pagination which contains fields which are present in the response.

I guess this could be solved with some kind of struct inheritance but Rust doesn't have that at the moment which restricts us to use composition.

Any ideas?

[oli-obk]

i think the title should be "flattening of struct member of struct type" or sth... The issue with this kind of feature is the same that struct inheritance has: fields defined in both the parent and the child will collide. I guess it would be possible to implement some kind of #[serde(flatten)] attribute. Maybe for now it would be easier for you to create a macro that inserts those fields.

[oli-obk]

An example:

macro_rules! pagination{
    (pub struct $name:ident {$($field:ident : $ty:ty)*}) => (
        #[derive(Debug)]
        pub struct $name {
            offset: i32,
            limit: i32,
            total: i32,
            $($field : $ty),*
        }
    )
}

#[derive(Debug)]
struct User;

pagination!{
    pub struct Users {
        users: Vec<User>
    }
}

PlayPen

I don't think this is a serde-related issue, flattening could be useful for structs in general... There could be a crate for this

[Diggsey]

You can do this the other way aroound:

struct Flattened<A, B>(pub A, pub B);

#[derive(Serialize, Deserialize, ...)]
struct Pagination {
    offset: i32,
    limit: i32,
    total: i32,
}

type Paginated<T> = Flattened<T, Pagination>;

Then manually implement Serialize/Deserialize for Flattened<A, B>, such that for each field you encounter, you first try to deserialize it to A, and then if that errors with UnknownFieldError, try to deserialize it to B. Serialization is just a case of merging the two instead.

[oli-obk]

A use case from http://stackoverflow.com/q/41042767/1103681:

#[serde(rename)] seems to be the right option, but the documentation does not state if it is possible or how to do it.

This JSON object:

    {
       "name" : "myobject"
       "info" : 
       {
          "counter" : "3"
          "foo" : "bar"
       }
    }

The corresponding flat Rust struct should be:

    #[derive(Deserialize)]
    struct Object {
        name: String,
        #[serde(rename="info.counter")] // wrong syntax here !!
        count: i32,
        #[serde(rename="info::foo")] // neither this works
        foo: String,
    }

[dtolnay]

Looks like that is the opposite use case actually? That post has hierarchy in the JSON but not the Rust type. Here we have hierarchy in the Rust type but not the JSON. I guess we need to think about both.

[dtolnay]

We would also want to support flattening members of enum type. This came up in IRC:

struct File {
    name: String,
    path: String,
    #[serde(flatten, rename = "fType")]
    f_type: FileType,
}

enum FileType {
    #[serde(rename = "file")]
    File,
    #[serde(rename = "directory")]
    Directory {
        contents: Vec<File>,
    }
}

and on the JavaScript side the server expects:

{ "name": "...", "path": "...", "fType": "directory", "contents": [{ "..." }] }

[rpjohnst]

A similar use case, where only the enum tag is flattened:

struct Event {
    index: i32,
    time: u64,
    line: u32,
    #[serde(flatten, tag = "eType")]
    data: EventKind,
}

enum EventKind {
    State { state: Vec<...>, ... },
    Output { ... },
    Terminate { ... },
}

serializes to this:

{ "index": ..., "time": ..., "line": ..., "eType": "State", "data": { "state": [ ... ], ... } }

[anowell]

I hit a similar case of wanting to flatten an enum variant, where I wanted to deserialize any string into this enum, opportunistically as one of the named variants, but falling back to Custom(String):

enum Model {
    ImageNet,
    ImageNetLinear,
    Bilinear,
    Custom(String),
}

I ended up just using a String field in the containing struct, but it seems like a similar #[serde(flatten)] attribute on the Custom(String) might accomplish my intent (assuming flattening a tuple struct makes sense in the more generic case).