Arbitrary precision numbers don't work with `serde(tag = ..)` and `serde(flatten)`

[idubrov]

The code below fails if arbitrary_precision feature is turned on (works fine if it is turned off):

#[macro_use]
extern crate serde_derive;

#[derive(Deserialize)]
struct Data {
    value: i32,
}

#[derive(Deserialize)]
#[serde(tag = "type")]
enum Wrapper {
    Data(Data),
}

fn main() {
    let json = r#"{"type":"Data","value":123}"#;
    // Okay
    let _data1: Data = serde_json::from_str(json).unwrap();
    // Fails!
    let _data2: Wrapper = serde_json::from_str(json).unwrap();
}

Error backtrace:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("invalid type: map, expected i32", line: 0, column: 0)', libcore/result.rs:1009:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::print
             at libstd/sys_common/backtrace.rs:71
             at libstd/sys_common/backtrace.rs:59
   2: std::panicking::default_hook::{{closure}}
             at libstd/panicking.rs:211
   3: std::panicking::default_hook
             at libstd/panicking.rs:227

Cargo.toml:

[package]
name = "serde-arbitrary-fail"
version = "0.1.0"
authors = ["Ivan Dubrov <dubrov.ivan@gmail.com>"]

[dependencies]
serde = "1.0.80"
serde_derive = "1.0.80"
serde_json = { version = "1.0.33", features = ["arbitrary_precision"]}

[igorbernstein]

It also fails for #[serde(untagged)]

data did not match any variant of untagged enum Wrapper

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("data did not match any variant of untagged enum Wrapper", line: 0, column: 0)', src/libcore/result.rs:997:5
stack backtrace:

[idubrov]

Same for #[flatten]:

use serde_derive::{Serialize, Deserialize};

#[derive(Debug, Serialize, Deserialize)]
struct Outer {
    #[serde(flatten)]
    inner: Inner,
}

#[derive(Debug, Serialize, Deserialize)]
struct Inner {
    value: usize,
}

fn main() {
    let v = Outer {
        inner: Inner {
            value: 10,
        }
    };
    let json = serde_json::to_string(&v).unwrap();
    let v2: Outer = serde_json::from_str(&json).unwrap();
    println!("{:#?}", v2);
}

Output:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("invalid type: map, expected usize", line: 1, column: 12)', src/libcore/result.rs:997:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:70
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:58
             at src/libstd/panicking.rs:200
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:215
   4: <std::panicking::begin_panic::PanicPayload<A> as core::panic::BoxMeUp>::get
             at src/libstd/panicking.rs:478
   5: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:385
   6: std::panicking::try::do_call
             at src/libstd/panicking.rs:312
   7: core::char::methods::<impl char>::escape_debug
             at src/libcore/panicking.rs:85
   8: core::result::unwrap_failed
             at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libcore/macros.rs:16
   9: <core::result::Result<T, E>>::unwrap
             at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libcore/result.rs:798
  10: ttt::main
             at src/main.rs:21
  11: std::rt::lang_start::{{closure}}
             at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/rt.rs:64
  12: std::panicking::try::do_call
             at src/libstd/rt.rs:49
             at src/libstd/panicking.rs:297
  13: panic_unwind::dwarf::eh::read_encoded_pointer
             at src/libpanic_unwind/lib.rs:92
  14: <std::panicking::begin_panic::PanicPayload<A> as core::panic::BoxMeUp>::get
             at src/libstd/panicking.rs:276
             at src/libstd/panic.rs:388
             at src/libstd/rt.rs:48
  15: std::rt::lang_start
             at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/rt.rs:64
  16: <ttt::Inner as core::fmt::Debug>::fmt

Cargo.toml:

[package]
name = "ttt"
version = "0.1.0"
edition = "2018"

[dependencies]
serde_json = { version = "1.0", features = ["arbitrary_precision"]}
serde = "1.0"
serde_derive = "1.0"

[idubrov]

Well, the workaround is as simple as deserializing to serde_json::Value first:

#[macro_use]
extern crate serde_derive;

#[derive(Deserialize)]
struct Data {
    value: i32,
}

#[derive(Deserialize)]
#[serde(tag = "type")]
enum Wrapper {
    Data(Data),
}

fn main() {
    let json = r#"{"type":"Data","value":123}"#;
    // Okay
    let _data1: Data = serde_json::from_str(json).unwrap();
    // Fails!
    //let _data2: Wrapper = serde_json::from_str(json).unwrap();
    // Works!
    let data2: serde_json::Value = serde_json::from_str(json).unwrap();
    let _data2: Wrapper = serde_json::from_value(data2).unwrap();
}

Not great, but okay for our purposes.

I think, what happens is serde JSON parser is confused when it is asked to deserialize into this Content (used internally by serde for tag / flatten processing). It thinks it needs to do an "arbitrary number" and yield a map, but in reality, it is not (parts of serde outside of Serde JSON don't know how to process that "arbitrary precision number map").

[idubrov]

Demonstration of what happens. The following code:

#[macro_use]
extern crate serde_derive;

#[derive(Debug, Deserialize)]
struct SneakyData {
    value: std::collections::HashMap<String, String>,
}

#[derive(Debug, Deserialize)]
#[serde(tag = "type")]
enum SneakyWrapper {
    Data(SneakyData),
}

fn main() {
    let json = r#"{"type":"Data","value":123}"#;
    // Sneaky!
    let data: SneakyWrapper = serde_json::from_str(json).unwrap();
    println!("{:#?}", data);
}

prints:

Data(
    SneakyData {
        value: {
            "$serde_json::private::Number": "123"
        }
    }

[marcelbuesing]

Thanks for the workaround I also ran in to this issue.

[marcelbuesing]

When having a u128 type in the struct i ran into a combination of the previously reported and u128 is not supported (although I'm using the arbitrary_precision feature).

See here:

marcelbuesing@dab262c#diff-9a86b31b12f3b48a02818b844654ab0cR35

test regression::issue505::test_i32 ... FAILED
test regression::issue505::test_u128 ... FAILED

failures:

---- regression::issue505::test_i32 stdout ----
thread 'regression::issue505::test_i32' panicked at 'called `Result::unwrap()` on an `Err` value: Error("invalid type: map, expected i32", line: 0, column: 0)', src/libcore/result.rs:999:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

---- regression::issue505::test_u128 stdout ----
thread 'regression::issue505::test_u128' panicked at 'called `Result::unwrap()` on an `Err` value: Error("u128 is not supported", line: 0, column: 0)', src/libcore/result.rs:999:5