xwidgets doesn't compile with version >3.10.3

[dimitry-ishenko]

Description

The xwidgets project, which uses this library stopped compiling after version 3.10.3. Specifically, commit 0e694b4 is responsible for the problem.

Even more specifically, redefinition of is_constructible_string_type from this:

json/include/nlohmann/detail/meta/type_traits.hpp

Lines 331 to 348 in aa0e847

	template<typename BasicJsonType, typename ConstructibleStringType,
	typename = void>
	struct is_constructible_string_type_impl : std::false_type {};
	template<typename BasicJsonType, typename ConstructibleStringType>
	struct is_constructible_string_type_impl <
	BasicJsonType, ConstructibleStringType,
	enable_if_t<is_detected_exact<typename BasicJsonType::string_t::value_type,
	value_type_t, ConstructibleStringType>::value >>
	{
	static constexpr auto value =
	is_constructible<ConstructibleStringType,
	typename BasicJsonType::string_t>::value;
	};
	template<typename BasicJsonType, typename ConstructibleStringType>
	struct is_constructible_string_type
	: is_constructible_string_type_impl<BasicJsonType, ConstructibleStringType> {};

to this:

json/include/nlohmann/detail/meta/type_traits.hpp

Lines 319 to 325 in 0e694b4

	template<typename BasicJsonType, typename ConstructibleStringType>
	struct is_constructible_string_type
	{
	static constexpr auto value =
	is_constructible<ConstructibleStringType,
	typename BasicJsonType::string_t>::value;
	};

is what's causing the error.

The exact part of xwidgets where the error happens is here:

    template <class T>
    inline void xwidgets_deserialize(T& value, const nl::json& j, const xeus::buffer_sequence&)
    {
        value = j.template get<T>();
    }

I know xwidgets says that they are only compatible with json version <3.10, but most up-to-date linux distros only supply 3.10.5. Plus, everything used to work up until the above commit.

Reproduction steps

1. Install xeus-dev and xproperty-dev dependencies from my ppa:ppa-verse/cling repo.

2. Download and unpack: xwidgets_0.26.1.orig.tar.gz

3. Download (remove .txt) and unpack inside xwidgets-0.26.1 above: xwidgets_0.26.1-0ppa2.debian.tar.xz.txt

4. Execute dpkg-buildpackage -uc -us -b

Expected vs. actual results

Expected: Successful compilation and creation of xwidgets debian package.

Actual: Failed compilation. (See error messages below and attached build log).

buildlog_ubuntu-jammy-amd64.xwidgets_0.26.1-0ppa2_ubuntu22.04_BUILDING.zip

Minimal code example

No response

Error messages

In file included from /root/xwidgets/include/xwidgets/xaudio.hpp:16,
                 from /root/xwidgets/src/xaudio.cpp:1:
/root/xwidgets/include/xwidgets/xmaterialize.hpp: In instantiation of 'xw::xmaterialize<B, P>::xmaterialize(bool, A&& ...) [with A = {const std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&}; B = xw::xlayout; P = {}]':
/root/xwidgets/include/xwidgets/xmaterialize.hpp:66:55:   required from 'xw::xmaterialize<B, P>::xmaterialize(A&& ...) [with A = {const std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&}; B = xw::xlayout; P = {}]'
/usr/include/nlohmann/detail/conversions/from_json.hpp:111:7:   required from 'void nlohmann::detail::from_json(const BasicJsonType&, ConstructibleStringType&) [with BasicJsonType = nlohmann::basic_json<>; ConstructibleStringType = xw::xmaterialize<xw::xlayout>; typename std::enable_if<(nlohmann::detail::is_constructible_string_type<BasicJsonType, ConstructibleStringType>::value && (! std::is_same<typename BasicJsonType::string_t, ConstructibleStringType>::value)), int>::type <anonymous> = 0]'
/usr/include/nlohmann/detail/conversions/from_json.hpp:470:25:   required from 'decltype (nlohmann::detail::from_json(j, forward<T>(val))) nlohmann::detail::from_json_fn::operator()(const BasicJsonType&, T&&) const [with BasicJsonType = nlohmann::basic_json<>; T = xw::xmaterialize<xw::xlayout>&; decltype (nlohmann::detail::from_json(j, forward<T>(val))) = void]'
/usr/include/nlohmann/adl_serializer.hpp:33:30:   required from 'static decltype ((nlohmann::{anonymous}::from_json(forward<BasicJsonType>(j), val), void())) nlohmann::adl_serializer<T, SFINAE>::from_json(BasicJsonType&&, TargetType&) [with BasicJsonType = const nlohmann::basic_json<>&; TargetType = xw::xmaterialize<xw::xlayout>; ValueType = xw::xmaterialize<xw::xlayout>; <template-parameter-1-2> = void; decltype ((nlohmann::{anonymous}::from_json(forward<BasicJsonType>(j), val), void())) = void]'
/usr/include/nlohmann/json.hpp:3073:45:   required from 'ValueType nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType>::get_impl(nlohmann::detail::priority_tag<0>) const [with ValueType = xw::xmaterialize<xw::xlayout>; typename std::enable_if<(nlohmann::detail::is_default_constructible<ValueType>::value && nlohmann::detail::has_from_json<nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType>, ValueType>::value), int>::type <anonymous> = 0; ObjectType = std::map; ArrayType = std::vector; StringType = std::__cxx11::basic_string<char>; BooleanType = bool; NumberIntegerType = long int; NumberUnsignedType = long unsigned int; NumberFloatType = double; AllocatorType = std::allocator; JSONSerializer = nlohmann::adl_serializer; BinaryType = std::vector<unsigned char>]'
/usr/include/nlohmann/json.hpp:3216:35:   required from 'constexpr decltype (declval<const basic_json_t&>().get_impl<ValueType>(nlohmann::detail::priority_tag<4>{})) nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType>::get() const [with ValueTypeCV = xw::xmaterialize<xw::xlayout>; ValueType = xw::xmaterialize<xw::xlayout>; ObjectType = std::map; ArrayType = std::vector; StringType = std::__cxx11::basic_string<char>; BooleanType = bool; NumberIntegerType = long int; NumberUnsignedType = long unsigned int; NumberFloatType = double; AllocatorType = std::allocator; JSONSerializer = nlohmann::adl_serializer; BinaryType = std::vector<unsigned char>; decltype (declval<const basic_json_t&>().get_impl<ValueType>(nlohmann::detail::priority_tag<4>{})) = xw::xmaterialize<xw::xlayout>; nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType>::basic_json_t = nlohmann::basic_json<>]'
/root/xwidgets/include/xwidgets/xcommon.hpp:46:34:   required from 'void xw::xwidgets_deserialize(T&, const json&, const buffer_sequence&) [with T = xw::xmaterialize<xw::xlayout>; nlohmann::json = nlohmann::basic_json<>; xeus::buffer_sequence = std::vector<std::vector<char> >]'
/root/xwidgets/include/xwidgets/xtransport.hpp:37:33:   required from 'void xw::set_property_from_patch(P&, const json&, const buffer_sequence&) [with P = xp::xproperty<xw::xmaterialize<xw::xlayout>, xw::xmaterialize<xw::xaudio> >; nlohmann::json = nlohmann::basic_json<>; xeus::buffer_sequence = std::vector<std::vector<char> >]'
/root/xwidgets/include/xwidgets/xwidget.hpp:73:32:   required from 'void xw::xwidget<D>::apply_patch(const json&, const buffer_sequence&) [with D = xw::xmaterialize<xw::xaudio>; nlohmann::json = nlohmann::basic_json<>; xeus::buffer_sequence = std::vector<std::vector<char> >]'
/root/xwidgets/include/xwidgets/xmedia.hpp:68:31:   required from 'void xw::xmedia<D>::apply_patch(const json&, const buffer_sequence&) [with D = xw::xmaterialize<xw::xaudio>; nlohmann::json = nlohmann::basic_json<>; xeus::buffer_sequence = std::vector<std::vector<char> >]'
/root/xwidgets/include/xwidgets/xaudio.hpp:71:31:   required from 'void xw::xaudio<D>::apply_patch(const json&, const buffer_sequence&) [with D = xw::xmaterialize<xw::xaudio>; nlohmann::json = nlohmann::basic_json<>; xeus::buffer_sequence = std::vector<std::vector<char> >]'
/root/xwidgets/include/xwidgets/xtransport.hpp:192:45:   required from 'void xw::xtransport<D>::handle_message(const xeus::xmessage&) [with D = xw::xmaterialize<xw::xaudio>]'
/root/xwidgets/src/xaudio.cpp:10:33:   required from here
/root/xwidgets/include/xwidgets/xmaterialize.hpp:74:69: error: no matching function for call to 'xw::xlayout<xw::xmaterialize<xw::xlayout> >::xlayout(const std::__cxx11::basic_string<char>&)'
   74 |         : base_type(std::forward<A>(args)...), m_generator(generator)
      |                                                                     ^
In file included from /root/xwidgets/include/xwidgets/xobject.hpp:18,
                 from /root/xwidgets/include/xwidgets/xlayout.hpp:16,
                 from /root/xwidgets/include/xwidgets/xwidget.hpp:15,
                 from /root/xwidgets/include/xwidgets/xmedia.hpp:18,
                 from /root/xwidgets/include/xwidgets/xaudio.hpp:17,
                 from /root/xwidgets/src/xaudio.cpp:1:
/root/xwidgets/include/xwidgets/xtransport.hpp:132:12: note: candidate: 'xw::xtransport<D>::xtransport(xw::xtransport<D>&&) [with D = xw::xmaterialize<xw::xlayout>]'
  132 |     inline xtransport<D>::xtransport(xtransport&& other)
      |            ^~~~~~~~~~~~~
In file included from /root/xwidgets/include/xwidgets/xwidget.hpp:15,
                 from /root/xwidgets/include/xwidgets/xmedia.hpp:18,
                 from /root/xwidgets/include/xwidgets/xaudio.hpp:17,
                 from /root/xwidgets/src/xaudio.cpp:1:
/root/xwidgets/include/xwidgets/xlayout.hpp:65:26: note:   inherited here
   65 |         using base_type::base_type;
      |                          ^~~~~~~~~
/root/xwidgets/include/xwidgets/xlayout.hpp:65:26: note:   an inherited constructor is not a candidate for initialization from an expression of the same or derived type
In file included from /root/xwidgets/include/xwidgets/xobject.hpp:18,
                 from /root/xwidgets/include/xwidgets/xlayout.hpp:16,
                 from /root/xwidgets/include/xwidgets/xwidget.hpp:15,
                 from /root/xwidgets/include/xwidgets/xmedia.hpp:18,
                 from /root/xwidgets/include/xwidgets/xaudio.hpp:17,
                 from /root/xwidgets/src/xaudio.cpp:1:
/root/xwidgets/include/xwidgets/xtransport.hpp:124:12: note: candidate: 'xw::xtransport<D>::xtransport(const xw::xtransport<D>&) [with D = xw::xmaterialize<xw::xlayout>]'
  124 |     inline xtransport<D>::xtransport(const xtransport& other)
      |            ^~~~~~~~~~~~~
In file included from /root/xwidgets/include/xwidgets/xwidget.hpp:15,
                 from /root/xwidgets/include/xwidgets/xmedia.hpp:18,
                 from /root/xwidgets/include/xwidgets/xaudio.hpp:17,
                 from /root/xwidgets/src/xaudio.cpp:1:
/root/xwidgets/include/xwidgets/xlayout.hpp:65:26: note:   inherited here
   65 |         using base_type::base_type;
      |                          ^~~~~~~~~
/root/xwidgets/include/xwidgets/xlayout.hpp:65:26: note:   an inherited constructor is not a candidate for initialization from an expression of the same or derived type
In file included from /root/xwidgets/include/xwidgets/xobject.hpp:18,
                 from /root/xwidgets/include/xwidgets/xlayout.hpp:16,
                 from /root/xwidgets/include/xwidgets/xwidget.hpp:15,
                 from /root/xwidgets/include/xwidgets/xmedia.hpp:18,
                 from /root/xwidgets/include/xwidgets/xaudio.hpp:17,
                 from /root/xwidgets/src/xaudio.cpp:1:
/root/xwidgets/include/xwidgets/xtransport.hpp:113:12: note: candidate: 'xw::xtransport<D>::xtransport(xeus::xcomm&&, bool) [with D = xw::xmaterialize<xw::xlayout>]'
  113 |     inline xtransport<D>::xtransport(xeus::xcomm&& comm, bool owning)
      |            ^~~~~~~~~~~~~
In file included from /root/xwidgets/include/xwidgets/xwidget.hpp:15,
                 from /root/xwidgets/include/xwidgets/xmedia.hpp:18,
                 from /root/xwidgets/include/xwidgets/xaudio.hpp:17,
                 from /root/xwidgets/src/xaudio.cpp:1:
/root/xwidgets/include/xwidgets/xlayout.hpp:65:26: note:   inherited here
   65 |         using base_type::base_type;
      |                          ^~~~~~~~~
In file included from /root/xwidgets/include/xwidgets/xobject.hpp:18,
                 from /root/xwidgets/include/xwidgets/xlayout.hpp:16,
                 from /root/xwidgets/include/xwidgets/xwidget.hpp:15,
                 from /root/xwidgets/include/xwidgets/xmedia.hpp:18,
                 from /root/xwidgets/include/xwidgets/xaudio.hpp:17,
                 from /root/xwidgets/src/xaudio.cpp:1:
/root/xwidgets/include/xwidgets/xtransport.hpp:113:52: note:   no known conversion for argument 1 from 'const std::__cxx11::basic_string<char>' to 'xeus::xcomm&&'
  113 |     inline xtransport<D>::xtransport(xeus::xcomm&& comm, bool owning)
      |                                      ~~~~~~~~~~~~~~^~~~
In file included from /root/xwidgets/include/xwidgets/xlayout.hpp:16,
                 from /root/xwidgets/include/xwidgets/xwidget.hpp:15,
                 from /root/xwidgets/include/xwidgets/xmedia.hpp:18,
                 from /root/xwidgets/include/xwidgets/xaudio.hpp:17,
                 from /root/xwidgets/src/xaudio.cpp:1:
/root/xwidgets/include/xwidgets/xobject.hpp:27:11: note: candidate: 'xw::xobject<xw::xmaterialize<xw::xlayout> >::xobject(const xw::xobject<xw::xmaterialize<xw::xlayout> >&)'
   27 |     class xobject : public xtransport<D>
      |           ^~~~~~~
In file included from /root/xwidgets/include/xwidgets/xwidget.hpp:15,
                 from /root/xwidgets/include/xwidgets/xmedia.hpp:18,
                 from /root/xwidgets/include/xwidgets/xaudio.hpp:17,
                 from /root/xwidgets/src/xaudio.cpp:1:
/root/xwidgets/include/xwidgets/xlayout.hpp:65:26: note:   inherited here
   65 |         using base_type::base_type;
      |                          ^~~~~~~~~
/root/xwidgets/include/xwidgets/xlayout.hpp:65:26: note:   an inherited constructor is not a candidate for initialization from an expression of the same or derived type
In file included from /root/xwidgets/include/xwidgets/xlayout.hpp:16,
                 from /root/xwidgets/include/xwidgets/xwidget.hpp:15,
                 from /root/xwidgets/include/xwidgets/xmedia.hpp:18,
                 from /root/xwidgets/include/xwidgets/xaudio.hpp:17,
                 from /root/xwidgets/src/xaudio.cpp:1:
/root/xwidgets/include/xwidgets/xobject.hpp:27:11: note: candidate: 'xw::xobject<xw::xmaterialize<xw::xlayout> >::xobject(xw::xobject<xw::xmaterialize<xw::xlayout> >&&)'
   27 |     class xobject : public xtransport<D>
      |           ^~~~~~~
In file included from /root/xwidgets/include/xwidgets/xwidget.hpp:15,
                 from /root/xwidgets/include/xwidgets/xmedia.hpp:18,
                 from /root/xwidgets/include/xwidgets/xaudio.hpp:17,
                 from /root/xwidgets/src/xaudio.cpp:1:
/root/xwidgets/include/xwidgets/xlayout.hpp:65:26: note:   inherited here
   65 |         using base_type::base_type;
      |                          ^~~~~~~~~
/root/xwidgets/include/xwidgets/xlayout.hpp:65:26: note:   an inherited constructor is not a candidate for initialization from an expression of the same or derived type
/root/xwidgets/include/xwidgets/xlayout.hpp:139:12: note: candidate: 'xw::xlayout<D>::xlayout() [with D = xw::xmaterialize<xw::xlayout>]'
  139 |     inline xlayout<D>::xlayout()
      |            ^~~~~~~~~~
/root/xwidgets/include/xwidgets/xlayout.hpp:139:12: note:   candidate expects 0 arguments, 1 provided
In file included from /root/xwidgets/include/xwidgets/xwidget.hpp:15,
                 from /root/xwidgets/include/xwidgets/xmedia.hpp:18,
                 from /root/xwidgets/include/xwidgets/xaudio.hpp:17,
                 from /root/xwidgets/src/xaudio.cpp:1:
/root/xwidgets/include/xwidgets/xlayout.hpp:25:11: note: candidate: 'xw::xlayout<xw::xmaterialize<xw::xlayout> >::xlayout(const xw::xlayout<xw::xmaterialize<xw::xlayout> >&)'
   25 |     class xlayout : public xobject<D>
      |           ^~~~~~~
/root/xwidgets/include/xwidgets/xlayout.hpp:25:11: note:   no known conversion for argument 1 from 'const std::__cxx11::basic_string<char>' to 'const xw::xlayout<xw::xmaterialize<xw::xlayout> >&'
/root/xwidgets/include/xwidgets/xlayout.hpp:25:11: note: candidate: 'xw::xlayout<xw::xmaterialize<xw::xlayout> >::xlayout(xw::xlayout<xw::xmaterialize<xw::xlayout> >&&)'
/root/xwidgets/include/xwidgets/xlayout.hpp:25:11: note:   no known conversion for argument 1 from 'const std::__cxx11::basic_string<char>' to 'xw::xlayout<xw::xmaterialize<xw::xlayout> >&&'
make[3]: *** [CMakeFiles/xwidgets-static.dir/build.make:93: CMakeFiles/xwidgets-static.dir/src/xaudio.cpp.o] Error 1

Compiler and operating system

Ubuntu 22.04, gcc 11.2.0

Library version

v3.10.3 + 0e694, v3.10.5, develop

Validation

• The bug also occurs if the latest version from the develop branch is used.
• I can successfully compile and run the unit tests.

[falbrechtskirchinger]

I can reproduce the issue and trace it back to the same get() call in xwidgets/xcommon.hpp. I have a band-aid in mind but it might cause more breakage.

[dimitry-ishenko]

@falbrechtskirchinger is this a band-aid for json or for xwidgets? Ideally it would be for xwidgets so I don't have to package my own patched json version.

[falbrechtskirchinger]

For json. I don't see an easy way of working around it in xwidgets. Unit tests are succeeding so far.

[dimitry-ishenko]

@falbrechtskirchinger also if you don't mind, I'd like to understand what's going on in here and why it's causing the error. I have no affiliation with either of the libraries and it took me a while to figure out what I have so far.

[falbrechtskirchinger]

The issue is this xmaterialize constructor which erroneously identifies xmaterialize as a StringType.

    template <template <class> class B, class... P>
    template <class... A>
    inline xmaterialize<B, P...>::xmaterialize(A&&... args)
        : xmaterialize(false, std::forward<A>(args)...)
    {
    }

The band-aid consists of re-adding the check for a value_type member typedef/using alias. That doesn't uniquely identify StringTypes either but is better than the current state.

I've started a discussion (#3603) to try and fix issues like this one in 4.0.

[dimitry-ishenko]

@falbrechtskirchinger thank you, I understand now.

As a workaround, what if I replaced j.template get<T>() with a fully specialized version of whatever function it calls?

[dimitry-ishenko]

... or I suppose, I can roll my own version with your fresh-off-the-press patch, since you are so fast. 🥇 🥇 🥇

[falbrechtskirchinger]

You could try calling nlohmann::from_json(const nlohmann:: json &, T &) directly but I'm not sure that's going to work.

Grabbing my patch is likely going to be more fruitful.

[dimitry-ishenko]

OK I will try both. Thank you for such a fast response. 😃

[falbrechtskirchinger]

Well, unless I'm using the wrong header, it's not actually working. :-(

In that case, you'll have to wait until tomorrow.

[dimitry-ishenko]

@falbrechtskirchinger yep, I can confirm that neither of the methods worked for me.

[dimitry-ishenko]

@falbrechtskirchinger FWIW if I patch v3.10.5 with your patch works fine...

[dimitry-ishenko]

Commit 261cc4e is the other culprit that's causing failure on the develop branch:

diff --git a/include/nlohmann/detail/conversions/from_json.hpp b/include/nlohmann/detail/conversions/from_json.hpp
index 207d3e3024..cc06f198b4 100644
--- a/include/nlohmann/detail/conversions/from_json.hpp
+++ b/include/nlohmann/detail/conversions/from_json.hpp
@@ -105,13 +105,12 @@ void from_json(const BasicJsonType& j, typename BasicJsonType::string_t& s)
 }
 
 template <
-    typename BasicJsonType, typename ConstructibleStringType,
+    typename BasicJsonType, typename StringType,
     enable_if_t <
-        is_constructible_string_type<BasicJsonType, ConstructibleStringType>::value&&
-        !std::is_same<typename BasicJsonType::string_t,
-                      ConstructibleStringType>::value,
-        int > = 0 >
-void from_json(const BasicJsonType& j, ConstructibleStringType& s)
+        std::is_assignable<StringType&, const typename BasicJsonType::string_t>::value
+        && !std::is_same<typename BasicJsonType::string_t, StringType>::value
+        && !is_json_ref<StringType>::value, int > = 0 >
+void from_json(const BasicJsonType& j, StringType& s)
 {
     if (JSON_HEDLEY_UNLIKELY(!j.is_string()))
     {