numo-narray: error: no member named 'flags' in 'struct RBasic'

[deepj]

To reproduce:

gem install numo-narray

Error:

Building native extensions. This could take a while...
ERROR:  Error installing numo-narray:
	ERROR: Failed to build gem native extension.

    current directory: ~/.gem/truffleruby/2.6.5/gems/numo-narray-0.9.1.6/ext/numo/narray
~/.rubies/truffleruby-19.3.1/bin/truffleruby -I ~/.rubies/truffleruby-19.3.1/lib/mri -r ./siteconf20200123-31672-1feb9yt.rb extconf.rb
checking for stdbool.h... yes
checking for stdint.h... yes
checking for bool in stdbool.h... yes
checking for u_int8_t in stdint.h... yes
checking for u_int16_t in stdint.h... yes
checking for int32_t in stdint.h... yes
checking for u_int32_t in stdint.h... yes
checking for int64_t in stdint.h... yes
checking for u_int64_t in stdint.h... yes
checking for exp10()... no
checking for rb_arithmetic_sequence_extract()... yes
checking for rb_cComplex... no
creating ~/.gem/truffleruby/2.6.5/gems/numo-narray-0.9.1.6/ext/numo/narray/numo/extconf.h
creating Makefile

current directory: ~/.gem/truffleruby/2.6.5/gems/numo-narray-0.9.1.6/ext/numo/narray
make "DESTDIR=" clean

current directory: ~/.gem/truffleruby/2.6.5/gems/numo-narray-0.9.1.6/ext/numo/narray
make "DESTDIR="
compiling narray.c
narray.c:141:51: warning: format specifies type 'unsigned long' but the argument has type 'VALUE' (aka 'void *') [-Wformat]
    printf("  id     = 0x%"PRI_VALUE_PREFIX"x\n", self);
                         ~~~~~~~~~~~~~~~~~~~~     ^~~~
narray.c:948:18: error: no member named 'flags' in 'struct RBasic'
    RBASIC(dst)->flags |= (RBASIC(src)->flags) &
    ~~~~~~~~~~~  ^
narray.c:948:41: error: no member named 'flags' in 'struct RBasic'
    RBASIC(dst)->flags |= (RBASIC(src)->flags) &
                           ~~~~~~~~~~~  ^
1 warning and 2 errors generated.
make: *** [narray.o] Error 1

make failed, exit code 2

Gem files will remain installed in ~/.gem/truffleruby/2.6.5/gems/numo-narray-0.9.1.6 for inspection.
Results logged to ~/.gem/truffleruby/2.6.5/extensions/x86_64-darwin/19.3.1/numo-narray-0.9.1.6/gem_make.out

[eregon]

It comes from https://github.com/ruby-numo/numo-narray/blob/50dee41baedb73f2a37927bc4d2b11591eb80d32/ext/numo/narray/narray.c#L948-L949

    RBASIC(dst)->flags |= (RBASIC(src)->flags) &
        (FL_USER1|FL_USER2|FL_USER3|FL_USER4|FL_USER5|FL_USER6|FL_USER7);

I think it's kind of deprecated to RBASIC() and access ->flags directly.
One could do instead:

    RB_FL_SET_RAW(dst, RB_FL_TEST_RAW(src,
        FL_USER1|FL_USER2|FL_USER3|FL_USER4|FL_USER5|FL_USER6|FL_USER7));

That is completely equivalent (exact same on MRI) but avoids digging in the internals of RBasic, which might change per implementation. It would also work on current TruffleRuby.
@deepj Could you submit a PR to numo-narray?

[XrXr]

Since the C code is interpreted, could it theoretically detect stores to the flags field of struct RBasic and act accordingly?

[eregon]

@XrXr Yes, that would work as long as the struct RBasic is not stored in native memory (including the stack). It shouldn't be too hard, we have similar things for rb_encoding_t, RDATA, etc in https://github.com/oracle/truffleruby/blob/master/lib/truffle/truffle/cext.rb.

In this case I was suggesting to update the code upstream because I don't think there is any reason to not use the macro. But indeed implementing (RBasic*)->flags would likely be more compatible with other gems accessing it directly. PR to implement that welcome :)

[deepj]

@eregon So what? I already have a time on this. Would I submit PR (I'm not familiar to C though) or is it better having the implementation of (RBasic*)->flags?

[eregon]

I already have a time on this.

What do you mean?

I think both can be done, but the fix to implement (RBasic*)->flags is more general, so we should do that.

[eregon]

Implementing (RBasic*)->flags would likely also fix these warnings in C++ code due to struct RBasic being empty:

In file included from truffleruby/lib/cext/include/ruby.h:33:
truffleruby/lib/cext/include/ruby/ruby.h:902:1: warning: empty struct has size 0 in C, size 1 in C++
[-Wextern-c-compat]
struct RUBY_ALIGNAS(SIZEOF_VALUE) RBasic {

[eregon]

Also happens for the allocation_tracer gem: #1978.

[jaimerave]

This issue looks also similar to #1884

[norswap]

I've implemented RBasic->flags on a branch, which solves this issue, but now numo-array fails to install with:

Fetching numo-narray-0.9.1.6.gem
Building native extensions. This could take a while...
ERROR:  Error installing numo-narray:
	ERROR: Failed to build gem native extension.

    current directory: /Users/norswap/.gem/jvm/gems/numo-narray-0.9.1.6/ext/numo/narray
/Users/norswap/work/ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/bin/truffleruby -I /Users/norswap/work/ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/mri -r ./siteconf20200414-95524-1cw3k89.rb extconf.rb
checking for stdbool.h... yes
checking for stdint.h... yes
checking for bool in stdbool.h... yes
checking for u_int8_t in stdint.h... yes
checking for u_int16_t in stdint.h... yes
checking for int32_t in stdint.h... yes
checking for u_int32_t in stdint.h... yes
checking for int64_t in stdint.h... yes
checking for u_int64_t in stdint.h... yes
checking for exp10()... no
checking for rb_arithmetic_sequence_extract()... no
checking for rb_cComplex... yes
creating /Users/norswap/.gem/jvm/gems/numo-narray-0.9.1.6/ext/numo/narray/numo/extconf.h
creating Makefile

current directory: /Users/norswap/.gem/jvm/gems/numo-narray-0.9.1.6/ext/numo/narray
make "DESTDIR=" clean

current directory: /Users/norswap/.gem/jvm/gems/numo-narray-0.9.1.6/ext/numo/narray
make "DESTDIR="
compiling narray.c
compiling array.c
compiling step.c
compiling index.c
compiling ndloop.c
compiling data.c
ruby /Users/norswap/.gem/jvm/gems/numo-narray-0.9.1.6/ext/numo/narray/gen/cogen.rb -l -o t_bit.c /Users/norswap/.gem/jvm/gems/numo-narray-0.9.1.6/ext/numo/narray/gen/def/bit.rb
compiling t_bit.c
/Users/norswap/.gem/jvm/gems/numo-narray-0.9.1.6/ext/numo/narray/gen/tmpl_bit/binary.c:119:5: error: use of undeclared identifier 'ndfunc_arg_'
    ndfunc_arg_
    ^
/Users/norswap/.gem/jvm/gems/numo-narray-0.9.1.6/ext/numo/narray/gen/tmpl_bit/binary.c:119:16: error: expected '}'
    ndfunc_arg_
               ^
/Users/norswap/.gem/jvm/gems/numo-narray-0.9.1.6/ext/numo/narray/gen/tmpl_bit/binary.c:117:1: note: to match this '{'
{
^
2 errors generated.
make: *** [t_bit.o] Error 1

make failed, exit code 2

Gem files will remain installed in /Users/norswap/.gem/jvm/gems/numo-narray-0.9.1.6 for inspection.
Results logged to /Users/norswap/.gem/jvm/extensions/x86_64-darwin/20.1.0-dev-9b59cfba/numo-narray-0.9.1.6/gem_make.out

binary.c seems to be a template file used in the generation of t_bit.c, though I'm not sure if the error is reported in that file (trying to interpret it as C, which would certainly not work but shouldn't fail so late?) or in the generated output (which does exist, and looks fine). ndfunc_arg_ doesn't even appear on that line. Needs more investigation.