handle case where cpp11_preserve_xptr is a null pointer

[kevinushey]

RStudio attempts to save and load options after session restarts (e.g. after installing and reloading an R package), leading to this state:

> getOption("cpp11_preserve_xptr")
<pointer: 0x7faf18c1c450>

Restarting R session...

Using libraries at paths:
- /Users/kevinushey/Library/R/4.0/library
- /Library/Frameworks/R.framework/Versions/4.0/Resources/library
> library(euclid)
> getOption("cpp11_preserve_xptr")
<pointer: 0x0>

This can unfortunately cause issues for packages which try to use cpp11, as (if I understand correctly) cpp11 assumes that this is a valid + live pointer. For example:

(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x28)
  * frame #0: 0x00000001094b4554 libR.dylib`CDR(e=0x0000000000000000) at memory.c:4124:29 [opt]
    frame #1: 0x000000010d9f3f30 euclid.so`cpp11::$_0::insert(this=<unavailable>, obj=0x00007fe5c10803e8) at protect.hpp:227:17 [opt]
    frame #2: 0x000000010d9f3e65 euclid.so`cpp11::r_vector<double>::r_vector(this=0x00007ffee8285148, data=0x00007fe5c10803e8) at r_vector.hpp:362:26 [opt]
    frame #3: 0x000000010d9d1c20 euclid.so`::_euclid_create_exact_numeric(SEXP) [inlined] cpp11::r_vector<double>::r_vector(this=<unavailable>, data=<unavailable>) at r_vector.hpp:365:33 [opt]
    frame #4: 0x000000010d9d1c18 euclid.so`::_euclid_create_exact_numeric(SEXP) [inlined] std::__1::enable_if<((!(is_smart_ptr<cpp11::r_vector<double> >::value)) && (std::is_class<cpp11::r_vector<double> >::value)) && (std::is_constructible<cpp11::r_vector<double>, SEXPREC*>::value), cpp11::r_vector<double> >::type cpp11::as_cpp<cpp11::r_vector<double> >(from=<unavailable>) at as.hpp:82 [opt]
    frame #5: 0x000000010d9d1c18 euclid.so`::_euclid_create_exact_numeric(numeric=<unavailable>) at cpp11.cpp:403 [opt]
    frame #6: 0x000000010942161c libR.dylib`R_doDotCall(ofun=<unavailable>, nargs=<unavailable>, cargs=0x00007ffee8287620, call=0x00007fe5f1311598) at dotcode.c:598:17 [opt]
< ... >

(lldb) f 1
frame #1: 0x000000010d9f3f30 euclid.so`cpp11::$_0::insert(this=<unavailable>, obj=0x00007fe5c10803e8) at protect.hpp:227:17 [opt]
   224      PROTECT(obj);
   225
   226      // Add a new cell that points to the previous end.
-> 227      SEXP cell = PROTECT(Rf_cons(list_, CDR(list_)));
   228
   229      SET_TAG(cell, obj);
   230
(lldb) f 2
frame #2: 0x000000010d9f3e65 euclid.so`cpp11::r_vector<double>::r_vector(this=0x00007ffee8285148, data=0x00007fe5c10803e8) at r_vector.hpp:362:26 [opt]
   359  template <typename T>
   360  inline r_vector<T>::r_vector(const SEXP data)
   361      : data_(valid_type(data)),
-> 362        protect_(preserved.insert(data)),
   363        is_altrep_(ALTREP(data)),
   364        data_p_(get_p(ALTREP(data), data)),
   365        length_(Rf_xlength(data)) {}

Would cpp11 also be willing to handle this case where cpp11_preserve_xptr is not a valid + live external pointer, or should RStudio just avoid serializing + deserializing this pointer?

[kevinushey]

For what it's worth, a somewhat similar reproducible example:

library(cpp11)

options(cpp11_preserve_xptr = "i am a bad person")

cpp_source(
  code = '#include "cpp11/integers.hpp"

  [[cpp11::register]]
  int num_odd(cpp11::integers x) {
    int total = 0;
    for (int val : x) {
      if ((val % 2) == 1) {
        ++total;
      }
    }
    return total;
  }
  ')

num_odd(as.integer(c(1:10, 15, 23)))

will give a crash.