make_iterator throws exception after re-initializing interpreter

[bkloster-sma]

Issue description

1. Initialize an interpreter
2. Call make_iterator on some begin/end pair of a container
3. Finalize interpreter, initialize it again
4. Call make_iterator again, same as above

Result: An exception is thrown "instance allocation failed: new instance has no pybind11-registered base types".

Reproducible example code

    std::vector<int> container;

    // Initialize an interpreter
    pybind11::initialize_interpreter();

    // Call make_iterator
    {
        pybind11::gil_scoped_acquire gil;
        auto iter = pybind11::make_iterator(container.begin(), container.end());
    }

    // Finalizer interpreter, initialize it again
    pybind11::finalize_interpreter();
    pybind11::initialize_interpreter();

    // Call make iterator again
    {
        pybind11::gil_scoped_acquire gil;
        auto iter = pybind11::make_iterator(container.begin(), container.end());
    }
    pybind11::finalize_interpreter();

Proposed Fix

The culprit seems to be static state in pybind11::detail::registered_local_types_cpp(). Clearing that container (deleting the heap-allocated values in it beforehand) in finalize_interpreter() appears to fix the issue.

[bstaletic]

This does look like an actual bug. Here's a cleaner example that reproduces the error:

#include <vector>
#include <pybind11/embed.h>
#include <pybind11/stl.h>
#include <pybind11/pybind11.h>
int main() {
	std::vector<int> container;
	{
		pybind11::scoped_interpreter g;
		auto iter = pybind11::make_iterator(container.begin(), container.end());
	}
	{
		pybind11::scoped_interpreter g;
		auto iter = pybind11::make_iterator(container.begin(), container.end());
	}
}

[bobbay]

I would really like to see a fix to this one. I am depending on this feature and my users can only run once and then close my app and start over.
I tried to find the location of the Proposed Fix in the headers files but could not find "registered_local_types_cpp".

[Skylion007]

@bobbay It probably was refactored, but if it's static registered state than it's likely in get_internals()

pybind11/include/pybind11/detail/internals.h

Line 101 in 4c7e509

type_map<type_info *> registered_types_cpp; // std::type_index -> pybind11's type information

Seems like registered_types_cpp might be what was referred to.

[Skylion007]

Confirmed that the above link is correct as #2650 renamed it.