There's a weird crash(?) tearing down the Renderer?

[zadjii-msft]

Carlos, Leonard and I have all hit this a few times now, in different branches. Seems like a subset of times, when dtoring the Renderer, we'll get a crash, along the stack of:

0:007> k
 # Child-SP          RetAddr               Call Site
00 00000056`fe2fe2a8 00007fff`e054a17c     ntdll!RtlFailFast2 [minkernel\ntos\rtl\amd64\misca.asm @ 51] 
01 00000056`fe2fe2b0 00007fff`e04d10d6     ntdll!RtlpHandleInvalidUserCallTarget+0x11c [minkernel\ntos\rtl\guard.c @ 2019] 
02 00000056`fe2fe310 00007fff`338dd7dc     ntdll!LdrpHandleInvalidUserCallTarget+0x46 [minkernel\ntos\rtl\amd64\cfgcheck.asm @ 284] 
03 00000056`fe2fe3d0 00007fff`33978abb     Microsoft_Terminal_Control!std::default_delete<Microsoft::Console::Render::IRenderThread>::operator()+0x5c [C:\Program Files\Microsoft Visual Studio\2022\Preview\VC\Tools\MSVC\14.31.30818\include\memory @ 3087] 
04 00000056`fe2fe420 00007fff`33b99f8b     Microsoft_Terminal_Control!std::unique_ptr<Microsoft::Console::Render::IRenderThread,std::default_delete<Microsoft::Console::Render::IRenderThread> >::reset+0x5b [C:\Program Files\Microsoft Visual Studio\2022\Preview\VC\Tools\MSVC\14.31.30818\include\memory @ 3232] 
05 00000056`fe2fe460 00007fff`3389e458     Microsoft_Terminal_Control!Microsoft::Console::Render::Renderer::~Renderer+0x4b [C:\Users\migrie\dev\public\terminal\src\renderer\base\renderer.cpp @ 57] 
06 00000056`fe2fe4a0 00007fff`3382433c     Microsoft_Terminal_Control!Microsoft::Console::Render::Renderer::`scalar deleting destructor'+0x18
07 00000056`fe2fe4d0 00007fff`3381a9ef     Microsoft_Terminal_Control!std::default_delete<Microsoft::Console::Render::Renderer>::operator()+0x3c [C:\Program Files\Microsoft Visual Studio\2022\Preview\VC\Tools\MSVC\14.31.30818\include\memory @ 3087] 
08 00000056`fe2fe510 00007fff`337c69ac     Microsoft_Terminal_Control!std::unique_ptr<Microsoft::Console::Render::Renderer,std::default_delete<Microsoft::Console::Render::Renderer> >::~unique_ptr<Microsoft::Console::Render::Renderer,std::default_delete<Microsoft::Console::Render::Renderer> >+0x4f [C:\Program Files\Microsoft Visual Studio\2022\Preview\VC\Tools\MSVC\14.31.30818\include\memory @ 3197] 
09 00000056`fe2fe550 00007fff`3383cd84     Microsoft_Terminal_Control!winrt::Microsoft::Terminal::Control::implementation::ControlCore::~ControlCore+0xfc [C:\Users\migrie\dev\public\terminal\src\cascadia\TerminalControl\ControlCore.cpp @ 203] 
0a 00000056`fe2fe580 00007fff`33847578     Microsoft_Terminal_Control!winrt::impl::heap_implements<winrt::Microsoft::Terminal::Control::implementation::ControlCore>::~heap_implements<winrt::Microsoft::Terminal::Control::implementation::ControlCore>+0x14
0b 00000056`fe2fe5b0 00007fff`33861b93     Microsoft_Terminal_Control!winrt::impl::heap_implements<winrt::Microsoft::Terminal::Control::implementation::ControlCore>::`scalar deleting destructor'+0x18
0c 00000056`fe2fe5e0 00007fff`3383b337     Microsoft_Terminal_Control!winrt::impl::root_implements<winrt::Microsoft::Terminal::Control::implementation::ControlCore,winrt::Microsoft::Terminal::Control::ControlCore,winrt::Microsoft::Terminal::Control::ICoreState>::NonDelegatingRelease+0x83 [C:\Users\migrie\dev\public\terminal\src\cascadia\TerminalControl\Generated Files\winrt\base.h @ 7385] 
0d 00000056`fe2fe640 00007fff`33866990     Microsoft_Terminal_Control!winrt::impl::root_implements<winrt::Microsoft::Terminal::Control::implementation::ControlCore,winrt::Microsoft::Terminal::Control::ControlCore,winrt::Microsoft::Terminal::Control::ICoreState>::Release+0x57 [C:\Users\migrie\dev\public\terminal\src\cascadia\TerminalControl\Generated Files\winrt\base.h @ 7267] 
0e 00000056`fe2fe680 00007fff`338a5079     Microsoft_Terminal_Control!winrt::implements<winrt::Microsoft::Terminal::Control::implementation::ControlCore,winrt::Microsoft::Terminal::Control::ControlCore,winrt::Microsoft::Terminal::Control::ICoreState>::Release+0x20 [C:\Users\migrie\dev\public\terminal\src\cascadia\TerminalControl\Generated Files\winrt\base.h @ 7860] 
0f 00000056`fe2fe6b0 00007fff`33836acb     Microsoft_Terminal_Control!winrt::com_ptr<winrt::Microsoft::Terminal::Control::implementation::ControlCore>::unconditional_release_ref+0x39 [C:\Users\migrie\dev\public\terminal\src\cascadia\TerminalControl\Generated Files\winrt\base.h @ 2545] 
10 00000056`fe2fe6f0 00007fff`3383b210     Microsoft_Terminal_Control!winrt::com_ptr<winrt::Microsoft::Terminal::Control::implementation::ControlCore>::release_ref+0x2b [C:\Users\migrie\dev\public\terminal\src\cascadia\TerminalControl\Generated Files\winrt\base.h @ 2540] 
11 00000056`fe2fe720 00007fff`338ca38b     Microsoft_Terminal_Control!winrt::com_ptr<winrt::Microsoft::Terminal::Control::implementation::ControlCore>::~com_ptr<winrt::Microsoft::Terminal::Control::implementation::ControlCore>+0x20 [C:\Users\migrie\dev\public\terminal\src\cascadia\TerminalControl\Generated Files\winrt\base.h @ 2371] 
12 00000056`fe2fe750 00007fff`338cd424     Microsoft_Terminal_Control!winrt::Microsoft::Terminal::Control::implementation::ControlInteractivity::~ControlInteractivity+0x1b
13 00000056`fe2fe780 00007fff`338ce7c8     Microsoft_Terminal_Control!winrt::impl::heap_implements<winrt::Microsoft::Terminal::Control::implementation::ControlInteractivity>::~heap_implements<winrt::Microsoft::Terminal::Control::implementation::ControlInteractivity>+0x14
14 00000056`fe2fe7b0 00007fff`338cfec3     Microsoft_Terminal_Control!winrt::impl::heap_implements<winrt::Microsoft::Terminal::Control::implementation::ControlInteractivity>::`scalar deleting destructor'+0x18
15 00000056`fe2fe7e0 00007fff`338d3877     Microsoft_Terminal_Control!winrt::impl::root_implements<winrt::Microsoft::Terminal::Control::implementation::ControlInteractivity,winrt::Microsoft::Terminal::Control::ControlInteractivity>::NonDelegatingRelease+0x83 [C:\Users\migrie\dev\public\terminal\src\cascadia\TerminalControl\Generated Files\winrt\base.h @ 7385] 
16 00000056`fe2fe840 00007fff`338d2ec0     Microsoft_Terminal_Control!winrt::impl::root_implements<winrt::Microsoft::Terminal::Control::implementation::ControlInteractivity,winrt::Microsoft::Terminal::Control::ControlInteractivity>::Release+0x57 [C:\Users\migrie\dev\public\terminal\src\cascadia\TerminalControl\Generated Files\winrt\base.h @ 7267] 
17 00000056`fe2fe880 00007fff`338d35c8     Microsoft_Terminal_Control!winrt::implements<winrt::Microsoft::Terminal::Control::implementation::ControlInteractivity,winrt::Microsoft::Terminal::Control::ControlInteractivity>::Release+0x20 [C:\Users\migrie\dev\public\terminal\src\cascadia\TerminalControl\Generated Files\winrt\base.h @ 7860] 
18 00000056`fe2fe8b0 00007fff`338a56f7     Microsoft_Terminal_Control!winrt::impl::produce_base<winrt::Microsoft::Terminal::Control::implementation::ControlInteractivity,winrt::Microsoft::Terminal::Control::IControlInteractivity,void>::Release+0x28 [C:\Users\migrie\dev\public\terminal\src\cascadia\TerminalControl\Generated Files\winrt\base.h @ 6935] 
19 00000056`fe2fe8e0 00007fff`33878feb     Microsoft_Terminal_Control!winrt::Windows::Foundation::IUnknown::unconditional_release_ref+0x57 [C:\Users\migrie\dev\public\terminal\src\cascadia\WinRTUtils\Generated Files\winrt\base.h @ 2147] 
1a 00000056`fe2fe930 00007fff`3383be20     Microsoft_Terminal_Control!winrt::Windows::Foundation::IUnknown::release_ref+0x2b [C:\Users\migrie\dev\public\terminal\src\cascadia\WinRTUtils\Generated Files\winrt\base.h @ 2142] 
1b 00000056`fe2fe960 00007fff`33c983e4     Microsoft_Terminal_Control!winrt::Windows::Foundation::IUnknown::~IUnknown+0x20 [C:\Users\migrie\dev\public\terminal\src\cascadia\WinRTUtils\Generated Files\winrt\base.h @ 2046] 
1c 00000056`fe2fe990 00007fff`338cda14     Microsoft_Terminal_Control!winrt::Windows::Foundation::IInspectable::~IInspectable+0x14
1d 00000056`fe2fe9c0 00007fff`338cd974     Microsoft_Terminal_Control!winrt::Microsoft::Terminal::Control::IControlInteractivity::~IControlInteractivity+0x14
1e 00000056`fe2fe9f0 00007fff`33940053     Microsoft_Terminal_Control!winrt::Microsoft::Terminal::Control::ControlInteractivity::~ControlInteractivity+0x14
1f 00000056`fe2fea20 00007fff`33984c04     Microsoft_Terminal_Control!winrt::Microsoft::Terminal::Control::implementation::TermControl::~TermControl+0x113 [C:\Users\migrie\dev\public\terminal\src\cascadia\TerminalControl\TermControl.cpp @ 552] 
20 00000056`fe2fea50 00007fff`33991c28     Microsoft_Terminal_Control!winrt::impl::heap_implements<winrt::Microsoft::Terminal::Control::implementation::TermControl>::~heap_implements<winrt::Microsoft::Terminal::Control::implementation::TermControl>+0x14
21 00000056`fe2fea80 00007fff`33967b53     Microsoft_Terminal_Control!winrt::impl::heap_implements<winrt::Microsoft::Terminal::Control::implementation::TermControl>::`scalar deleting destructor'+0x18
22 00000056`fe2feab0 00007fff`339ae507     Microsoft_Terminal_Control!winrt::impl::root_implements<winrt::Microsoft::Terminal::Control::implementation::TermControl,winrt::Microsoft::Terminal::Control::TermControl,winrt::Microsoft::Terminal::Control::IDirectKeyListener,winrt::Microsoft::Terminal::Control::IMouseWheelListener,winrt::Microsoft::Terminal::Control::ICoreState,winrt::composing,winrt::Windows::UI::Xaml::Controls::IControlOverrides,winrt::Windows::UI::Xaml::Controls::IControlOverrides6,winrt::Windows::UI::Xaml::IFrameworkElementOverrides,winrt::Windows::UI::Xaml::IFrameworkElementOverrides2,winrt::Windows::UI::Xaml::IUIElementOverrides,winrt::Windows::UI::Xaml::IUIElementOverrides7,winrt::Windows::UI::Xaml::IUIElementOverrides8,winrt::Windows::UI::Xaml::IUIElementOverrides9,winrt::Windows::UI::Xaml::Markup::IComponentConnector,winrt::Windows::UI::Xaml::Markup::IComponentConnector2>::NonDelegatingRelease+0x83 [C:\Users\migrie\dev\public\terminal\src\cascadia\TerminalControl\Generated Files\winrt\base.h @ 7385] 
23 00000056`fe2feb10 00007fff`339ad440     Microsoft_Terminal_Control!winrt::impl::root_implements<winrt::Microsoft::Terminal::Control::implementation::TermControl,winrt::Microsoft::Terminal::Control::TermControl,winrt::Microsoft::Terminal::Control::IDirectKeyListener,winrt::Microsoft::Terminal::Control::IMouseWheelListener,winrt::Microsoft::Terminal::Control::ICoreState,winrt::composing,winrt::Windows::UI::Xaml::Controls::IControlOverrides,winrt::Windows::UI::Xaml::Controls::IControlOverrides6,winrt::Windows::UI::Xaml::IFrameworkElementOverrides,winrt::Windows::UI::Xaml::IFrameworkElementOverrides2,winrt::Windows::UI::Xaml::IUIElementOverrides,winrt::Windows::UI::Xaml::IUIElementOverrides7,winrt::Windows::UI::Xaml::IUIElementOverrides8,winrt::Windows::UI::Xaml::IUIElementOverrides9,winrt::Windows::UI::Xaml::Markup::IComponentConnector,winrt::Windows::UI::Xaml::Markup::IComponentConnector2>::Release+0x57 [C:\Users\migrie\dev\public\terminal\src\cascadia\TerminalControl\Generated Files\winrt\base.h @ 7267] 
24 00000056`fe2feb50 00007fff`339ae0e8     Microsoft_Terminal_Control!winrt::implements<winrt::Microsoft::Terminal::Control::implementation::TermControl,winrt::Microsoft::Terminal::Control::TermControl,winrt::Microsoft::Terminal::Control::IDirectKeyListener,winrt::Microsoft::Terminal::Control::IMouseWheelListener,winrt::Microsoft::Terminal::Control::ICoreState,winrt::composing,winrt::Windows::UI::Xaml::Controls::IControlOverrides,winrt::Windows::UI::Xaml::Controls::IControlOverrides6,winrt::Windows::UI::Xaml::IFrameworkElementOverrides,winrt::Windows::UI::Xaml::IFrameworkElementOverrides2,winrt::Windows::UI::Xaml::IUIElementOverrides,winrt::Windows::UI::Xaml::IUIElementOverrides7,winrt::Windows::UI::Xaml::IUIElementOverrides8,winrt::Windows::UI::Xaml::IUIElementOverrides9,winrt::Windows::UI::Xaml::Markup::IComponentConnector,winrt::Windows::UI::Xaml::Markup::IComponentConnector2>::Release+0x20 [C:\Users\migrie\dev\public\terminal\src\cascadia\TerminalControl\Generated Files\winrt\base.h @ 7860] 
25 00000056`fe2feb80 00007fff`acd0f395     Microsoft_Terminal_Control!winrt::impl::produce_base<winrt::Microsoft::Terminal::Control::implementation::TermControl,winrt::Microsoft::Terminal::Control::ITermControl,void>::Release+0x28 [C:\Users\migrie\dev\public\terminal\src\cascadia\TerminalControl\Generated Files\winrt\base.h @ 6935] 
26 00000056`fe2febb0 00007fff`acd814a9     Windows_UI_Xaml!ctl::ComPtr<DirectUI::DependencyObject>::InternalRelease+0x35

It might be caused by #11619, but I believe Leonard was seeing this too long before that ever merged.

Running the local tests (runut /name:*local*) is a good way to hit this. That'll spawn a ton of TermControls and tear them down quickly, though, I think the actual crash will occur in the running of the subsequent test.

Since it's crashing the local tests so consistently, I figured it's best to get this tracked and fixed.

[lhecker]

I was unable to make the address sanitizer work and gave up after a while. I feel like this might be a msbuild / linking issue. As such I think this isn't a pressing issue. We don't see this crash in actual releases right?

#11889 removes the virtual base class for RenderThread which at least "hides" the issue for me. I'm waiting for @miniksa to decide on what to remove as well. This might fix similar problems in the future.

[miniksa]

I was unable to make the address sanitizer work and gave up after a while. I feel like this might be a msbuild / linking issue. As such I think this isn't a pressing issue. We don't see this crash in actual releases right?

#11889 removes the virtual base class for RenderThread which at least "hides" the issue for me. I'm waiting for @miniksa to decide on what to remove as well. This might fix similar problems in the future.

Sorry I'm looking now. I ended up down other paths today.

[lhecker]

This is likely fixed. I'll close this issue until it resurfaces.