Crash due to TextureRect and RichTextLabel causing a size-fitting infinite loop with certain Containers

[Lattency]

Godot version

4.1.1

System information

Godot v4.1.1.stable - Windows 10.0.22621 - Vulkan (Mobile) - dedicated NVIDIA GeForce GTX 1660 Ti (NVIDIA; 31.0.15.3667) - AMD Ryzen 7 4800H with Radeon Graphics (16 Threads)

Issue description

While experimenting with Control nodes as a new user, I encountered a crash that occurs if a Margin or Panel Container has a TextureRect with Expand Mode set to "Fit Width" and a RichTextLabel with "Fit Content" turned on as child nodes.

I am not 100% sure, but my guess is that if enough text is entered into the RichTextLabel, it will try to wrap the text around and consequently increase the Container's height. The TextureRect increases in height to fill the new Container's height, but also increases width due to its expansion settings. The RichTextLabel fills to the Container's resulting new width and, while fitting the content, ends up decreasing its height. At this point, the interaction between the two nodes is enough to cause an infinite loop, causing the editor to crash.

The following is the error generated in the logs, which seems indicative of an infinite loop:

USER ERROR: Failed method: CanvasItem::_redraw_callback. Message queue out of memory. Message queue out of memory. Try increasing 'memory/limits/message_queue/max_size_mb' in project settings.
   at: push_callablep (core/object/message_queue.cpp:96)

Further testing suggests that the loop relies on specific conditions that allows this size-fitting behavior to occur. It doesn't seem to happen for other Containers such as Box Containers, where child and parent size are not equally related as in Margin or Panel Containers. It also doesn't happen if fitting or filling behaviors are set in a way that logically leaves the loop incomplete (ex. it runs perfectly fine if TextureRect's Expand Mode is set to "Fit Height" instead).

Steps to reproduce

1. Create a MarginContainer or PanelContainer.
2. Add a TextureRect with Expand Mode set to "Fit Width" as a child to the container, and assign a texture to it (the default icon.svg is enough for this)
3. Add a RichTextLabel with "Fit Content" checked/turned on as a child to the container.
4. Start writing text into the RichTextLabel until it causes the Container to expand. This should trigger the loop, causing the editor to hang and eventually crash.

Minimal reproduction project

N/A.

[vnznznz]

Reproduced this issue on Godot 4.3-dev3

OpenGL API 4.6 (Core Profile) Mesa 23.3.6 - Compatibility - Using Device: AMD - AMD Radeon RX 6750 XT (radeonsi, navi22, LLVM 17.0.6, DRM 3.57, 6.7.4-200.fc39.x86_64)
 
Failed method: CanvasItem::_redraw_callback. Message queue out of memory. Message queue out of memory. Try increasing 'memory/limits/message_queue/max_size_mb' in project settings.

================================================================
Failed method: EditorNode::_print_handler_impl. Message queue out of memory. Message queue out of memory. Try increasing 'memory/limits/message_queue/max_size_mb' in project settings.

Open Repro project -> main.tscn -> Resize Panel Node for a bit -> Godot Engine disappears.

Screencast from 2024-02-27 09-28-32.webm
Repro82033_CallbackQueue_TextureRect.zip

[paulmiller]

I have a repro without RichTextLabel, just TextureRects and Containers.

Setting the selected TextureRect to "Fit Height" or "Fit Height Proportional" crashes the editor with the same "Message queue" error. Using Godot_v4.2.2-stable_linux.x86_64.

Repro project: test.zip

[paulmiller]

@KoBeWi, who wrote #58517

[KoBeWi]

Looks like duplicate of #73071
The issue got some partial fixes for specific containers.