[COMPLETED] Adding simple egui-like immediate mode UI API to Bevy UI

[PPakalns]

Combining Immediate Mode UI with Retained Mode UI

Hello, I am the author of egui_taffy and currently working on a game called Settletopia.

I believe Bevy has a unique opportunity to combine the simplicity of an immediate mode UI API (like egui) with the retained mode UI that Bevy’s ECS excels at.

---

The Problem with Building UI in Bevy

Right now, creating deep hierarchies of interactable UI nodes is complex. Common tasks like:

• Updating text every frame,
• Creating toggle buttons,
• Adding/removing UI elements interactively upon state changes

require boilerplate (extra marker components, systems, and bundles). This makes UI code harder to reason about and slows iteration.

We need simpler API for fast prototype iteration and for UI code to be more easier to reason about.

---

Introducing Immediate Mode to Bevy

Imagine if we could build interactive UI in Bevy with a single function, extended with retained mode UI where necessary.

Here is a conceptual API sketch:

fn immediate_ui_system(ui: ImmediateUICtx, world: &mut World) {
    ui.root("unique_root_id")
       .on_insert_add_bundle(|| (
             // Overwrite default UI components with custom ones
            ui::Node {
                ..default()
            },
            // Or add additional components
            BackgroundColor(Color::BLACK),
        ))
       .show(|ui| {
            // Add new UI nodes directly
            ui.build()
                .on_insert_add_bundle(|| ui::Node { ..default() })
                .label("Text");

            // Create deeper hierarchies inline
            ui.build().add(|ui| {
                ui.build().label("Text 1");
                ui.build().label("Text 2");
            });

            // Access world state (resources or queries)
            let state = world.resource_mut::<UiState>();

            // Or use SystemParam for richer state
            #[derive(SystemParam)]
            struct MyParam<'w, 's> {
                ui_state: ResMut<'w, UiState>,
                query: Query<'w, (Entity, &'static TextToList)>,
                _marker: PhantomData<&'s ()>,
            }

            let system_state = ui.world_get_state_mut::<MyParam>(world);
            let state = &mut system_state.ui_state;

            // Interactable UI nodes with retained state
            let node_ref = ui.build()
                .clickable()
                .selectable(state.selected)
                .add(|ui| {
                    ui.build().label("Toggle");
                });

            // React to interactions
            if node_ref.clicked() {
                state.selected = !state.selected;
            }

            // Dynamically add/remove UI elements based on state
            if state.selected {
                ui.build()
                    .id("my_own_id") // Improve node tracking by providing manual ID
                    .add_widget(TextEdit::new(&mut state.editable_text));
            }

            ui.label(state.editable_text);

            // Display queries directly in UI
            for (entity, text) in state.query.iter() {
                ui.build()
                    .id((entity, 1))
                    .label(&text.text);
            }

            // Escalate to retained mode for performance-critical paths
            for (entity, text) in state.query.iter() {
                ui.build()
                    .id((entity, 2))
                    .on_insert_add_bundle(|| MyTextShowBundle::new(entity))
                    .add_as_retained();
            }
       });
}
#[derive(Resource)]
struct UiState {
   selected: bool,
   editable_text: String,
}

#[derive(Component)]
struct TextToList {
   text: String,
}

This example shows how we could:

• Use simple if statements and for loops for UI branching,
• Combine immediate and retained UI,
• Access World state directly while constructing UI,
• Reduce boilerplate while keeping ECS flexibility when needed.

I am already writing code like this using egui. Would love to use similar approach in bevy, where i could fall back to fully ECS implemented widgets where needed.

---

Why Don’t We Have This Yet?

In my understanding:

• Currently most similar ui approaches in bevy use reactive approach with memoization optimizations and calculating UI differences to manage changes.
• This is partly because UI nodes are stored in the same ECS World as game data, making it difficult to simultaneously insert/remove UI entities while accessing ECS data. Therefore additional data store is needed where differences need to be stored until they can be applied.
• Immediate mode could avoid diff calculation entirely if UI entities/resources could be separated during construction.

Conceptually:

World:
[UI entities, UI resources, Data entities, Data resources]

During immediate mode:
[UI entities, UI resources]   /// split ///   [Data entities, Data resources]

This would allow immediate updates to UI while still accessing/ modifying game data.

---

Possible Approaches

1. Scoped Worlds
   Extend world.resource_scope(...) to allow something like

   world.world_scope::<UIMarker>(|ui_world: &mut World, data_world: &mut World| { ... });

   I’m unsure if this is technically feasible with Bevy’s ECS internals.

2. Separate UI World (similar to RenderWorld)

   • Have UI nodes live in a dedicated UiWorld.
   • To build widgets we need to provide ergonomic API to query both UiWorld and AppWorld from the same SystemParam. (there have been attempts to do this: Systems with parallelizable mutable access to multiple worlds. #16933)
   • UI state could then be extracted into RenderWorld as needed.
     This feels like the cleanest approach. But currently bevy tries to store everything in the main world.

3. Commands-based diffs

   • Use commands to insert/remove entities after UI construction is done.
   • Downsides: this reintroduces diff-tracking between old and new states to minimize the applied changes. Reduces API possibilities where immediate mode could directly modify these values. For example, in immediate mode we could want to update ui::Node or other component directly. Like node_ref.component_mut::<SomeComponent>().some_field = 10;

---

Related Work

• egui_taffy: demonstrates constructing and syncing a retained layout tree (taffy Styles) and egui UI nodes from immediate mode input.
• https://github.com/viridia/quill - reactive mode UI library for bevy.
• https://github.com/databasedav/haalka - ergonomic reactive Bevy UI library powered by FRP signals

and there are many others.

---

Summary

I believe combining immediate mode ergonomics with retained mode ECS power could make Bevy’s UI:

• Simpler for prototyping,
• More flexible for complex interactions,
• More ergonomic for everyday game development.

This is not a fully fleshed-out design, but rather a proposal to explore this direction. I hope this sparks discussion and inspires contributors to consider new approaches to Bevy UI.

Additional thoughts

It should be possible to even add immediate mode widgets inside retained mode widgets.

[viridia]

Working with egui, one quickly notices that the layout options are quite limited. You can have rows and columns of nodes, with different sizes, but you don't have all the layout options of flexbox or grid. The reason is that auto-sizing calculations requires two layout passes: a first pass to measure the minimum size of all elements, and a second pass to calculate the actual size.

For example, in a flexbox scenario, you first calculate the minimum size of each item, and then if there is any space left over, you distribute that extra space to each child depending on the value of their flex_grow property.

This is difficult to do in an immediate-mode UI, since there's no way to determine the minimum size of the next sibling during layout, as it hasn't been created yet. Instead, layout constraints in an immediate-mode framework tend to be strictly "top-down". Depending on your layout needs, this tends to be either extremely frustrating or it doesn't affect you at all.

[PPakalns]

Check out egui_taffy (I am the author).

In my project, I combined immediate mode with retained mode logic to get modern layout support in egui.
Under the hood it uses the taffy library — the same layout engine that Bevy UI is built on.

Egui (immediate mode) ui example with modern layout from my project:

The way it works is by constructing the egui node tree while updating the taffy node tree at the same time.
With a similar approach, we could design an immediate-mode API for Bevy as well. Bevy’s retained mode would still handle layout calculations (as it already does through ui::Node styling backed by taffy) and rendering. Immediate mode would only provide nice to use API to manage UI node insertion, removal, modification, interactions.

I actually wanted to build a proof of concept for Bevy using bevy_ui, but ran into a limitation:
it’s not currently possible to insert, remove, or modify UI node entities while constructing the UI, because UI construction needs access to the same World/Storage as the data that is being displayed.

That’s why I wrote this post — to outline what is almost possible today based on my experience with egui_taffy.

To truly implement this approach, we might need to move UI entities into a separate UiWorld. Add support for bevy_ui to be able to render entities from not only the MainWorld, but UiWorld too.

[PPakalns]

Got immediate mode working in bevy using 3rd approach.

I insert, remove entities and components using Commands and I access already inserted UI data using queries.

Video:

immediate_mode.mov

API looks very similar to suggested API, except functions are implemented as normal systems. Will try to create a crate from this.

[PPakalns]

Crate published: bevy_immediate