box-shadow clipping fix

[ickshonpe]

Objective

Instead of clipping the non-visable sections of box-shadows, the shadow is scaled to fit into the remaining area after clipping because the normalized coordinates that are meant to border the unclipped subsection of the shadow are always set to [Vec2::ZERO, Vec2::X, Vec2::ONE, Vec2::Y],

Solution

Calculate the coordinates for the corners of the visible area.

Testing

Test app:

use bevy::color::palettes::css::RED;
use bevy::color::palettes::css::WHITE;
use bevy::prelude::*;

fn main() {
    App::new()
        .add_plugins(DefaultPlugins)
        .add_systems(Startup, setup)
        .run();
}

fn setup(mut commands: Commands) {
    commands.spawn(Camera2d);
    commands
        .spawn(Node {
            ..Default::default()
        })
        .with_children(|commands| {
            commands
                .spawn((
                    Node {
                        width: Val::Px(100.),
                        height: Val::Px(100.),
                        margin: UiRect {
                            left: Val::Px(100.),
                            top: Val::Px(300.),
                            ..Default::default()
                        },
                        overflow: Overflow::clip(),
                        ..Default::default()
                    },
                    BackgroundColor(WHITE.into()),
                ))
                .with_children(|commands| {
                    commands.spawn((
                        Node {
                            position_type: PositionType::Absolute,
                            left: Val::Px(50.),
                            top: Val::Px(50.),
                            width: Val::Px(100.),
                            height: Val::Px(100.),
                            ..Default::default()
                        },
                        BackgroundColor(RED.into()),
                        BoxShadow::from(ShadowStyle {
                            x_offset: Val::ZERO,
                            y_offset: Val::ZERO,
                            spread_radius: Val::Px(50.),
                            blur_radius: Val::Px(6.),
                            ..Default::default()
                        }),
                    ));
                });
        });
}

Main:

This PR:

[UkoeHB]

Btw possibly related to this PR, this line looks odd to me:

let positions = QUAD_VERTEX_POSITIONS
    .map(|pos| (box_shadow.transform * (pos * rect_size).extend(1.)).xyz());

Shouldn't it be transform + (pos * rect_size)?

[ickshonpe]

Btw possibly related to this PR, this line looks odd to me:

let positions = QUAD_VERTEX_POSITIONS
    .map(|pos| (box_shadow.transform * (pos * rect_size).extend(1.)).xyz());

Shouldn't it be transform + (pos * rect_size)?

transform is a 4x4 matrix so addition doesn't work here, to apply the matrix transformation we have to extend the scaled position to a Vec4 and multiply. The matrix here normally represents a translation though in which case it does all cancel out to just an addition between two Vec3s.