CompositionTarget.Rendering event slows down when there are no UI updates

[dotMorten]

• .NET Core Version:3.0.100-rc1-014190
• Windows version: 1903
• Does the bug reproduce also in WPF for .NET Framework 4.8?: Yes
• Is this bug related specifically to tooling in Visual Studio (e.g. XAML Designer, Code editing, etc...)? No

Problem description:
When relying on CompositionTarget.Rendering, I've noticed that the event throttles down to around 50fps when there are no UI updates. As soon as UI updates are occuring, the event goes back up to firing at 60fps.

This becomes problematic when you're doing performance metrics and want to ensure performance stays a consistent 60fps, but when things stops moving, it shows in the performance numbers as really bad performance.

Notice here where no UI updates are done, the framerate is around 50fps. Once animation is started, the framerate goes up to 60:

Actual behavior:
Clock timer slows down.

Expected behavior:
Event sticks to 60fps

Minimal repro:
Use the toggle button to turn the animation on and off. Monitor the output window for framerate updates every 0.5 seconds.

MainWindow.xaml:

<Window x:Class="FrameRateTest.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        xmlns:local="clr-namespace:FrameRateTest"
        mc:Ignorable="d"
        Title="MainWindow" Height="450" Width="800">
        <Window.Resources>
            <Storyboard x:Key="sb" RepeatBehavior="Forever">
               <DoubleAnimation Storyboard.TargetName="tt" 
                       Storyboard.TargetProperty="X" From="-250" To="250" 
                       Duration="0:0:5" BeginTime="0:0:0"/>
               <DoubleAnimation Storyboard.TargetName="tt" 
                       Storyboard.TargetProperty="X" From="250" To="-250" 
                       Duration="0:0:5" BeginTime="0:0:5"/>
            </Storyboard>
        </Window.Resources>

    <Grid>
        
        <Button HorizontalAlignment="Left" VerticalAlignment="Top" Content="Toggle Animation" Click="ToggleAnimation_Click" />
        
        <Ellipse Width="30" Height="30" Fill="Red" HorizontalAlignment="Center" VerticalAlignment="Center">
           <Ellipse.RenderTransform>
              <TranslateTransform x:Name="tt" />
           </Ellipse.RenderTransform>
        </Ellipse>
    </Grid>
    
</Window>

MainWindow.xaml.cs:

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Windows;
using System.Windows.Media;
using System.Windows.Media.Animation;

namespace FrameRateTest
{
    /// <summary>
    /// Interaction logic for MainWindow.xaml
    /// </summary>
    public partial class MainWindow : Window
    {
        public MainWindow()
        {
            InitializeComponent();
            CompositionTarget.Rendering += OnRender;
            stopwatch.Start();
        }

        Stopwatch stopwatch = new Stopwatch();
        TimeSpan lastFrame = TimeSpan.Zero;
        MovingAverage avr = new MovingAverage(60);
        int frame = 0;
        public void OnRender(object sender, EventArgs e)
        {
            frame++;
            var ts = stopwatch.Elapsed;
            avr.AddSample((ts - lastFrame).TotalMilliseconds);
            lastFrame = ts;
            if (frame % 30 == 0)
            {
                Debug.WriteLine((1000 / avr.Average).ToString("0.000"));
            }
        }
        bool isAnimating;
        public void ToggleAnimation_Click(object sender, RoutedEventArgs e)
        {
            var sb = (Storyboard)Resources["sb"];
            if (isAnimating)
                sb.Begin();
            else
                sb.Stop();
            isAnimating = !isAnimating;

        }
    }

    public class MovingAverage
    {
        private Queue<double> _samples;
        private int _windowSize;
        private double _sum;

        public double Average
        {
            get
            {
                int cnt = _samples.Count;
                return cnt != 0 ? _sum / cnt : double.NaN;
            }
        }

        public MovingAverage(int windowSize)
        {
            _windowSize = windowSize;
            _samples = new Queue<double>(windowSize);
        }

        public void AddSample(double newSample)
        {
            if (!double.IsNaN(newSample))
            {
                if (_samples.Count == _windowSize)
                {
                    _sum -= _samples.Dequeue();
                }
                _samples.Enqueue(newSample);
                _sum += newSample;
            }
        }
    }
}

[weltkante]

I do not agree with the expected behavior, WPF can be in a rendering mode where it only updates dirty areas when animations are off, if there's nothing dirty it shouldn't be rendering with 60 FPS. Also WPF shouldn't force the system into multimedia mode (more precision in timers but potentially higher power usage due to waking up the CPU more frequently) when its just displaying UI without animations. The observed behavior could very well just be that the system exits multimedia mode and has lower timer precision (of course thats just one possibility, it could also have other causes).

Generally I'd also expect it to render faster than 60 FPS when the display refresh rate is different (if it doesn't do this now this is certainly something to be expected to change in the future). I'd expect the rendering speed to be only limited by vsync (rendering faster than the system updates the display makes no sense). If vsync is slower than 60 fps for some reason, rendering should also be slower.

So basically, don't use CompositionTarget.Rendering to do timing for you, that isn't what its semantics are.

[randomascii]

I think that the real bug is that the event rate doesn't slow to 0 fps when there are no UI updates. It has been a persistent complaint about WPF that it leaves the Windows timer interrupt raised even when it is doing nothing. It should return the timer interrupt frequency to normal and then wait for something to happen. This avoids wasteful interrupts, and avoids wasteful wakeups, allowing WPF programs to be power efficient when idle. Profiling can be done when UI updates are active. Some relevant blog posts:
On the importance of not raising the timer interrupt frequency:
https://randomascii.wordpress.com/2013/07/08/windows-timer-resolution-megawatts-wasted/
On the importance of going completely idle when there's no work to be done
https://randomascii.wordpress.com/2016/03/08/power-wastage-on-an-idle-laptop/

If WPF apps don't go 100% idle then I am likely to uninstall them.

[dotMorten]

I do not agree with the expected behavior, WPF can be in a rendering mode where it only updates dirty areas when animations are off, if there's nothing dirty it shouldn't be rendering with 60 FPS

That's not how CompositionTarget.Rendering event works. Once you start listening to this, it constantly runs the timer (which is why they only recommend listening to it when necessary). That's what it's supposed to do, and how you drive a frame-synced native DirectX Rendering thread (but it slows down if that native rendering thread is being smart and not performing updates where none is needed).

I'd expect the rendering speed to be only limited by vsync

There is code in there indicating that's exactly what is happening, but without MILCore open sourced, it's tough to say if that's really what's happening at the DX Level.

If vsync is slower than 60 fps for some reason, rendering should also be slower.

Of course. But it's 60fps on this PC. In remote desktop it (correctly) slows to 30fps. If the frame rendering takes too long (ie > 16ms) it slows even more.

I think that the real bug is that the event rate doesn't slow to 0 fps when there are no UI updates

No. Again that's not what this event is for, and would be a major breaking behavior (you should stop listening for this event instead if you're done updating UI)

It has been a persistent complaint about WPF that it leaves the Windows timer interrupt raised even when it is doing nothing

Hence the recommendation to not listen for this event when you don't need it.

If WPF apps don't go 100% idle then I am likely to uninstall them.

Just running a timer is almost no work at all, as long as you don't actually perform a render-pass, which is exactly what is happening in this case, and why this problem appears. My native renderer is smart enough to realize that nothing changed, and decides to not perform an expensive UI Update.

Both answers above are completely besides the point, and gives no indication why there's a 10fps drop in this event.

And yes I've tried driving the rendering pulse on a separate thread, but the only way to get decent rendering performance is to lock the D3DImage during this event - locking at any other point in time gives the D3DImage surface lock times varying between 2 and 20ms, causing lots of dropped frames. Of cause running on this event causes your custom native rendering to now also affect the main rendering thread if your rendering gets heavy and slows things down, and makes touch interaction especially poor. This is why I'm running a second (!) UI thread just for rendering my native content. Apart from the complexity this adds, it provides the best DirectX performance you can squeeze out of D3DImage. So yes I've been very deep down the WPF Performance rabbit hole. This event is the best one to use for that, and gives me smooth 60fps performance. The weird thing is just I don't get 60fps when I try and be smart and do no work. This is problematic, because our performance test benchmarks will show that we are dropping frames when idling, when in fact there's this mysterious slow-down when we're going into idle-mode, which definitely should not be the case.

[weltkante]

You are right, registering a CompositionTarget.Rendering is apparently treated as if an animation is running here. I got the semantics wrong and thats my mistake, sorry. That still doesn't change the rest of what I explained.

Both answers above are completely besides the point, and gives no indication why there's a 10fps drop in this event.

You skipped past the line where I explained that the system is probably not in multimedia mode, in normal operation the timers have a noticeably lower precision. I've run into this problem in the past and I think I once had the line where WPF ensured that high precision timers were used with animations, I can try and check if I find it again.

If I'm wrong with my suspicion that the multimedia mode is the cause of the problem thats ok, but no need to be aggressive about it. You already agreed with that WPF should not necessarily render in fixed 60 FPS but depend on system parameters like monitor refresh rate, which was not clear from your issue description and which was half of my concern when saying that I don't agree with the expectation is to always render at 60 FPS.

[randomascii]

Thanks for the explanation. In particular it may be that my unhappiness with WPF is from Visual Studio listening to this event - who knows.

[dotMorten]

no need to be aggressive about it.

Sorry didn't mean to come off like that. I see now how it could be read like that. I think I got a little frustrated being told I shouldn't be using something that is being used for what it's meant for, rather than focusing on the specific issue and how we can address it.

[weltkante]

Can't reconstruct my findings where WPF changes behavior when animations are enabled, might have involved setting breakpoints at native APIs and looking at the stack trace back when I debugged that years ago. Anyways, I've done some tests calling timeBeginPeriod (which is one way to force higher timer precision, there are other APIs which have the same effect but I can never remember them) but this does not seem to affect the measurements you do, so this is not the same effect I was seeing back when I debugged something similar.

You're probably right that this needs access to the render engine source for further debugging.

[fabiant3-zz]

In general high precision multimedia timers are prone to cause battery issues, so there's a good chance they are not used in WPF code anymore.
Given this is not a regression, I'm moving this issue to Future for now, once the rendering engine code is being open sourced, we can spend more time debugging though this issue. Thanks for the report @dotMorten.

[reinux]

Is there a workaround for this?
I'd like to use WPF for an application which isn't a game but requires smooth animation.

[weltkante]

In general high precision multimedia timers are prone to cause battery issues, so there's a good chance they are not used in WPF code anymore.

@fabiant3-zz apparently multimedia timers are still used and cause energy report warnings in WPF apps, see #2970

[randomascii]

Visual Studio uses WPF and when Visual Studio is idle it frequently triggers energy report warnings due to WPF leaving the timer interrupt frequency raised.

It is unfortunate that Microsoft seems to have abandoned WPF and left it in a state where it violates Microsoft's own guidelines for dealing with the system-global timer interrupt frequency.

[blakepell]

Thanks for sharing @dotMorten. In a personal project I'm writing with my kids I used an animation on a 1x1 rectangle that kept the framerate at 59-60fps consistently (I'm not writing professional grade software, so it totally works at the expense of a small amount of GPU). WPF seems to be smart enough to negate this unless I had it somewhere on the visible surface.

To keep the framerate from spiking well above 60fps at times I used the RenderingEventArgs.RenderingTime to make sure it was different than the last rendering time.

When I switch to my Surface Pro 9 that has a 120hz refresh rate it drops to 30 fps with or without the governor (so mileage will vary, it doesn't seem to be a resource bottleneck though).

For posterity here's the code I used in case it's useful for anyone else:

    /// <summary>
    /// Contains event handler for the <see cref="CompositionTarget"/> Rendering method that
    /// will put a governor on the Rendering event and require that the <see cref="RenderingEventArgs"/>
    /// RenderingTime is different than the last time it was fired (this seems to have the effect of
    /// keeping rendering firing at 59-60fps).
    /// </summary>
    public static class CompositionTargetRendering
    {
        /// <summary>
        /// The RenderingTime of the last CompositionTarget_Rendering event.
        /// </summary>
        private static TimeSpan _lastRender = TimeSpan.Zero;

        private static event EventHandler<RenderingEventArgs>? _frameUpdating;

        public static event EventHandler<RenderingEventArgs> FrameUpdating
        {
            add
            {
                if (_frameUpdating == null)
                {
                    CompositionTarget.Rendering += CompositionTarget_Rendering;
                }

                _frameUpdating += value;
            }

            remove
            {
                _frameUpdating -= value;

                if (_frameUpdating == null)
                {
                    CompositionTarget.Rendering -= CompositionTarget_Rendering;
                }
            }
        }

        /// <summary>
        /// Rendering function that fires (hopefully) 60 times a second.
        /// </summary>
        /// <param name="sender"></param>
        /// <param name="e"></param>
        private static void CompositionTarget_Rendering(object? sender, EventArgs e)
        {
            if (e is RenderingEventArgs args && args.RenderingTime != _lastRender)
            {
                _lastRender = args.RenderingTime;
                _frameUpdating?.Invoke(sender, args);
            }
        }
    }