[Bug] Fatal iOS 13 memory leak when using TextToSpeach.SpeekAsync

[baskren]

Description

TextToSpeech.SpeakAsync leaks significant memory when used with iOS 13. If repeated enough times, this will trigger the iOS kernel to terminate the app using TextToSpeech.SpeakAsync.

Steps to Reproduce

1. Clone demo project and open in VisualStudio Mac Enterprise.
2. Set the deployment target to Debug > iPad Pro (11-inch) iOS 12.2 simulator.
3. Build, deploy, run on simulator, and terminate app.
4. In Solution Explorer, right click on Speak_n_Leak.iOS and select Start Profiling Item
5. Upon Xamarin Profiler launch, be sure the right platform app and device are selected and then click [Next] to go to the configuration dialog.

6. In the configuration dialog, in the Common Options section, select Minimum for Level of detail and then click [Start Profiling].

7. App will start.
8. In Xamarin Profiler, note the Allocations Max value.
9. In the app, click the [Speak n Leak : n ] button.
10. Observer how much (or little) the Allocations Max increases with each SpeakAsync.
11. Click [Speak n Leak : n ] again when the counter reaches 100.
12. In Xamarin Profiler, note the Allocations Max value.
13. In Xamarin Profiler, stop the app.
14. Quit Xamarin Profiler (or else you're likely to end up running the above app in the iOS 12.2 simulator again).
15. In VisualStudio Mac, switch the deployment target to Debug > iPad Pro (11-inch) iOS 13.1 (or higher).
16. Repeat steps 3 - 13 again.

Optional: Repeat all of the above using actual iOS device. Get similar results.

Optional: Try the following (I could not get this to work on the iOS 12 build of the app):

17. Return to VisualStudio Mac, go to the menu bar, click on the ** Tools / (iOS and Mac) Instruments** to start Instruments application.
18. In Instruments, in the Choose a profiling template for dialog, select the iPad Pro (11-inch) (13.1) > Speak_n_Leak target, verify that Allocations is selected, and click [Choose].
19. Click the start button (top left in Instruments) to start profiling.
20. Click the [Speak n Leak : 0] button and click it again (to stop) when it says [Speak n Leak: 10].
21. Select the time span between the start and stop of the speaking.

Expected Behavior

• When profiling the app on a iOS 12 device (steps 3-10), the result of step 8 is 189.5MB:

• And step 12 is 198.3MB :

• The change is 8.8MB (0.088MB per SpeakAsync)

• When profiling using Instruments, expect the app to not create a bunch of live objects during speaking.

Actual Behavior

• When profiling the app on a iOS 13 device (steps 3-10), the result of step 8 is 142.4MB:

• And step 12 is 299.8MB :

• The change is 157.4MB (1.57MB per SpeakAsync)

• The result of Instruments profiling, before step 20 is 30.03MB:

• And after step 21 is 274.26MB:

**Notice the following: **
- The change in Persistent memory: 244.23MB (2.44 MB per SpeakAsync)!
- The large number of live TextToSpeachBundleSupport objects at +500KB each!

• In our production app, on actual devices, this leak leads to the following fatal crash. This is because, if leaked to 1.4GB, iOS kernel will kill the app:

default	07:04:06.960949-0500	kernel	68888.699 memorystatus: killing_specific_process pid 822 [TMCS.iOS] (per-process-limit 10) 1484807KB - memorystatus_available_pages: 14750
default	07:04:07.231722-0500	ReportCrash	starting prolongation transaction timer
default	07:04:07.231765-0500	ReportCrash	Attempting to write jetsam report
default	07:04:07.231815-0500	ReportCrash	Process TMCS.iOS [822] killed by jetsam reason per-process-limit

Basic Information

• Version with issue: all
• Last known good version: none
• IDE: VisualStudio Enterprise Mac Enterprise 8.4.5 (build 19)
• Platform Target Frameworks:
  • iOS: info.plist Deployment Target from 8.0 to 13.0
• Nuget Packages: Xamarin.Forms 4.3.0.908675
• Affected Devices: all iOS 13 devices

Screenshots

See above

Reproduction Link

https://github.com/baskren/Speak_n_Leak

[Mrnikbobjeff]

I investigated this, we just need a using block around the AVSpeechUtterance. This means we need to eagerly await our Speak function to have the using dispose correctly.

[baskren]

I actually tried this as well but it didn’t work for me - which my mean I didn’t do it right. I’ll try to share my source when I can get to my laptop.

…

Sent from my iPhone

On Feb 24, 2020, at 6:12 AM, Niklas Schilli ***@***.***> wrote:  I investigated this, we just need a using block around the AVSpeechUtterance. This means we need to eagerly await our Speak function to have the using dispose correctly. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[Mrnikbobjeff]

We also need to dispose the SpeechSynthesizer, just adding that to the finally block should do the trick.

[baskren]

Niklas Schilli (@Mrnikbobjeff)

In the hope of you showing me where I'm doing it wrong, I have added the use-using branch to the demo repo. It also leaks profusely. A few things noteworthy:

1. As you had suggested, the instance of AVSpeechUtterance is in a using block:

                using (var speechUtterance = GetSpeechUtterance(text, options))
                {
                    weakRef = new WeakReference(speechUtterance);
                    await SpeakUtterance(speechUtterance, cancelToken);
                    ...
                }

2. Unlike you had suggested, the instance of AVSpeechSynthesizer is not in a using block but rather is disposed (as well as its IDisposable properties) right before either of the tcsUtterance?.TrySetResult(true); calls are made:

            void TryCancel()
            {
                speechSynthesizer?.StopSpeaking(AVSpeechBoundary.Word);
                Dispose();
                tcsUtterance?.TrySetResult(true);
            }

            void OnFinishedSpeechUtterance(object sender, AVSpeechSynthesizerUteranceEventArgs args)
            {
                if (speechUtterance == args.Utterance)
                {
                    Dispose();
                    tcsUtterance?.TrySetResult(true);
                }
            }

            void Dispose()
            {
                if (!disposed && disposeWhenDone)
                {
                    disposed = true;
                    speechSynthesizer.Delegate?.Dispose();
                    speechSynthesizer.Delegate = null;
                    speechSynthesizer.WeakDelegate?.Dispose();
                    speechSynthesizer.WeakDelegate = null;
                    speechSynthesizer.Dispose();
                    System.Diagnostics.Debug.WriteLine("SYNTHESIZER DISPOSED");
                }

                GC.Collect();
                GC.WaitForPendingFinalizers();
                GC.Collect();
                if (weakRef.IsAlive)
                    System.Diagnostics.Debug.WriteLine("[" + DateTime.Now.ToString("yyyy’-‘MM’-‘dd’T’HH’:’mm’:’ss.fffffffK") + "] PlatformRenderer.SpeakUtterance: text=[" + text + "]  weakRef.IsAlive  speechSynthesizer.RetainCount=[" + speechSynthesizer.RetainCount + "]");
            }

3. In both of the above, I have made a weak reference to the NSObject in question so that I can test if it's still alive after a garbage collection. If they are still alive, then I check their NSObject RetainCount value. This is where things get interesting (or I'm clueless, which is not unlikely). In both cases, the weak references are still alive but their RetainCount values are zero.

4. May not be relevant, but the console is very noisy with each call to SpeakAsync:

2020-02-24 15:32:04.893249-0500 Speak_n_Leak.iOS[61854:1165125] Got the query meta data reply for: com.apple.MobileAsset.VoiceServicesVocalizerVoice, response: 2
2020-02-24 15:32:04.893494-0500 Speak_n_Leak.iOS[61854:1165125] [AXTTSCommon] Error running custom voice query XML not present
2020-02-24 15:32:04.921808-0500 Speak_n_Leak.iOS[61854:1165125] Got the query meta data reply for: com.apple.MobileAsset.VoiceServices.VoiceResources, response: 2
2020-02-24 15:32:04.921994-0500 Speak_n_Leak.iOS[61854:1165125] [AXTTSCommon] Error running query for voice resource asset: Voice resource, Languages: en-US, ContentVersion: (null), MasteredVersion: (null), error: XML not present
2020-02-24 15:32:04.924235-0500 Speak_n_Leak.iOS[61854:1164962] Got the query meta data reply for: com.apple.MobileAsset.VoiceServices.VoiceResources, response: 2
2020-02-24 15:32:05.156007-0500 Speak_n_Leak.iOS[61854:1164962] Got the download meta data reply for com.apple.MobileAsset.VoiceServicesVocalizerVoice, response: 3
2020-02-24 15:32:05.406068-0500 Speak_n_Leak.iOS[61854:1164962] Got the download meta data reply for com.apple.MobileAsset.VoiceServices.VoiceResources, response: 3

Any insights you may have would be greatly appreciated!

[Mrnikbobjeff]

If you tested said code in Debug mode it will never collect the objects as they are still live in Debug configurations. That is just a peculiarity to improve debug experience, all variables in the local call frame will be kept alive until the block is exited.
That being said, I would ask to use the Dispose(true) overload.

[baskren]

Niklas Schilli (@Mrnikbobjeff), thank you for being willing to engage me on this. Greatly appreciated!

Niklas Schilli (@Mrnikbobjeff) said: If you tested said code in Debug mode it will never collect the objects as they are still live in Debug configurations. That is just a peculiarity to improve debug experience, all variables in the local call frame will be kept alive until the block is exited.

Sounds reasonable ... unfortunately I cannot get Xamarin.Profiler to start profiling a release build of this small app. When I try to start the app, Xamarin.Profiler is stuck on "Starting application ...". When I stop profiling, Xamarin.Profiler presents the message:

An error has occurred Cannot access a disposed object. Object name:System.Net.Sockets.Socket.

Niklas Schilli (@Mrnikbobjeff) said: That being said, I would ask to use the Dispose(true) overload.

Maybe I'm missing something ... there doesn't appear to be a public void Dispose(bool disposed) method for any of the objects in question:

• AVSpeechUtterance
• AVSpeechSynthesisVoice
• NSAttributedString
• AVSpeechSynthsizer

[baskren]

Niklas Schilli (@Mrnikbobjeff)

Just tried profiling the demo app, in a release build, in Instruments.

The good news: Unlike Xamarin.Profiler, Instruments was able to profile the app.

The bad news: It still leaks.

[Mrnikbobjeff]

Well the easy fix for us would be to cache the AvSpeechSynthesizer and see if this solves the leak. I had another Idea I would like to test if it still leaks if you actually set a valid voice. Currently it is only set if there is an options object being passed. Perhaps then it does not have to do whatever it does. You actually also did not identify the object being allocated, if you check the column it reads resposible library

[baskren]

Niklas Schilli (@Mrnikbobjeff)

Caching AVSpeechSynthesizer works!

For our application, this appears to be an acceptable short term solution in that we already have a queue in place upstream - which should prevent multiple concurrent subscribers to speechSynthesizer.DidFinishSpeechUtterance. Out in the wild, I suspect it would be possible to get multiple concurrent subscribers to speechSynthesizer.DidFinishSpeechUtterance.

You actually also did not identify the object being allocated, if you check the column it reads resposible library

Not sure I'm following you here and I would like to be sure I understand. Could you be referring to Xamarin.Profiler or Instruments? Can you elaborate?

[baskren]

FWIW:

• I've put the AVSpeechCaching changes in a new branch of the demo project.

• There is still a small leak (a bit over 2MB over 100 calls to SpeakAsync). For our purposes, I still call this a victory.

[jamesmontemagno]

Ben Askren (@baskren) been following the thread and thanks for looking into this. Do you want to send a PR down for review?

[Mrnikbobjeff]

Ben Askren (@baskren) In your initial issue post you wrote

The large number of live TextToSpeachBundleSupport objects at +500KB each!

If you check the last image you posted (which is the only time that string is ever visible), you can see that the column header reads "responsible library". You assumed that the name of the Objekt being leaked is displayed there, but that is just the offending library.