iPad sometimes crashes with ?fuzz

[Nancy-Salpepi]

Test device
iPad 9th generation

Operating System
17.5.1

Browser
safari

Problem description
For phetsims/qa#1107, when I add ?fuzz to the end of the url, the sim crashed in 3/5 attempts. Two of those occurred with 30 seconds and the third happened after about 3 minutes. Twice I saw no crash during 5 minutes of fuzzing.

I don't see any issues on the mac.

Steps to reproduce
Add ?fuzz to the sim url

Troubleshooting information:
!!!!! DO NOT EDIT !!!!!
Name: ‪Gas Properties‬
URL: https://phet-dev.colorado.edu/html/gas-properties/1.1.0-rc.1/phet/gas-properties_all_phet.html
Version: 1.1.0-rc.1 2024-07-11 00:18:26 UTC
Features missing: applicationcache, applicationcache
Flags: pixelRatioScaling
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.5 Safari/605.1.15
Language: en-US
Window: 810x977
Pixel Ratio: 2/1
WebGL: WebGL 1.0
GLSL: WebGL GLSL ES 1.0 (1.0)
Vendor: WebKit (WebKit WebGL)
Vertex: attribs: 16 varying: 31 uniform: 1024
Texture: size: 16384 imageUnits: 16 (vertex: 16, combined: 32)
Max viewport: 16384x16384
OES_texture_float: true
Dependencies JSON: {}

[arouinfar]

I can't reproduce on iPad Pro 11" (4th gen) running on iPad OS 17.5.1/Safari. I made 4 attempts, letting the sim run for 5+ minutes each time -- no crashes here.

[pixelzoom]

Tested on my iPad (6th gen) + iPadOS 17.5.1 + Safari. I could not reproduce with 5 attempts, each for 5+ minutes.

Heap size stablizes at ~140MB, so this is unlikely to be an out-of-memory problem.

@arouinfar and I both tested on older iPads (2020 and 2018 respectively), while @Nancy-Salpepi observed this problem on a newer iPad (9th gen, 2021). So it's possible that there's something different that's occurring on newer iPads. @KatieWoe Does QA have another newer iPad (not the same device that @Nancy-Salpepi used) that you could test with?

[KatieWoe]

I haven't managed to reproduce it on my iPad Pro. Sorry. I'm going to try to get into the office tomorrow or Wednesday, and if I find something that does show the issue I'll let you know.

@pixelzoom edit: @KatieWoe's iPad Pro is generation 1 (2017) with iPadOS 17.5.1.

[arouinfar]

@arouinfar and I both tested on older iPads (2020 and 2018 respectively)

Small correction -- my iPad Pro 11" 4th gen was released in 2022.

[pixelzoom]

Actually... a big correction. Apple apparently uses different generation numbering for the 11" iPad Pro model, and @arouinfar's iPad was indeed released in 2022. That means that her device is actually newer than @Nancy-Salpepi's iPad 9th gen (2019).

[pixelzoom]

So to summarize, here are the tests that we have so far, all with iPad OS 17.5.1:

• @Nancy-Salpepi, iPad 9th gen, A13 Bionic (2021) - crashes
• @arouinfar, iPad Pro 11" 4th gen, M2 (2022) - OK
• @pixelzoom, iPad 6th gen, A10 Fusion (2018) - OK
• @KatieWoe, iPad Pro 1st gen, A9X (2017) - OK

I think this rules out the "problem specific to newer devices" hypothesis. Since we can't reproduce and @Nancy-Salpepi is out, I'm inclined to just close this issue.

@arouinfar your opinion?

[pixelzoom]

@arouinfar and I discussed. We're going to proceed with 1.1.0-rc.2, include this in "Issues to Verify", and request testing on at least 2 iPad devices.

[pixelzoom]

@jbphet was kind enough to test this on his iPad 8th gen, and noted that the page reloads when the sim crashes. So if you don't watch the sim constantly while it's fuzzing, it's easy to miss the crash. I retested with my iPad 6th gen, and I did indeed see the page reload twice.

Next step is to tether to my Mac, to see if I can get any useful info from Safari Dev Tools.

From Slack#quality-assurance thread:

@pixelzoom If it’s tethered to a Mac + Safari Dev Tools, I believe you can pause if it hits an error. But an out-of-memory condition will still cause a page reload, and you’ll get no info in Safari Dev Tools. Wondering why we’d be causing an out-of-memory error when memory tests are fine: #222. @jonathanolson are there other potential memory issues that do not show up in heap snapshots? Like WebGL contexts? This sim uses Sprites to render the particle systems.

@jonathanolson Hard to tell. Maybe it’s more likely to be something else. I think the memory limit is for the process, not just the heap, so it depends how much overhead there is.

[jbphet]

As @pixelzoom mentioned above, I tested this on an iPad 8. I ran 5 tests of 5 min each and saw one crash, and opened a new tab for each one. The crash was at about the 2 minute mark on the first run.

[pixelzoom]

I'm testing using this URL:
https://phet-dev.colorado.edu/html/gas-properties/1.1.0-rc.1/phet/gas-properties_all_phet.html?fuzz

I tethered my iPad6 to my MacBook Pro, and enabled all breakpoints in Safari Dev Tools, as shown in the screenshot below. (There's an unrelated exception that occurs when the sim starts, so I had to start with all breakpoints disabled, then enable them after the splash screen beccomes visible.)

I saw the crash occur multiple times, reloading the page automatically. There was no information (or even notification) in Dev Tools.

Tethering to Dev Tools did slow down the reload enough that I was able to capture it in a video recording. Here's what is displayed after the sim crashes and before the page is reloaded:

This type of automatic reload (and message) is typical of a memory problem. But I've seen the crash occur as soon as 10 seconds after the sim starts, and #222 indicates no memory problem. So it's difficult for me to conclude that this is a memory problem, or to troubleshoot it as a memory problem.

In Settings > Safari on my iPad6, I did "Clear History and Website Data" in case caching was contributing to memory problems. I also looked for any settings that might disable automatic reloading of pages, but found no such setting.

So... I'm stumped and unsure of what to do next.

[pixelzoom]

I'm testing using this URL:
https://phet-dev.colorado.edu/html/gas-properties/1.1.0-rc.1/phet/gas-properties_all_phet.html?fuzz

I also tested with assertions enabled in the debug version, with identical results:
https://phet-dev.colorado.edu/html/gas-properties/1.1.0-rc.1/phet/gas-properties_all_phet_debug.html?ea&fuzz

[pixelzoom]

@samreid and @zepumph were kind enough to brainstorm about this issue with me at standup this moring. A few notes...

• Test with ?webgl=false to rule out WebGL problems. This will cause Sprites to use 'canvas' as the fallback.
• @samreid noted that heap comparisons indicate leaks with PaintColorProperty (related to StopwatchNode) and SpriteInstance. Hard to believe that this would crash the sim in 10 seconds, but I'll reopen Memory test for 1.1 release #222 and investigate.
• Use http-server to serve the working copy on my MacBook, so that I can run changes on my iPad without having to deploy a dev version. (I had this set up for apache, but a macOS update broken it.)

[pixelzoom]

• Test with ?webgl=false to rule out WebGL problems. This will cause Sprites to use 'canvas' as the fallback.

Testing with ?webgl=false, I experienced zero crashes in 5 runs of the sim.

But I immediately noticed that particles are no longer scaled properly, see screenshot below. Then it hit me -- this is identical to phetsims/faradays-electromagnetic-lab#182, also a reported crash on iPad. The soultion there was (sadly) to disable WebGL on mobile safari, see phetsims/faradays-electromagnetic-lab@07a4fef. We will likely need to do the same here, as well as fix the canvas fallback.

Also interesting is that rotating the iPad to portrait orientation causes the particles to be scaled correctly, screeshot below. Rotating back to landscape causes them to be scaled incorrectly, as shown above.

[pixelzoom]

The 4 commits above are the primary fix to (1) set the correct transformType for SpriteInstance, and (2) set webgl: false when running on mobile Safari.

The next commits will be patches to the release branches.

[pixelzoom]

Patches to the release branches completed in the above commits.

[pixelzoom]

Tested on my iPad6, with my working copy while fixing, then with https://phet-dev.colorado.edu/html/gas-properties/1.2.0-dev.2/phet/gas-properties_all_phet.html (with ?fuzz and ?fuzz&disableModals). I did not see any page reloads, or any other indications of the sim crashing.

This is ready for verification in 1.1.0-rc.2.

[pixelzoom]

Please verify for phetsims/qa#1123, phetsims/qa#1123, and phetsims/qa#1125.

To verify:

• On iPad, run the sim 3 times with ?fuzz, for at least 3 minutes each time. Watch the sim while it is running, and verify that it does not reload due to crashing.
• Repeat this on at least 2 iPads, both running iPadOS 17.5.1.

If everything looks OK, please close this issue.

[Nancy-Salpepi]

For rc.2, no crashes occurred while fuzz testing the 3 sims in this suite following the directions above.
I'm on an iPad 9th generation using iPadOS 17.5.1.

[KatieWoe]

Looked ok to me in rc.2 as well.