Create performance benchmarks for DevTools

[kenzieschmoll]

See package https://pub.dev/packages/web_benchmarks. We will need somewhere to submit our benchmark numbers to. Flutter uses Skia Perf so maybe we can do this for DevTools too.

CC @mkustermann @kevmoo @sigmundch adding these benchmarks will help with the dart2wasm investigation.

[kevmoo]

Wonderful!

…

On Wed, Oct 18, 2023 at 9:40 AM Kenzie Davisson ***@***.***> wrote: See package https://pub.dev/packages/web_benchmarks. We will need somewhere to submit our benchmark numbers to. Flutter uses Skia Perf so maybe we can do this for DevTools too. CC @mkustermann <https://github.com/mkustermann> @kevmoo <https://github.com/kevmoo> @sigmundch <https://github.com/sigmundch> adding these benchmarks will help with the dart2wasm investigation. — Reply to this email directly, view it on GitHub <#6552>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAEFCV262HATVAWXZ2OMR3YAABAFAVCNFSM6AAAAAA6F2I73OVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE2TAMJZHA4TMMI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[kenzieschmoll]

@kevmoo @mkustermann @srujzs @sigmundch

What types of performance can we expect dart2wasm to improve? It would be good to get a small set of benchmarks put together that we expect to show improvements with dart2wasm. For example, do we expect IPL to improve? overall UI perf (scrolling, page transitions, etc.)? Business logic? RPC messaging to / from the VM service? Json parsing? etc.

We have some known expensive operations like collecting a heap snapshot over the VM and parsing it, but I'm not sure where we can expect perf improvements from wasm

[kevmoo]

Great question, @kenzieschmoll ! Ideally we'd see improvements across the board. Everything you list should get better–or, hopefully–not worse!

[kevmoo]

Starting with IPL is great. Any work we could do to track janking frames would be great, too.

[kenzieschmoll]

Do we have any IPL benchmarks for other Dart / Flutter products we could use as an example?

[mkustermann]

As @kevmoo says, we're interested in benchmarks that are a proxy for user experience of DevTools. The set of benchmarks that would (if dart2wasm is on-par of better than dart2js) convince DevTools team to enable WASM for end users. If dart2wasm is meaningfully worse anywhere, we'd like to know about it and work on it.

Do we have any IPL benchmarks for other Dart / Flutter products we could use as an example?

@yjbanov What benchmarking infrastructure does flutter use for measuring IPL / ...?

[mkustermann]

/cc @osa1

[kevmoo]

I've managed to wire up IPL measurements with Google Analytics.

function() {
	if (window.performance && window.performance.getEntriesByType) {
		var entries = window.performance.getEntriesByType("navigation");
		if (entries.length > 0) {
			var navTiming = entries[0];
			var pageLoadTime = navTiming.loadEventEnd - navTiming.startTime;
		return Math.round(((pageLoadTime / 1000) + Number.EPSILON) * 100) / 100;
		}
	}
}

https://measureschool.com/track-ga4-page-load-time/

Wiring it up is...interesting. Not sure if we want to move to tag manager for this.

But I'm happy to stumble through it with someone.

For devtools, I feel there are better places to wire up analytics. Like time to load a performance dump, etc.