Merge iOS changes into flutter_releases/3.10.3 #20
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Make work with rust_struct patch Cleaned up the engine code (including disabling for non-release) and made work with rust_struct patch. It doesn't (yet) send along all the information that the updater wants, but at least some of the plumbing is there. * Make work with rust_struct patch Cleaned up the engine code (including disabling for non-release) and made work with rust_struct patch. It doesn't (yet) send along all the information that the updater wants, but at least some of the plumbing is there. * attempt to expose symbols on Android * Remove MacOS changes * Remove more MacOS changes
This loads shorebird.yaml as a string (from the java side) and passes it to the C++ side of the engine which then passes it along (via a C api) to the updater Rust library for parsing. This plumbs through support for product_id, base_url and channel and we can now add more to shorebird.yaml without having to touch this code again.
This is the last piece of data (at least the ones we're currently using) that was not plumbed through to the updater.
This will cause the updater to not recommend that patch again.
This is a quick hack, but it should work to unblock Felix.
Previously the path was always "libapp.so", for whatever reason the engine currently has a quirk of having ["libapp.so", "real/path/libapp.so"] in that array. Presumably a bug. I also fixed shorebird_current_version to shorebird_current_patch_number while I was editing the engine since there was a TODO in the header.
This required conditionally linking against separate builds of libupdater.a as well as providing a stub for getauxval. If we find a way to move off of using ring (via rust-tls, via reqwests) we could get rid of this. More likely Flutter will just move to a newer version of SDK/NDK: flutter/flutter#82000
This makes a single engine git hash correctly imply the full dependency chain instead of inserting libupdater from the side. We still need to teach GN/Ninja to build the updater but this is a first step.
Also updated paths to depend on updater.h and libupdater.a from third_party directly now that DEPS knows how to pull down updater for us. We still don't build updater from gn/ninja yet (we should) but this at least fixes the ninja null build to be null again previously it was rebuilding common.shell.o every time, unclear why.
* Fix unterminated string causing flaky behavior. Fixes shorebirdtech/shorebird#235 The root cause was we were not terminating the strings passed for original_libapp_paths. We used a Vector<char> but didn't append a '\0' to the end before calling data(), so when the c/rust side walked to find the end of the char* it sometimes found extra characters at the end of the string unexpectedly. This fixes to use a shorter (but probably equally dangerous?) method of using c_str() which we had tried before but previously had used c_str() on a temporary. This attempt carefully uses a reference to the string so the c_str()'s lifetime is tied to the string in application_library_paths rather than a temporary copy. This is still slightly dangerous in that c_str() lives as long as a std::string does, so long as it's not modified. * Fix comments
#8) * Bump updater dependency to include fix for shorebirdtech/shorebird#235
* feat: Support async updates from the updater library. This is the engine side of: shorebirdtech/updater#11 I also removed passing vmpath (unused) and adjusted names to be more consistent. * Add DEPS change
* feat: Support async updates from the updater library. This is the engine side of: shorebirdtech/updater#11 I also removed passing vmpath (unused) and adjusted names to be more consistent. * Add DEPS change
* feat: add versionCode to ConfigureShorebird * Include the version_code in the release_version --------- Co-authored-by: Felix Angelov <felix@shorebird.dev>
* fix: gracefully handle missing shorebird.yaml asset * format * more formatting * revert changes to flutter_main.cc
allows to supply gn args that do not have their cli switches (yet), like this: ```sh python3 ./tools/gn --gn-args 'use_default_linux_sysroot=false' ``` *List which issues are fixed by this PR. You must list at least one issue.* flutter/flutter#126197 *If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].* [C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
felangel
pushed a commit
that referenced
this pull request
Jul 18, 2024
The culprit test asks different threads to access some stack variables, but the stack might return earlier than the threads' completion. This PR fixes it by adding latches.
This test, despite being possibly fine for now, consistently triggers the following asan error when I make some unrelated trivial changes.
The thrown error:
<details>
<pre>
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from EmbedderTest
[ RUN ] EmbedderTest.EmbedderThreadHostUseCustomThreadConfig
=================================================================
==14398==ERROR: AddressSanitizer: stack-use-after-scope on address 0x00031833b530 at pc 0x00013248d543 bp 0x000318740390 sp 0x00031873fb58
WRITE of size 4 at 0x00031833b530 thread T2
#0 0x13248d542 in pthread_getschedparam+0x3d2 (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x2c542)
#1 0x100577671 in flutter::testing::EmbedderTest_EmbedderThreadHostUseCustomThreadConfig_Test::TestBody()::$_0::operator()() const+0x1a1 (embedder_unittests:x86_64+0x10030b671)
#2 0x1005774a2 in decltype(std::declval<flutter::testing::EmbedderTest_EmbedderThreadHostUseCustomThreadConfig_Test::TestBody()::$_0&>()()) std::_fl::__invoke[abi:v15000]<flutter::testing::EmbedderTest_EmbedderThreadHostUseCustomThreadConfig_Test::TestBody()::$_0&>(flutter::testing::EmbedderTest_EmbedderThreadHostUseCustomThreadConfig_Test::TestBody()::$_0&)+0x22 (embedder_unittests:x86_64+0x10030b4a2)
#3 0x100577402 in void std::_fl::__invoke_void_return_wrapper<void, true>::__call<flutter::testing::EmbedderTest_EmbedderThreadHostUseCustomThreadConfig_Test::TestBody()::$_0&>(flutter::testing::EmbedderTest_EmbedderThreadHostUseCustomThreadConfig_Test::TestBody()::$_0&)+0x22 (embedder_unittests:x86_64+0x10030b402)
#4 0x1005773ba in std::_fl::__function::__alloc_func<flutter::testing::EmbedderTest_EmbedderThreadHostUseCustomThreadConfig_Test::TestBody()::$_0, std::_fl::allocator<flutter::testing::EmbedderTest_EmbedderThreadHostUseCustomThreadConfig_Test::TestBody()::$_0>, void ()>::operator()[abi:v15000]()+0x2a (embedder_unittests:x86_64+0x10030b3ba)
#5 0x1005734f6 in std::_fl::__function::__func<flutter::testing::EmbedderTest_EmbedderThreadHostUseCustomThreadConfig_Test::TestBody()::$_0, std::_fl::allocator<flutter::testing::EmbedderTest_EmbedderThreadHostUseCustomThreadConfig_Test::TestBody()::$_0>, void ()>::operator()()+0x26 (embedder_unittests:x86_64+0x1003074f6)
#6 0x10066a525 in std::_fl::__function::__value_func<void ()>::operator()[abi:v15000]() const+0x105 (embedder_unittests:x86_64+0x1003fe525)
#7 0x10066a3f2 in std::_fl::function<void ()>::operator()() const+0x22 (embedder_unittests:x86_64+0x1003fe3f2)
#8 0x100c229dc in fml::MessageLoopImpl::FlushTasks(fml::FlushType)+0x37c (embedder_unittests:x86_64+0x1009b69dc)
#9 0x100c22577 in fml::MessageLoopImpl::RunExpiredTasksNow()+0x27 (embedder_unittests:x86_64+0x1009b6577)
#10 0x100c8aa2f in fml::MessageLoopDarwin::OnTimerFire(__CFRunLoopTimer*, fml::MessageLoopDarwin*)+0x2f (embedder_unittests:x86_64+0x100a1ea2f)
#11 0x7ff817b54508 in __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__+0x13 (CoreFoundation:x86_64+0x97508)
#12 0x7ff817b540b8 in __CFRunLoopDoTimer+0x310 (CoreFoundation:x86_64+0x970b8)
#13 0x7ff817b53ce9 in __CFRunLoopDoTimers+0x11c (CoreFoundation:x86_64+0x96ce9)
#14 0x7ff817b380ac in __CFRunLoopRun+0x83b (CoreFoundation:x86_64+0x7b0ac)
#15 0x7ff817b372a8 in CFRunLoopRunSpecific+0x22c (CoreFoundation:x86_64+0x7a2a8)
#16 0x100c8b155 in fml::MessageLoopDarwin::Run()+0x305 (embedder_unittests:x86_64+0x100a1f155)
#17 0x100c223d0 in fml::MessageLoopImpl::DoRun()+0x150 (embedder_unittests:x86_64+0x1009b63d0)
#18 0x100c1f1fa in fml::MessageLoop::Run()+0x2a (embedder_unittests:x86_64+0x1009b31fa)
#19 0x100c7baa4 in fml::Thread::Thread(std::_fl::function<void (fml::Thread::ThreadConfig const&)> const&, fml::Thread::ThreadConfig const&)::$_0::operator()() const+0x194 (embedder_unittests:x86_64+0x100a0faa4)
#20 0x100c7b8e2 in decltype(std::declval<fml::Thread::Thread(std::_fl::function<void (fml::Thread::ThreadConfig const&)> const&, fml::Thread::ThreadConfig const&)::$_0&>()()) std::_fl::__invoke[abi:v15000]<fml::Thread::Thread(std::_fl::function<void (fml::Thread::ThreadConfig const&)> const&, fml::Thread::ThreadConfig const&)::$_0&>(fml::Thread::Thread(std::_fl::function<void (fml::Thread::ThreadConfig const&)> const&, fml::Thread::ThreadConfig const&)::$_0&)+0x22 (embedder_unittests:x86_64+0x100a0f8e2)
#21 0x100c7b842 in void std::_fl::__invoke_void_return_wrapper<void, true>::__call<fml::Thread::Thread(std::_fl::function<void (fml::Thread::ThreadConfig const&)> const&, fml::Thread::ThreadConfig const&)::$_0&>(fml::Thread::Thread(std::_fl::function<void (fml::Thread::ThreadConfig const&)> const&, fml::Thread::ThreadConfig const&)::$_0&)+0x22 (embedder_unittests:x86_64+0x100a0f842)
#22 0x100c7b7fa in std::_fl::__function::__alloc_func<fml::Thread::Thread(std::_fl::function<void (fml::Thread::ThreadConfig const&)> const&, fml::Thread::ThreadConfig const&)::$_0, std::_fl::allocator<fml::Thread::Thread(std::_fl::function<void (fml::Thread::ThreadConfig const&)> const&, fml::Thread::ThreadConfig const&)::$_0>, void ()>::operator()[abi:v15000]()+0x2a (embedder_unittests:x86_64+0x100a0f7fa)
#23 0x100c77ad6 in std::_fl::__function::__func<fml::Thread::Thread(std::_fl::function<void (fml::Thread::ThreadConfig const&)> const&, fml::Thread::ThreadConfig const&)::$_0, std::_fl::allocator<fml::Thread::Thread(std::_fl::function<void (fml::Thread::ThreadConfig const&)> const&, fml::Thread::ThreadConfig const&)::$_0>, void ()>::operator()()+0x26 (embedder_unittests:x86_64+0x100a0bad6)
#24 0x10066a525 in std::_fl::__function::__value_func<void ()>::operator()[abi:v15000]() const+0x105 (embedder_unittests:x86_64+0x1003fe525)
#25 0x10066a3f2 in std::_fl::function<void ()>::operator()() const+0x22 (embedder_unittests:x86_64+0x1003fe3f2)
#26 0x100c75388 in fml::ThreadHandle::ThreadHandle(std::_fl::function<void ()>&&)::$_0::operator()(void*) const+0xe8 (embedder_unittests:x86_64+0x100a09388)
#27 0x100c75227 in fml::ThreadHandle::ThreadHandle(std::_fl::function<void ()>&&)::$_0::__invoke(void*)+0xc7 (embedder_unittests:x86_64+0x100a09227)
#28 0x1324b3e76 in asan_thread_start(void*)+0x46 (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x52e76)
#29 0x7ff817a5c18a in _pthread_start+0x62 (libsystem_pthread.dylib:x86_64+0x618a)
#30 0x7ff817a57ae2 in thread_start+0xe (libsystem_pthread.dylib:x86_64+0x1ae2)
Address 0x00031833b530 is located in stack of thread T0 at offset 144 in frame
#0 0x1002d051f in flutter::testing::EmbedderTest_EmbedderThreadHostUseCustomThreadConfig_Test::TestBody()+0xf (embedder_unittests:x86_64+0x10006451f)
This frame has 12 object(s):
[32, 40) 'thread_host' (line 3445)
[64, 112) 'ref.tmp' (line 3446)
[144, 148) 'ui_policy' (line 3449) <== Memory access at offset 144 is inside this variable
[160, 168) 'ui_param' (line 3450)
[192, 200) 'ref.tmp2' (line 3452)
[224, 272) 'ref.tmp5' (line 3452)
[304, 320) 'agg.tmp'
[336, 340) 'io_policy' (line 3458)
[352, 360) 'io_param' (line 3459)
[384, 392) 'ref.tmp6' (line 3460)
[416, 464) 'ref.tmp10' (line 3460)
[496, 512) 'agg.tmp11'
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
(longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-use-after-scope (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x2c542) in pthread_getschedparam+0x3d2
Shadow bytes around the buggy address:
0x00031833b280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x00031833b300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x00031833b380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x00031833b400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x00031833b480: 00 00 00 00 f1 f1 f1 f1 00 f2 f2 f2 f8 f8 f8 f8
=>0x00031833b500: f8 f8 f2 f2 f2 f2[f8]f2 f8 f2 f2 f2 f8 f2 f2 f2
0x00031833b580: f8 f8 f8 f8 f8 f8 f2 f2 f2 f2 00 00 f2 f2 f8 f2
0x00031833b600: f8 f2 f2 f2 f8 f2 f2 f2 f8 f8 f8 f8 f8 f8 f2 f2
0x00031833b680: f2 f2 00 00 f3 f3 f3 f3 00 00 00 00 00 00 00 00
0x00031833b700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x00031833b780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Thread T2 created by T0 here:
#0 0x1324ae92d in pthread_create+0x5d (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x4d92d)
#1 0x100c73901 in fml::ThreadHandle::ThreadHandle(std::_fl::function<void ()>&&)+0x401 (embedder_unittests:x86_64+0x100a07901)
#2 0x100c73cca in fml::ThreadHandle::ThreadHandle(std::_fl::function<void ()>&&)+0x2a (embedder_unittests:x86_64+0x100a07cca)
#3 0x100c748c9 in std::_fl::__unique_if<fml::ThreadHandle>::__unique_single std::_fl::make_unique[abi:v15000]<fml::ThreadHandle, fml::Thread::Thread(std::_fl::function<void (fml::Thread::ThreadConfig const&)> const&, fml::Thread::ThreadConfig const&)::$_0>(fml::Thread::Thread(std::_fl::function<void (fml::Thread::ThreadConfig const&)> const&, fml::Thread::ThreadConfig const&)::$_0&&)+0x149 (embedder_unittests:x86_64+0x100a088c9)
#4 0x100c74335 in fml::Thread::Thread(std::_fl::function<void (fml::Thread::ThreadConfig const&)> const&, fml::Thread::ThreadConfig const&)+0x255 (embedder_unittests:x86_64+0x100a08335)
#5 0x100c746b2 in fml::Thread::Thread(std::_fl::function<void (fml::Thread::ThreadConfig const&)> const&, fml::Thread::ThreadConfig const&)+0x32 (embedder_unittests:x86_64+0x100a086b2)
#6 0x106ee0b1a in std::_fl::__unique_if<fml::Thread>::__unique_single std::_fl::make_unique[abi:v15000]<fml::Thread, std::_fl::function<void (fml::Thread::ThreadConfig const&)> const&, fml::Thread::ThreadConfig&>(std::_fl::function<void (fml::Thread::ThreadConfig const&)> const&, fml::Thread::ThreadConfig&)+0x4a (embedder_unittests:x86_64+0x106c74b1a)
#7 0x106ee08e8 in flutter::ThreadHost::CreateThread(flutter::ThreadHost::Type, std::_fl::optional<fml::Thread::ThreadConfig>, flutter::ThreadHost::ThreadHostConfig const&) const+0x1b8 (embedder_unittests:x86_64+0x106c748e8)
#8 0x106ee1466 in flutter::ThreadHost::ThreadHost(flutter::ThreadHost::ThreadHostConfig const&)+0x2f6 (embedder_unittests:x86_64+0x106c75466)
#9 0x106ee19ca in flutter::ThreadHost::ThreadHost(flutter::ThreadHost::ThreadHostConfig const&)+0x2a (embedder_unittests:x86_64+0x106c759ca)
#10 0x100b6f730 in flutter::EmbedderThreadHost::CreateEngineManagedThreadHost(std::_fl::function<void (fml::Thread::ThreadConfig const&)> const&)+0x3a0 (embedder_unittests:x86_64+0x100903730)
#11 0x100b6e51c in flutter::EmbedderThreadHost::CreateEmbedderOrEngineManagedThreadHost(FlutterCustomTaskRunners const*, std::_fl::function<void (fml::Thread::ThreadConfig const&)> const&)+0xcc (embedder_unittests:x86_64+0x10090251c)
#12 0x1002d0748 in flutter::testing::EmbedderTest_EmbedderThreadHostUseCustomThreadConfig_Test::TestBody()+0x238 (embedder_unittests:x86_64+0x100064748)
#13 0x1089ce93b in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x21b (embedder_unittests:x86_64+0x10876293b)
#14 0x108955bdf in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x2cf (embedder_unittests:x86_64+0x1086e9bdf)
#15 0x108955755 in testing::Test::Run()+0x3a5 (embedder_unittests:x86_64+0x1086e9755)
#16 0x108957b23 in testing::TestInfo::Run()+0x593 (embedder_unittests:x86_64+0x1086ebb23)
#17 0x10895a8f2 in testing::TestSuite::Run()+0x6e2 (embedder_unittests:x86_64+0x1086ee8f2)
#18 0x108977d7a in testing::internal::UnitTestImpl::RunAllTests()+0xb9a (embedder_unittests:x86_64+0x10870bd7a)
#19 0x1089ee88b in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*)+0x21b (embedder_unittests:x86_64+0x10878288b)
#20 0x1089770ac in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*)+0x2cc (embedder_unittests:x86_64+0x10870b0ac)
#21 0x108976c3b in testing::UnitTest::Run()+0x28b (embedder_unittests:x86_64+0x10870ac3b)
#22 0x1083e45e2 in RUN_ALL_TESTS()+0x22 (embedder_unittests:x86_64+0x1081785e2)
#23 0x1083e43c8 in main+0x6e8 (embedder_unittests:x86_64+0x1081783c8)
#24 0x213ff0365 (<unknown module>)
==14398==ABORTING
[1] 14398 abort ../out/host_debug_unopt/embedder_unittests
</pre>
</details>
The changes I made (isolated from a larger change, therefore it doesn't seem reasonable by itself):
```diff
// shell/common/platform_view.cc, L33
void PlatformView::DispatchPointerDataPacket(
std::unique_ptr<PointerDataPacket> packet) {
- delegate_.OnPlatformViewDispatchPointerDataPacket(
- pointer_data_packet_converter_.Convert(std::move(packet)));
+ delegate_.OnPlatformViewDispatchPointerDataPacket(std::move(packet));
}
```
Base engine rev: de2a73c
Environment: Macbook with M1 Max Chip, OS 14.4.1
Compiling with:
```
./engine/src/flutter/tools/gn --goma --unoptimized --asan --xcode-symlinks
./depot_tools/ninja -j200 -C ./engine/src/out/host_debug_unopt
./out/host_debug_unopt/embedder_unittests
```
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.