Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SIGBUS/use-after-free on ARM #381

Closed
hadess opened this issue Jul 5, 2016 · 11 comments
Closed

SIGBUS/use-after-free on ARM #381

hadess opened this issue Jul 5, 2016 · 11 comments

Comments

@hadess
Copy link
Contributor

hadess commented Jul 5, 2016

Following on from #378

During the tests, apart from the test failures, which I'll look at again later on, there are a number of crashers, either SIGBUS, or SIGSEGV ones.

From the 9 core dumps I gathered running the test suite, I found those unique SIGBUS ones:

#0  0xb6c66244 in g_mutex_lock (mutex=mutex@entry=0x32323232) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread-posix.c:1337
#1  0xb6c34382 in g_source_destroy_internal (source=0x7f8d4d78, context=0x32323232, have_lock=have_lock@entry=0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1159
#2  0xb6c34874 in g_source_destroy (source=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1227
#3  0xb6e57042 in soup_session_dispose (object=0x7f5f0898) at soup-session.c:309
#4  0xb6ce49ce in g_object_unref (_object=0x7f5f0898) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3133
#5  0xb6ef2282 in thread_closure_unref (thread_closure=0x7f601c18) at src/libostree/ostree-fetcher.c:143
#6  0xb6ef23e6 in ostree_fetcher_session_thread (data=0x7f601c18) at src/libostree/ostree-fetcher.c:486
#7  0xb6c50ffa in g_thread_proxy (data=0x7f5f5d20) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread.c:764
#8  0xb6f1bf88 in start_thread (arg=0xb62cb360) at pthread_create.c:311
#9  0xb6b261fc in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

#0  0xb6c78244 in g_mutex_lock (mutex=mutex@entry=0xb9b9b9b9) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread-posix.c:1337
#1  0xb6c46382 in g_source_destroy_internal (source=0x7f789b00, context=0xb9b9b9b9, have_lock=have_lock@entry=0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1159
#2  0xb6c46874 in g_source_destroy (source=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1227
#3  0xb6e69042 in soup_session_dispose (object=0x7f5d5898) at soup-session.c:309
#4  0xb6cf69ce in g_object_unref (_object=0x7f5d5898) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3133
#5  0xb6f04282 in thread_closure_unref (thread_closure=0x7f5c6c18) at src/libostree/ostree-fetcher.c:143
#6  0xb6f04eda in _ostree_fetcher_finalize (object=0x7f5cb3b0) at src/libostree/ostree-fetcher.c:537
#7  0xb6cf6a48 in g_object_unref (_object=0x7f5cb3b0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3170
#8  0xb6f05dd0 in _ostree_metalink_finalize (object=0x7f5cf6a0) at src/libostree/ostree-metalink.c:378
#9  0xb6cf6a48 in g_object_unref (_object=0x7f5cf6a0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3170
#10 0xb6eed54a in glnx_local_obj_unref (v=<synthetic pointer>) at ./libglnx/glnx-local-alloc.h:63
#11 ostree_repo_pull_with_options (self=self@entry=0x7f5c4078, remote_name_or_baseurl=<optimized out>, options=<optimized out>, progress=progress@entry=0x0, cancellable=cancellable@entry=0x0, 
    error=error@entry=0xbeca5fd0) at src/libostree/ostree-repo-pull.c:2176
#12 0x7f59da02 in ostree_builtin_pull (argc=2, argv=0xbeca6174, cancellable=<optimized out>, error=0xbeca5fd0) at src/ostree/ot-builtin-pull.c:260
#13 0x7f5988ec in ostree_run (argc=3, argv=0xbeca6174, commands=0x7f5b8008 <commands>, res_error=0xbeca6000) at src/ostree/ot-main.c:209
#14 0x7f593166 in main (argc=4, argv=0xbeca6174) at src/ostree/main.c:78

And a couple of SEGVs too:

#0  0xb6c9f240 in g_mutex_lock (mutex=mutex@entry=0x3747470) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread-posix.c:1337
#1  0xb6c6d382 in g_source_destroy_internal (source=0x7f74e778, context=0x3747470, have_lock=have_lock@entry=0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1159
#2  0xb6c6d874 in g_source_destroy (source=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1227
#3  0xb6e90042 in soup_session_dispose (object=0x7f597098) at soup-session.c:309
#4  0xb6d1d9ce in g_object_unref (_object=0x7f597098) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3133
#5  0xb6f2b282 in thread_closure_unref (thread_closure=0x7f5a9018) at src/libostree/ostree-fetcher.c:143
#6  0xb6f2b3e6 in ostree_fetcher_session_thread (data=0x7f5a9018) at src/libostree/ostree-fetcher.c:486
#7  0xb6c89ffa in g_thread_proxy (data=0x7f59df20) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread.c:764
#8  0xb6f54f88 in start_thread (arg=0xb6304360) at pthread_create.c:311
#9  0xb6b5f1fc in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

#0  0xb6bdd1da in g_source_unref_internal (source=0x7f6b2898, context=0x0, have_lock=have_lock@entry=0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:2012
#1  0xb6bde296 in g_source_unref (source=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:2063
#2  0xb6e00048 in soup_session_dispose (object=0x7f5c1098) at soup-session.c:310
#3  0xb6c8d9ce in g_object_unref (_object=0x7f5c1098) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3133
#4  0xb6e9b282 in thread_closure_unref (thread_closure=0x7f5d2418) at src/libostree/ostree-fetcher.c:143
#5  0xb6e9b3e6 in ostree_fetcher_session_thread (data=0x7f5d2418) at src/libostree/ostree-fetcher.c:486
#6  0xb6bf9ffa in g_thread_proxy (data=0x7f5c8120) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread.c:764
#7  0xb6ec4f88 in start_thread (arg=0xb6279360) at pthread_create.c:311
#8  0xb6acf1fc in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

#0  0xb6f2d44c in pending_uri_unref (pending=0x0) at src/libostree/ostree-fetcher.c:208
208   if (!g_atomic_int_dec_and_test (&pending->ref_count))
(gdb) bt
#0  0xb6f2d44c in pending_uri_unref (pending=0x0) at src/libostree/ostree-fetcher.c:208
#1  0xb6c6484a in g_hash_table_remove_all_nodes (hash_table=0x7f63ed88, notify=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/ghash.c:503
#2  0xb6c64ede in g_hash_table_unref (hash_table=0x7f63ed88) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/ghash.c:1042
#3  0xb6f2d2d8 in thread_closure_unref (thread_closure=0x7f64e870) at src/libostree/ostree-fetcher.c:162
#4  0xb6f2deda in _ostree_fetcher_finalize (object=0x7f645398) at src/libostree/ostree-fetcher.c:537
#5  0xb6d1fa48 in g_object_unref (_object=0x7f645398) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3170
#6  0xb6f1687a in ostree_repo_pull_with_options (self=self@entry=0x7f64c078, remote_name_or_baseurl=<optimized out>, options=<optimized out>, progress=progress@entry=0x0, cancellable=cancellable@entry=0x0, 
    error=error@entry=0xbeeb6fd0) at src/libostree/ostree-repo-pull.c:2853
#7  0x7f617a02 in ostree_builtin_pull (argc=3, argv=0xbeeb7174, cancellable=<optimized out>, error=0xbeeb6fd0) at src/ostree/ot-builtin-pull.c:260
#8  0x7f6128ec in ostree_run (argc=4, argv=0xbeeb7174, commands=0x7f632008 <commands>, res_error=0xbeeb7000) at src/ostree/ot-main.c:209
#9  0x7f60d166 in main (argc=5, argv=0xbeeb7174) at src/ostree/main.c:78

#0  0xb6c1c240 in g_mutex_lock (mutex=mutex@entry=0x50505050) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread-posix.c:1337
#1  0xb6bea382 in g_source_destroy_internal (source=0x7f75adc0, context=0x50505050, have_lock=have_lock@entry=0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1159
#2  0xb6bea874 in g_source_destroy (source=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1227
#3  0xb6e0d042 in soup_session_dispose (object=0x7f5b3898) at soup-session.c:309
#4  0xb6c9a9ce in g_object_unref (_object=0x7f5b3898) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3133
#5  0xb6ea8282 in thread_closure_unref (thread_closure=0x7f5c3008) at src/libostree/ostree-fetcher.c:143
#6  0xb6ea8eda in _ostree_fetcher_finalize (object=0x7f5b9338) at src/libostree/ostree-fetcher.c:537
#7  0xb6c9aa48 in g_object_unref (_object=0x7f5b9338) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3170
#8  0xb6e923aa in ostree_repo_pull_with_options (self=self@entry=0x7f5c0078, remote_name_or_baseurl=0x7f5bfe58 "origin", options=<optimized out>, progress=progress@entry=0x0, cancellable=cancellable@entry=0x0, 
    error=error@entry=0xbebb3fc0) at src/libostree/ostree-repo-pull.c:2653
#9  0x7f58ba02 in ostree_builtin_pull (argc=2, argv=0xbebb4164, cancellable=<optimized out>, error=0xbebb3fc0) at src/ostree/ot-builtin-pull.c:260
#10 0x7f5868ec in ostree_run (argc=5, argv=0xbebb4164, commands=0x7f5a6008 <commands>, res_error=0xbebb3ff0) at src/ostree/ot-main.c:209
#11 0x7f581166 in main (argc=6, argv=0xbebb4164) at src/ostree/main.c:78

The packages are:

ii  libsoup2.4-1:armhf                     2.48.0-1                                                 armhf        HTTP library implementation in C -- Shared library
ii  libglib2.0-0:armhf                     2.42.1-1+b1                                              armhf        GLib library of C routines
ii  ostree                                 2016.6-2.1                                               armhf        content-addressed filesystem for operating system binaries

They all seem related to a refcounting/access after free issue. libglnx bug?
@cgwalters
Copy link
Member

I think this is a dup of #373

@cgwalters
Copy link
Member

@hadess can you try the patches in #383 too?

@hadess
Copy link
Contributor Author

hadess commented Jul 9, 2016

Do I need both the patches in the #383 PR and the libsoup changes?

@cgwalters
Copy link
Member

Try with just #383 first.

@hadess
Copy link
Contributor Author

hadess commented Jul 11, 2016 •
edited
Loading

After running the whole test suite, I get similar traces to this times 5 (either SIGBUS or SIGSEGV):

#0  0xb6be2244 in g_mutex_lock (mutex=mutex@entry=0x11111111) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread-posix.c:1337
#1  0xb6bb0382 in g_source_destroy_internal (source=0x13c6d0, context=0x11111111, have_lock=have_lock@entry=0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1159
#2  0xb6bb0874 in g_source_destroy (source=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1227
#3  0xb6dd3042 in soup_session_dispose (object=0x4a098) at soup-session.c:309
#4  0xb6c609ce in g_object_unref (_object=0x4a098) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3133
#5  0xb6e6c00e in ostree_fetcher_session_thread (data=0x5a818) at src/libostree/ostree-fetcher.c:486
#6  0xb6bccffa in g_thread_proxy (data=0x50d20) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread.c:764
#7  0xb6e94f88 in start_thread (arg=0xb627c360) at pthread_create.c:311
#8  0xb6aa21fc in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

@hadess
Copy link
Contributor Author

hadess commented Jul 11, 2016

That was with ostree git master.

@cgwalters
Copy link
Member

Interesting. The patches from #383 definitely made things more reliable for me (x86_64/CentOS7+ CAHC).

@hadess
Copy link
Contributor Author

hadess commented Jul 11, 2016

So, libsoup gnome-3-14 + patch from GNOME bz, and the ostree master, and it's not working any better from what I can tell.

Thread 3 (Thread 0xb591d360 (LWP 27436)):
#0  0xb6ad79d0 in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0xb6bee39e in g_main_context_poll (priority=2147483647, n_fds=2, fds=0x288a60, timeout=60000, context=0x302a00) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:4076
#2  g_main_context_iterate (context=0x302a00, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3776
#3  0xb6bee668 in g_main_loop_run (loop=0x302e48) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3975
#4  0xb6ea7fdc in ostree_fetcher_session_thread (data=0x5cc70) at src/libostree/ostree-fetcher.c:478
#5  0xb6c08ffa in g_thread_proxy (data=0x50f50) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread.c:764
#6  0xb6ed0f88 in start_thread (arg=0xb591d360) at pthread_create.c:311
#7  0xb6ade1fc in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 2 (Thread 0xb6f93000 (LWP 27434)):
#0  0xb6ad79d0 in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0xb6bee39e in g_main_context_poll (priority=2147483647, n_fds=1, fds=0x682f8, timeout=-1, context=0x1ef400) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:4076
#2  g_main_context_iterate (context=context@entry=0x1ef400, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3776
#3  0xb6bee44c in g_main_context_iteration (context=context@entry=0x1ef400, may_block=may_block@entry=1) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3842
#4  0xb6ea9282 in _ostree_fetcher_request_uri_to_membuf (fetcher=0x502c0, uri=uri@entry=0x56000, add_nul=add_nul@entry=0, allow_noent=allow_noent@entry=1, out_contents=out_contents@entry=0xbeedddf0, 
    max_size=max_size@entry=10485760, cancellable=0x0, error=error@entry=0xbeede024) at src/libostree/ostree-fetcher.c:1184
#5  0xb6e93158 in fetch_uri_contents_membuf_sync (pull_data=<optimized out>, pull_data=<optimized out>, error=0xbeede024, cancellable=<optimized out>, out_contents=0xbeedddf0, allow_noent=1, add_nul=0, 
    uri=0x56000) at src/libostree/ostree-repo-pull.c:374
#6  request_static_delta_superblock_sync (error=0xbeede024, cancellable=<optimized out>, out_delta_superblock=<synthetic pointer>, 
    to_revision=0x175930 "b7f70a14a776b7bd3fc2d78fee80a8321ecceb0170c292ee8de390e69d5cf030", from_revision=0x0, pull_data=0xbeedde30) at src/libostree/ostree-repo-pull.c:1436
#7  ostree_repo_pull_with_options (self=self@entry=0x59078, remote_name_or_baseurl=remote_name_or_baseurl@entry=0x58c68 "R1", options=<optimized out>, progress=progress@entry=0x0, 
    cancellable=cancellable@entry=0x0, error=error@entry=0xbeede024) at src/libostree/ostree-repo-pull.c:2710
#8  0x000222c0 in ostree_builtin_pull (argc=2, argv=0xbeede1b4, cancellable=0x0, error=0xbeede024) at src/ostree/ot-builtin-pull.c:260
#9  0x0001d8ec in ostree_run (argc=3, argc@entry=4, argv=argv@entry=0xbeede1b4, commands=commands@entry=0x3cc80 <commands>, res_error=res_error@entry=0xbeede04c) at src/ostree/ot-main.c:209
#10 0x000189ee in main (argc=4, argv=0xbeede1b4) at src/ostree/main.c:78

Thread 1 (Thread 0xb62eb360 (LWP 27435)):
#0  0x0041e348 in ?? ()
#1  0xb6bec1e2 in g_source_unref_internal (source=0x1ef4f8, context=0x0, have_lock=have_lock@entry=0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:2016
#2  0xb6bed296 in g_source_unref (source=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:2063
#3  0xb6e0f048 in soup_session_dispose (object=0x4b898) at soup-session.c:310
#4  0xb6c9c9ce in g_object_unref (_object=0x4b898) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3133
#5  0xb6ea800e in ostree_fetcher_session_thread (data=0x5cc18) at src/libostree/ostree-fetcher.c:486
#6  0xb6c08ffa in g_thread_proxy (data=0x50f20) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread.c:764
#7  0xb6ed0f88 in start_thread (arg=0xb62eb360) at pthread_create.c:311
#8  0xb6ade1fc in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

@hadess
Copy link
Contributor Author

hadess commented Jul 11, 2016

Sorry, I didn't install libsoup properly (the /usr/lib/ files are namespaced as well). So, with libsoup from gnome-3-14 with the patch added on top.

It hangs much sooner in the test suite, when running /home/chip/ostree/tests/.libs/lt-test-pull-c -k --tap:

#0  0xb690f9d0 in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0xb69a939e in g_main_context_poll (priority=2147483647, n_fds=1, fds=0x557a0, timeout=-1, context=0x229038) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:4076
#2  g_main_context_iterate (context=context@entry=0x229038, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3776
#3  0xb69a944c in g_main_context_iteration (context=context@entry=0x229038, may_block=may_block@entry=1) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3842
#4  0xb6f9e2a2 in _ostree_fetcher_request_uri_to_membuf (fetcher=0x32830, uri=uri@entry=0x37f60, add_nul=add_nul@entry=0, allow_noent=allow_noent@entry=1, out_contents=0xbebd3d98, max_size=10485760, 
    cancellable=0x0, error=0xbebd3fb4) at src/libostree/ostree-fetcher.c:1184
#5  0xb6f88a5c in fetch_uri_contents_membuf_sync (pull_data=0xbebd3dd8, pull_data=0xbebd3dd8, error=0xbebd3fb4, cancellable=0x0, out_contents=0xbebd3d98, allow_noent=1, add_nul=0, uri=0x37f60)
    at src/libostree/ostree-repo-pull.c:374
#6  ostree_repo_pull_with_options (self=self@entry=0x39078, remote_name_or_baseurl=remote_name_or_baseurl@entry=0x114f0 "origin", options=options@entry=0x328c0, progress=progress@entry=0x0, 
    cancellable=cancellable@entry=0x0, error=error@entry=0xbebd3fb4) at src/libostree/ostree-repo-pull.c:2431
#7  0xb6f7fb5a in ostree_repo_pull_one_dir (self=0x39078, remote_name=remote_name@entry=0x114f0 "origin", dir_to_pull=dir_to_pull@entry=0x0, refs_to_fetch=refs_to_fetch@entry=0xbebd3fb8, 
    flags=flags@entry=OSTREE_REPO_PULL_FLAGS_NONE, progress=progress@entry=0x0, cancellable=cancellable@entry=0x0, error=error@entry=0xbebd3fb4) at src/libostree/ostree-repo.c:3791
#8  0xb6f7fba0 in ostree_repo_pull (self=<optimized out>, remote_name=remote_name@entry=0x114f0 "origin", refs_to_fetch=refs_to_fetch@entry=0xbebd3fb8, flags=flags@entry=OSTREE_REPO_PULL_FLAGS_NONE, 
    progress=progress@entry=0x0, cancellable=cancellable@entry=0x0, error=error@entry=0xbebd3fb4) at src/libostree/ostree-repo.c:3749
#9  0x00011120 in test_pull_multi_nochange (data=0xbebd414c) at tests/test-pull-c.c:77
#10 0xb69c315e in test_case_run (tc=0x32a18) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gtestutils.c:2059
#11 g_test_run_suite_internal (suite=suite@entry=0x4d930, path=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gtestutils.c:2120
#12 0xb69c327c in g_test_run_suite_internal (suite=suite@entry=0x4d940, path=<optimized out>, path@entry=0xb6a12e08 "") at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gtestutils.c:2131
#13 0xb69c3508 in g_test_run_suite (suite=0x4d940) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gtestutils.c:2184
#14 0xb69c3556 in g_test_run () at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gtestutils.c:1488
#15 0x00010f68 in main (argc=1, argv=0xbebd42c4) at tests/test-pull-c.c:129

And just to make sure:

$ cat /proc/7333/smaps  | grep libsoup
b6d7f000-b6de7000 r-xp 00000000 00:0f 428638     /usr/lib/arm-linux-gnueabihf/libsoup-2.4.so.1.7.0
b6de7000-b6df6000 ---p 00068000 00:0f 428638     /usr/lib/arm-linux-gnueabihf/libsoup-2.4.so.1.7.0
b6df6000-b6e05000 rw-p 00067000 00:0f 428638     /usr/lib/arm-linux-gnueabihf/libsoup-2.4.so.1.7.0
$ ls -l /usr/lib/arm-linux-gnueabihf/libsoup-2.4.so.1.7.0
-rwxr-xr-x 1 root root 1907800 Jul 11 19:19 /usr/lib/arm-linux-gnueabihf/libsoup-2.4.so.1.7.0
$ date
Mon 11 Jul 19:39:12 UTC 2016

That really was the library compiled before I ran the test suite.

I'll try running against libsoup master + that patch now, if the dependencies allow me.

@hadess
Copy link
Contributor Author

hadess commented Jul 11, 2016

OK, libsoup master + the patch gets stuck much further, running test-remote-gpg-import.sh I think:

chip     21496  0.0  2.0  27300 10300 pts/0    Sl+  20:02   0:00 ostree --repo=repo pull R1:main

Thread 2 (Thread 0xb6235360 (LWP 21497)):
#0  0xb6abc9d0 in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0xb6bd339e in g_main_context_poll (priority=2147483647, n_fds=1, fds=0x7f759748, timeout=-1, context=0x7f6504f8) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:4076
#2  g_main_context_iterate (context=0x7f6504f8, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3776
#3  0xb6bd3668 in g_main_loop_run (loop=0x7f650ed8) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3975
#4  0xb6e94fdc in ostree_fetcher_session_thread (data=0x7f63e418) at src/libostree/ostree-fetcher.c:478
#5  0xb6bedffa in g_thread_proxy (data=0x7f648120) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread.c:764
#6  0xb6ebdf88 in start_thread (arg=0xb6235360) at pthread_create.c:311
#7  0xb6ac31fc in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 1 (Thread 0xb6f18000 (LWP 21496)):
#0  0xb6abc9d0 in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0xb6bd339e in g_main_context_poll (priority=2147483647, n_fds=1, fds=0x7f7eabf0, timeout=-1, context=0x7f651fa0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:4076
#2  g_main_context_iterate (context=context@entry=0x7f651fa0, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3776
#3  0xb6bd344c in g_main_context_iteration (context=context@entry=0x7f651fa0, may_block=may_block@entry=1) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3842
#4  0xb6e96282 in _ostree_fetcher_request_uri_to_membuf (fetcher=0x7f650960, uri=uri@entry=0x7f652b20, add_nul=add_nul@entry=0, allow_noent=allow_noent@entry=1, out_contents=0xbea8dde0, max_size=10485760, 
    cancellable=0x0, error=0xbea8e020) at src/libostree/ostree-fetcher.c:1184
#5  0xb6e80a3c in fetch_uri_contents_membuf_sync (pull_data=0xbea8de20, pull_data=0xbea8de20, error=0xbea8e020, cancellable=0x0, out_contents=0xbea8dde0, allow_noent=1, add_nul=0, uri=0x7f652b20)
    at src/libostree/ostree-repo-pull.c:374
#6  ostree_repo_pull_with_options (self=self@entry=0x7f64e078, remote_name_or_baseurl=0x7f64f788 "R1", options=<optimized out>, progress=progress@entry=0x0, cancellable=cancellable@entry=0x0, 
    error=error@entry=0xbea8e020) at src/libostree/ostree-repo-pull.c:2431
#7  0x7f6199e2 in ostree_builtin_pull (argc=2, argv=0xbea8e1c4, cancellable=<optimized out>, error=0xbea8e020) at src/ostree/ot-builtin-pull.c:260
#8  0x7f6148cc in ostree_run (argc=3, argv=0xbea8e1c4, commands=0x7f634008 <commands>, res_error=0xbea8e050) at src/ostree/ot-main.c:209
#9  0x7f60f146 in main (argc=4, argv=0xbea8e1c4) at src/ostree/main.c:78

But the SIGBUS errors in test-basic.sh are gone. I'll try to bisect, so we can choose a good minimal requirement, or backport the fixes.

@hadess
Copy link
Contributor Author

hadess commented Jul 13, 2016

It crashes without the libsoup patch on top of master, and I still see random "hangs" as above, with the application being stuck waiting with the patch applied (also on top of master).

@hadess hadess mentioned this issue Aug 6, 2016
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@cgwalters @hadess