Question: does download_range_to_stream have size limitation please? currently core dumps on 48M or bigger download on multithreaded app.

[yxiang92128]

Hi,
I am trying to download a blob of size 64M into a 64M buffer and it ran into the following coredump.
The only way to work around it is to set_parallelism_factor to be 1. But that limits the download speed to a crawl.

Any insight into what might have gone wrong here?

Thanks,

Yang X
(gdb) bt
#0 0x00007ffff75dd9db in raise () at /lib64/libpthread.so.0
#1 0x00007ffff7941d56 in pplx::details::_ExceptionHolder::_ExceptionHolder() (this=0x7fff980438d8, __in_chrg=)
at /usr/local/include/pplx/pplxtasks.h:945
#2 0x00007ffff796b43a in __gnu_cxx::new_allocatorpplx::details::_ExceptionHolder::destroypplx::details::_ExceptionHolder(pplx::details::_ExceptionHolder*) (this=0x7fff980438d0, __p=0x7fff980438d8) at /usr/include/c++/4.8/ext/new_allocator.h:124
#3 0x00007ffff7967c89 in std::allocator_traits<std::allocatorpplx::details::_ExceptionHolder >::_S_destroypplx::details::_ExceptionHolder(std::allocatorpplx::details::_ExceptionHolder&, pplx::details::_ExceptionHolder*) (__a=..., __p=0x7fff980438d8)
at /usr/include/c++/4.8/bits/alloc_traits.h:281
#4 0x00007ffff79649f5 in std::allocator_traits<std::allocatorpplx::details::_ExceptionHolder >::destroypplx::details::_ExceptionHolder(std::allocatorpplx::details::_ExceptionHolder&, pplx::details::_ExceptionHolder*) (__a=..., __p=0x7fff980438d8)
at /usr/include/c++/4.8/bits/alloc_traits.h:405
#5 0x00007ffff795f033 in std::_Sp_counted_ptr_inplace<pplx::details::_ExceptionHolder, std::allocatorpplx::details::_ExceptionHolder, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (this=0x7fff980438c0) at /usr/include/c++/4.8/bits/shared_ptr_base.h:407
#6 0x00007ffff792d276 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (this=0x7fff980438c0)
at /usr/include/c++/4.8/bits/shared_ptr_base.h:144
#7 0x00007ffff792a7f5 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count() (this=0x7fffa8001cc0, __in_chrg=) at /usr/include/c++/4.8/bits/shared_ptr_base.h:546
#8 0x00007ffff7942342 in std::__shared_ptr<pplx::details::_ExceptionHolder, (__gnu_cxx::_Lock_policy)2>::__shared_ptr() (this=0x7fffa8001cb8, __in_chrg=) at /usr/include/c++/4.8/bits/shared_ptr_base.h:781
#9 0x00007ffff794235c in std::shared_ptrpplx::details::_ExceptionHolder::shared_ptr() (this=0x7fffa8001cb8, __in_chrg=) at /usr/include/c++/4.8/bits/shared_ptr.h:93
#10 0x00007ffff79425da in pplx::details::_Task_impl_base::_Task_impl_base() (this=0x7fffa8001ca8, __in_chrg=)
at /usr/local/include/pplx/pplxtasks.h:1709
#11 0x00007ffff7964374 in pplx::details::_Task_impl::_Task_impl() (this=0x7fffa8001ca8, __in_chrg=)
at /usr/local/include/pplx/pplxtasks.h:2412
#12 0x00007ffff796b32d in __gnu_cxx::new_allocator<pplx::details::_Task_impl >::destroy<pplx::details::_Task_impl >(pplx::details::_Task_impl) (this=0x7fffa8001ca0, __p=0x7fffa8001ca8)
at /usr/include/c++/4.8/ext/new_allocator.h:124
#13 0x00007ffff7967b17 in std::allocator_traits<std::allocator<pplx::details::_Task_impl > >::_S_destroy<pplx::details::_Task_impl >(std::allocator<pplx::details::_Task_impl >&, pplx::details::_Task_impl) (__a=..., __p=0x7fffa8001ca8) at /usr/include/c++/4.8/bits/alloc_traits.h:281
#14 0x00007ffff7964883 in std::allocator_traits<std::allocator<pplx::details::_Task_impl > >::destroy<pplx::details::_Task_impl >(std::allocator<pplx::details::_Task_impl >&, pplx::details::_Task_impl) (__a=..., __p=0x7fffa8001ca8) at /usr/include/c++/4.8/bits/alloc_traits.h:405
#15 0x00007ffff795e9a1 in std::_Sp_counted_ptr_inplace<pplx::details::_Task_impl, std::allocator<pplx::details::_Task_impl >, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (this=0x7fffa8001c90) at /usr/include/c++/4.8/bits/shared_ptr_base.h:407
#16 0x00007ffff792d276 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (this=0x7fffa8001c90)
at /usr/include/c++/4.8/bits/shared_ptr_base.h:144
#17 0x00007ffff3a74daf in pplx::task::_ContinuationTaskHandle<void, void, azure::storage::cloud_blob::download_range_to_s---Type to continue, or q to quit---
tream_async(Concurrency::streams::basic_ostream, unsigned long, unsigned long, azure::storage::access_condition const&, azure::storage::blob_request_options const&, azure::storage::operation_context)::{lambda(pplx::task)#1}::operator()(pplx::task) const::{lambda()#1}::operator()() const::{lambda()#1}::operator()() const::{lambda(pplx::task)#1}, std::integral_constant<bool, true>, pplx::details::_TypeSelectorNoAsync>::_ContinuationTaskHandle() () at /opt/teradata/tdobject/lib/libazurestorage.so.5
#18 0x00007ffff7948619 in pplx::details::_AutoDeleterpplx::details::_TaskProcHandle::_AutoDeleter() (this=0x7fffe3b73c70, __in_chrg=) at /usr/local/include/pplx/pplx.h:99
#19 0x00007ffff794197b in pplx::details::_TaskProcHandle::_RunChoreBridge(void) (_Parameter=0x7fffa8001db0)
at /usr/local/include/pplx/pplx.h:116
#20 0x00007ffff7349731 in boost::asio::detail::completion_handler<boost::_bi::bind_t<void, void ()(void), boost::_bi::list1<boost::_bi::value<void*> > > >::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long) () at /opt/teradata/tdobject/lib/libcpprest.so.2.9
#21 0x00007ffff72758af in boost::asio::detail::task_io_service::run(boost::system::error_code&) ()
at /opt/teradata/tdobject/lib/libcpprest.so.2.9
#22 0x00007ffff7275c56 in crossplat::threadpool::thread_start(void*) () at /opt/teradata/tdobject/lib/libcpprest.so.2.9
#23 0x00007ffff75d5744 in start_thread () at /lib64/libpthread.so.0
#24 0x00007ffff2b3eaad in clone () at /lib64/libc.so.6
(gdb)

[katmsft]

Because I'm seeing libcpprest.so.2.9 which means this is obviously not the latest code. Can we know which version of azure-storage-cpp you are using? A parallel download bug was resolved in the recent releases.

[yxiang92128]

I am using 5.0 with libcpprest sdk 2.9.1 because that's the only version of the cpprestsdk which would compile on SLES12SP3.
Do you have any recommendation in terms of which cpprest sdk I should attempt to use?

Thanks,

Yang

[katmsft]

It seems that this issue cannot be reproduced on Ubuntu 14.04 and Windows. We will try to investigate why the crash happens on SLES12SP3, however, this will be a low priority item since the platform is not officially in our support list.
Is the statement 'libcpprest sdk 2.9.1 is the only version of cpprestsdk that could compile on SLES12SP3' true? Have you tried the recently released 2.10.3?

[yxiang92128]

I will attempt for CPP rest sdk 2.10.3 and report back to you. Thanks.

[yxiang92128]

Hi Kan,
Anything higher than cpp rest sdk 2.9.1 and above won't work due to the following compilation error:
azure-storage-cpp/Microsoft.WindowsAzure.Storage/includes/wascore/executor.h:370:20: error: ‘do_while’ is not a member of ‘pplx::details’
return pplx::details::do_while(instance -> pplx::task
^
****pplx::details::do_while is no longer defined after cpp rest sdk 2.9.1 in include/pplx/pplxtask.h inside namespace details.

**** Not sure how you can work around it by using newer version of cpp rest sdk release.

Many thanks,

Yang

[katmsft]

Hi,

Cpprestsdk has some interface changes from 2.9.1 to 2.10.x versions. Please see in the latest README our tested cpprestsdk version on each of the azure-storage-cpp version.