Support for Coro / Threads [rt.cpan.org #60950]

[timbunce]

Migrated from rt.cpan.org#60950 (status was 'open')

Requestors:

• eugene.toropov@gmail.com

From eugene.toropov@gmail.com on 2010-09-01 19:15:44:

Greetings,

We have just tried to profile our application that uses Coro package and got an error:

NYTProf panic: stack is confused, giving up!
Segmentation fault

I understand that it's not so easy to add support of another low level implementation of threads, but still hope it's possible. Thank you in advance.

Cheers
Eugene.

From tim.bunce@pobox.com on 2010-09-02 09:33:02:

On Wed, Sep 01, 2010 at 03:15:54PM -0400, Eugene Toropov via RT wrote:
> Transaction: Ticket created by eugene.toropov@gmail.com
>  Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=60950 >
> 
> Greetings,
> 
> We have just tried to profile our application that uses Coro package and got an error:
> NYTProf panic: stack is confused, giving up!
> Segmentation fault

I'm not surprised.

> I understand that it's not so easy to add support of another low level
> implementation of threads, but still hope it's possible. Thank you in
> advance.

Realistically it's not going to happen anytime soon, and not without
much help from the Coro developers. Your best bet is to submit a bug
report to Coro asking for them to work with NYTProf devs.

Tim.

From eugene.toropov@gmail.com on 2010-09-07 08:19:42:

Hi Tim,

I've had a discussion on this subject with Marc Lehmann and here is what he 
wrote to me:

<schmorp:#anyevent> well, all they need to do is use the current thread as 
extra index into their data structures
<schmorp:#anyevent> $NYTPROFDATA{$Coro::current+0} for example

Cheers
Eugene


> <URL: http://rt.cpan.org/Ticket/Display.html?id=60950 >
>
> On Wed, Sep 01, 2010 at 03:15:54PM -0400, Eugene Toropov via RT wrote:
>> Transaction: Ticket created by eugene.toropov@gmail.com
>>  Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=60950 >
>>
>> Greetings,
>>
>> We have just tried to profile our application that uses Coro package and 
>> got an error:
>> NYTProf panic: stack is confused, giving up!
>> Segmentation fault
>
> I'm not surprised.
>
>> I understand that it's not so easy to add support of another low level
>> implementation of threads, but still hope it's possible. Thank you in
>> advance.
>
> Realistically it's not going to happen anytime soon, and not without
> much help from the Coro developers. Your best bet is to submit a bug
> report to Coro asking for them to work with NYTProf devs.
>
> Tim.
> 

From tim.bunce@pobox.com on 2010-09-12 10:37:15:

On Tue, Sep 07, 2010 at 04:19:43AM -0400, Eugene Toropov via RT wrote:
> 
> I've had a discussion on this subject with Marc Lehmann and here is what he 
> wrote to me:
> 
> <schmorp:#anyevent> well, all they need to do is use the current thread as 
> extra index into their data structures
> <schmorp:#anyevent> $NYTPROFDATA{$Coro::current+0} for example

So essentially we need to support threading first. That's unlikely to
get done by me anytime soon, but it would be reasonably straightforward
for someone else to do. Volunteers welcome!

Tim.

From tim.bunce@pobox.com on 2010-09-24 10:24:02:

On Thu, Sep 23, 2010 at 05:12:06PM +0100, Roger Picton wrote:
>    Tim:

Hello Roger.

>    Excuse me emailing you directly but I haven't got a login id for the CPAN bug system (and I'm not sure
>    whether I'd be allowed to add a comment to this bug even if I had!).

Once logged in you can comment on any ticket, and add yourself as a CC
on any ticket.

>    I found the same problem when trying to profile a program which uses Coro.  Since the error message said
>    "stack is confused" and coroutines tinker with the stack I thought I'd try switching off subroutine
>    profiling (subs=0 in the NYTPROF environment variable).  When I did that the profiler worked fine.

That's useful to know. I'll add a note to the docs.

>    I
>    obviously only got line and block level results in that case, but that was certainly good enough to help
>    me identify the hot spots.  On the other hand, I presume this ploy wouldn't work for multiple threads.

Per the docs at
http://search.cpan.org/~timb/Devel-NYTProf-4.05/lib/Devel/NYTProf.pm#Threads_and_Multiplicity
    "Meanwhile, profiling is disabled when a thread is created, and NYTProf
    tries to ignore any activity from perl interpreters other than the first
    one that loaded it."
So, no. Sorry.

>    For reference I was using ActivePerl 5.12.2.1202 (which includes Devel::NYTProf 4.04) and Coro 5.23 on
>    Windows Vista Home Premium SP2.
> 
>    Thanks for taking on this excellent profiler!

You're welcome.

I'd be very supportive of anyone wanting to make NYTProf thread/coro safe.
It should be a relatively straightforward and typical process, i.e.,
create a struct, pass it around, start moving the globals into the struct.
That can be an incremental process, each in a separate commit, with the
test suite kept passing. Once that work is done, actual thread safety
shouldn't be hard to add as a final step.

I'm looking for a volunter!

Tim.

From mmcleric@cpan.org on 2013-04-09 12:01:41:

I've created an open quest on Play Perl for this: http://play-perl.org/quest/5164001d0d7965996100005a
(This is the most important feature missing from Devel::NYTProf for me and our team, but I don't have time to do it myself.)

Let's see if anyone picks it up.