Skip to content

jl_print_task_backtraces() seems to be failing to decode IP addresses in our custom sysimg #47934

Open

Description

The task stack traces dumped from jl_print_task_backtraces() in prod seem to sometimes fail to decode.

The code in our custom (RAI) codebase, as users of Julia, seems to have 3 possible outputs in the stack traces:

  1. It is decoded correctly. For example:
    macro expansion at /nix/store/p6p0vcy5fcmz4fw2wp368b4pasiri003-rai-server-binary-linux/rai- 
    server/packages/ThreadingUtils/src/ThreadingUtils.jl:182 [inlined]
    
  2. It shows up as unknown function. For example:
    unknown function (ip: 0x7f09a272436f)
    
  3. It shows up as jfptr_YY.*. For example:
    jfptr_YY.1_301099 at /nix/store/p6p0vcy5fcmz4fw2wp368b4pasiri003-rai-server-binary-linux/rai-server/rai-server/lib/julia/sys.so (unknown line)
    

For example, here are some stacks I see in the logs:

wait at ./asyncevent.jl:155 [inlined]
macro expansion at /nix/store/p6p0vcy5fcmz4fw2wp368b4pasiri003-rai-server-binary-linux/rai-server/packages/ThreadingUtils/src/ThreadingUtils.jl:182 [inlined]
#1 at ./threadingconstructs.jl:321
jfptr_YY.1_301099 at /nix/store/p6p0vcy5fcmz4fw2wp368b4pasiri003-rai-server-binary-linux/rai-server/rai-server/lib/julia/sys.so (unknown line)
jl_apply at /build/source/src/julia.h:1842 [inlined]
start_task at /build/source/src/task.c:932
wait at ./asyncevent.jl:155 [inlined]
macro expansion at /nix/store/p6p0vcy5fcmz4fw2wp368b4pasiri003-rai-server-binary-linux/rai-server/packages/ThreadingUtils/src/ThreadingUtils.jl:182 [inlined]
#19 at ./threadingconstructs.jl:321
unknown function (ip: 0x7f09a272436f)
jl_apply at /build/source/src/julia.h:1842 [inlined]
start_task at /build/source/src/task.c:932
wait at ./task.jl:939
wait at ./condition.jl:124
jfptr_wait_86001 at /nix/store/p6p0vcy5fcmz4fw2wp368b4pasiri003-rai-server-binary-linux/rai-server/rai-server/lib/julia/sys.so (unknown line)
_wait at ./task.jl:308
wait at ./task.jl:347 [inlined]
wait at /nix/store/p6p0vcy5fcmz4fw2wp368b4pasiri003-rai-server-binary-linux/rai-server/.julia/packages/HTTP/RQd4C/src/Servers.jl:125 [inlined]
#120 at /nix/store/p6p0vcy5fcmz4fw2wp368b4pasiri003-rai-server-binary-linux/rai-server/src/Server/Server.jl:2006 [inlined]
#with_rel_server#118 at /nix/store/p6p0vcy5fcmz4fw2wp368b4pasiri003-rai-server-binary-linux/rai-server/src/Server/Server.jl:1989
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    bugIndicates an unexpected problem or unintended behaviorobservabilitymetrics, timing, understandability, reflection, logging, ...

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions