Hi folks, I know this is an unusual request and not quite the right place; just hoping for a pointer or two 🙏

Motivation

My company, RelationalAI (GitHub, Website) has a complex server written in Julia, and uses Datadog heavily for observability, including writing our own datadog metrics and tracing (not OSS yet) libraries. We've also added an allocation profiler and heap snapshotter to Julia.

We'd love to use Datadog profiling for Julia too! Sadly it's not supported yet, as Julia not a common language for server apps.

Thankfully, Julia already has a profiler which outputs PProf files, so I was hoping we could mimic the official SDKs — i.e. post a PProf file to the same endpoint, and have it show up. In the meantime, we are sending around .pb.gz files and screenshots of the pprof -web tool on Slack 😛

Expectation

I wrote up https://github.com/vilterp/DatadogProfileUploader.jl, containing my attempt to mimic the request made by this package.

I even set the tags to show the language as Go, just in case unknown languages were filtered out. Was hoping that the profile would show up as if it were a Go profile.

Result

I ran it against a local agent (localhost:8126). The agent returned 200, but the profile never showed up in the profile list in the UI. Posting directly to Datadog with my API key also returned a 200, but with no profile in the UI.

Another experiment

To validate that other things were correct, including my local agent, the pprof file, my API key, etc, I tried another hack: made a branch of the Go SDK (this repo) which would read a Julia pprof file from disk and upload it, instead of taking a Go profile.

This works, showing Julia stack frames in the UI:

Profile: https://app.datadoghq.com/profiling/search?query=&event=AQAAAYP3wP6gxJKPQAAAAABBWVAzd01yN0FBRDJPXzRhVV9sYlFnQUE&my_code=disabled&profileId=AYP3wMr7AAD2O_4aU_lbQgAA&viz=stream&start=1666294591761&end=1666308991761&paused=false

Branch here: https://github.com/DataDog/dd-trace-go/compare/main...vilterp:dd-trace-go:trying-julia?expand=1

So there is hope that it's possible :) It just seems like something about the request I'm making is subtly different from the one coming from the official Go SDK, and I have no way of telling what it is, since the API doesn't return an error, and I can't see any internal logs with an error in them.

Here's an example request from my Julia code: https://gist.github.com/vilterp/92978cf5441b6794f64dda81749afeec (intercepted locally by requesting to nc -l 8126)

Request for help

Is there anything obviously wrong with the request I'm making? Is there a better channel to go through to get help on this? It'd be appreciated, as we have a lot of performance issues which we'd love to use DD profiling for. If all else fails, we could use a Go sidecar process (forked version of the Go SDK) to upload our Julia profiles 😅 Thanks! 🙏