chore(llmobs): support running experiment evals

[Yun-Kim]

MLOB-3269

Adds support for running experiment evaluator functions, and merging task/evaluator results into one object.

Writing result objects to datadog will come in a future PR.

Checklist

• PR author has checked that all the criteria below are met
• The PR description includes an overview of the change
• The PR description articulates the motivation for the change
• The change includes tests OR the PR description describes a testing strategy
• The PR description notes risks associated with the change, if any
• Newly-added code is easy to change
• The change follows the library release note guidelines
• The change includes or references documentation updates if necessary
• Backport labels are set (if applicable)

Reviewer Checklist

• Reviewer has checked that all the criteria below are met
• Title is accurate
• All changes are related to the pull request's stated goal
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Newly-added code is easy to change
• Release note makes sense to a user of the library
• If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[github-actions]

CODEOWNERS have been resolved as:

ddtrace/llmobs/_constants.py                                            @DataDog/ml-observability
ddtrace/llmobs/_experiment.py                                           @DataDog/ml-observability
ddtrace/llmobs/_llmobs.py                                               @DataDog/ml-observability
ddtrace/llmobs/_writer.py                                               @DataDog/ml-observability

[github-actions]

Bootstrap import analysis

Comparison of import times between this PR and base.

Summary

The average import time from this PR is: 284 ± 4 ms.

The average import time from base is: 288 ± 4 ms.

The import time difference between this PR and base is: -3.3 ± 0.2 ms.

Import time breakdown

The following import paths have shrunk:

ddtrace.auto 2.114 ms (0.74%)

ddtrace.bootstrap.sitecustomize 1.429 ms (0.50%)

ddtrace.bootstrap.preload 1.429 ms (0.50%)

ddtrace.internal.remoteconfig.client 0.682 ms (0.24%)

ddtrace 0.685 ms (0.24%)

ddtrace.internal._unpatched 0.032 ms (0.01%)

json 0.032 ms (0.01%)

json.decoder 0.032 ms (0.01%)

re 0.032 ms (0.01%)

enum 0.032 ms (0.01%)

types 0.032 ms (0.01%)

[pr-commenter]

Benchmarks

Benchmark execution time: 2025-07-15 02:19:22

Comparing candidate commit 25cf67f in PR branch yunkim/dne-run-evals with baseline commit c70bb0d in branch yunkim/dne-run-tasks.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 548 metrics, 2 unstable metrics.