NaN values kill Hamilton UI (can't serialize to JSON)

[Phrogz]

Executing a driver with nan values works fine by itself, but when the HamiltonTracker adapter is used to connect to Hamilton UI, the run fails.

Current behavior

Stack Traces

Traceback (most recent call last):
  File "/private/tmp/test/.venv/lib/python3.13/site-packages/requests/models.py", line 510, in prepare_body
    body = complexjson.dumps(json, allow_nan=False)
  File "/opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
          ~~~~~~^^^^^
  File "/opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/json/encoder.py", line 200, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/json/encoder.py", line 261, in iterencode
    return _iterencode(o, 0)
ValueError: Out of range float values are not JSON compliant: nan

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/private/tmp/test/hello.py", line 28, in <module>
    res: dict[str, float] = driver.execute(
                            ~~~~~~~~~~~~~~^
        inputs={"inp": math.nan},
        ^^^^^^^^^^^^^^^^^^^^^^^^^
        final_vars=["out"],
        ^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/private/tmp/test/.venv/lib/python3.13/site-packages/hamilton/driver.py", line 623, in execute
    self.adapter.call_all_lifecycle_hooks_sync(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        "pre_graph_execute",
        ^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
        overrides=overrides,
        ^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/private/tmp/test/.venv/lib/python3.13/site-packages/hamilton/lifecycle/base.py", line 1022, in call_all_lifecycle_hooks_sync
    getattr(adapter, hook_name)(**kwargs)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/private/tmp/test/.venv/lib/python3.13/site-packages/hamilton_sdk/adapters.py", line 159, in pre_graph_execute
    dw_run_id = self.client.create_and_start_dag_run(
        dag_template_id=dag_template_id,
    ...<2 lines>...
        outputs=final_vars,
    )
  File "/private/tmp/test/.venv/lib/python3.13/site-packages/hamilton_sdk/api/clients.py", line 467, in create_and_start_dag_run
    response = requests.post(
        f"{self.base_url}/dag_runs?dag_template_id={dag_template_id}",
    ...<11 lines>...
        verify=self.verify,
    )
  File "/private/tmp/test/.venv/lib/python3.13/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
  File "/private/tmp/test/.venv/lib/python3.13/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/test/.venv/lib/python3.13/site-packages/requests/sessions.py", line 575, in request
    prep = self.prepare_request(req)
  File "/private/tmp/test/.venv/lib/python3.13/site-packages/requests/sessions.py", line 484, in prepare_request
    p.prepare(
    ~~~~~~~~~^
        method=request.method.upper(),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<10 lines>...
        hooks=merge_hooks(request.hooks, self.hooks),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/private/tmp/test/.venv/lib/python3.13/site-packages/requests/models.py", line 370, in prepare
    self.prepare_body(data, files, json)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/test/.venv/lib/python3.13/site-packages/requests/models.py", line 512, in prepare_body
    raise InvalidJSONError(ve, request=self)
requests.exceptions.InvalidJSONError: Out of range float values are not JSON compliant: nan

Steps to replicate behavior

1. Run the code below once to prove that it works without the adapter.
2. Uncomment the code (and start a hamilton ui instance with the right username and project id) and run again.

import sys
import math

import hamilton.driver
import hamilton_sdk.adapters

def out(inp: float) -> float:
    return inp * 2

builder = hamilton.driver.Builder().with_modules(sys.modules[__name__])

# builder = builder.with_adapters(
#     hamilton_sdk.adapters.HamiltonTracker(
#         project_id=1,
#         username="gkistner",
#         dag_name="TestingEndToEnd",
#     )
# )

driver = builder.build()
res = driver.execute(inputs={"inp": math.nan},final_vars=["out"])
print(res)
#=> {'out': nan}

Library & System Information

• macOS v15.3.1
• Python v3.11.10 and v3.13.2

sf-hamilton        1.88.0
sf-hamilton-lsp    0.1.0
sf-hamilton-sdk    0.8.0
sf-hamilton-ui     0.0.17

Expected behavior

The execution runs without issue, possibly logging warnings or errors.

Ideally the Hamilton UI shows NaN values as they flow through. At worst, the run is unvisualizable.

Additional context

My codebase requires nan values to catch errors flowing through the system.

[Phrogz]

Potential fix:

1. In hamilton_sdk/tracking/utils.py change line 61

   json.dumps(item)  # Check if item is json serializable

   to

   json.dumps(item, allow_nan=False)  # Check if item is json serializable

2. Modify line 67—the except block for that call—to catch ValueError in addition to TypeError:

   except (TypeError, ValueError):

[skrawcz]

Thanks @Phrogz nice catch. I think things work if they're in a pandas series, but I don't think anyone has directly done like you have here.

Does your change work with the SDK to get around this? I see the unit tests but wondering whether it solves it end to end?

[Phrogz]

Does your change work with the SDK to get around this?

Sorry, I'm not sure what you mean. This does ~solve the problem end-to-end: the run succeeds, and the Hamilton UI shows "nan" for an input value. (Might be nicer if it showed math.nan or NaN instead of a string, but I wasn't sure how best to accomplish that.)

[elijahbenizzy]

@Phrogz @skrawcz I think this does solve it e2e. Seems like a good change!