finalize_upload (evaluation type) fails: missing version= and stale description= in datasets.upload_file()

[solarrezaei11]

Summary

POST /finalize_upload (route for evaluation dataset type) fails because function_app.py calls project_client.datasets.upload_file(...) without the now-required version= kwarg, and with a description= kwarg the current SDK no longer accepts.

Same pattern in a second call site (_register_voicelive_output_as_foundry_dataset).

Affected call sites

Both in evaluation_agent/deploy/azure-functions/function_app.py:

• Line ~1360 (in the finalize_upload evaluation-type branch):

  dataset = project_client.datasets.upload_file(
      name=dataset_name,
      file_path=tmp_path,
      description=f"Evaluation dataset: {entry_count} entries",
  )

• Line ~3580 (when registering a VoiceLive processor output as a Foundry dataset):

  dataset = project_client.datasets.upload_file(
      name=dataset_name,
      file_path=local_path,
      description=f"VoiceLive processing output (job: {job_id})",
  )

A third call site at line ~2358 (the agent eval-orchestrator path) already passes version=new_version and works — so the SDK requirement is known there but wasn’t propagated to the other two.

Repro

Fresh azd up of the evaluation_agent flavor with default SDK pins (azure-ai-projects>=2.0.0b4):

FUNC=https://<func-app>.azurewebsites.net/api
KEY=<function key>

# 1) Get an upload URL for an "evaluation" dataset
curl -s -X POST "$FUNC/get_upload_url" \
  -H "x-functions-key: $KEY" -H "Content-Type: application/json" \
  -d '{"dataset_name":"smoke-eval","dataset_type":"evaluation"}'

# 2) PUT a tiny query/response JSONL to the returned SAS URL
curl -X PUT "<SAS_URL>" -H "x-ms-blob-type: BlockBlob" --data-binary @smoke.jsonl

# 3) Finalize -> FAILS
curl -X POST "$FUNC/finalize_upload" \
  -H "x-functions-key: $KEY" -H "Content-Type: application/json" \
  -d '{"upload_id":"<id>","dataset_name":"smoke-eval","dataset_type":"evaluation"}'

Observed (first call)

{"error": "DatasetsOperations.upload_file() missing 1 required keyword-only argument: 'version'"}

Observed (after adding version="1")

{"error": "Session.request() got an unexpected keyword argument 'description'"}

Expected

{"action":"finalize_upload","status":"success","foundry_dataset_id":"azureai://...","version":"1","entries":3}

Root cause

azure-ai-projects DatasetsOperations.upload_file() changed signature:

• version is now a required keyword-only argument.
• description is no longer accepted at this call (gets forwarded into Session.request and rejected).

The line ~2358 path was updated (computes next version, retries on conflict). The other two sites weren’t.

Suggested fix

Mirror the line ~2358 pattern at both other sites:

# Compute next available version
try:
    existing = list(project_client.datasets.list())
    existing_versions = [d for d in existing if d.name == dataset_name]
    new_version = str(max(int(d.version) for d in existing_versions) + 1) if existing_versions else "1"
except Exception:
    new_version = "1"

# Drop description= (no longer accepted)
for attempt in range(5):
    try:
        dataset = project_client.datasets.upload_file(
            name=dataset_name,
            version=new_version,
            file_path=tmp_path,
        )
        break
    except Exception as e:
        if "already exists" in str(e).lower() and attempt < 4:
            new_version = str(int(new_version) + 1)
        else:
            raise

Validated: with this change applied to both sites and redeployed, finalize_upload returned {"status":"success", "foundry_dataset_id":"azureai://.../data/smoke-eval-v3/versions/1", "version":"1", "entries":3}, and list_foundry_datasets confirmed the dataset.

Happy to send a PR.

Environment

• evaluation_agent flavor on main, deployed via azd up 2026-05-29
• Region: eastus2
• azure-ai-projects>=2.0.0b4 (resolved to current latest at deploy time)
• Python 3.11 Function App