Skip to content

HumanLabeledDataset.from_csv silently coerces numeric response and objective text #2978

Description

@Garyouki

Describe the bug

HumanLabeledDataset.from_csv() lets pandas infer the types of text columns, then converts the inferred values back to strings. This loses leading zeros, scientific notation, and decimal formatting before the response is scored. Objectives and harm categories are affected as well. Both the UTF-8 and Latin-1 paths use type inference.

Steps/Code to Reproduce

from pathlib import Path
from tempfile import TemporaryDirectory
from pyrit.score import HumanLabeledDataset, MetricsType

with TemporaryDirectory() as directory:
    path = Path(directory) / "sample.csv"
    path.write_text("assistant_response,objective,human_score\n00123,00456,1\n", encoding="utf-8")
    entry = HumanLabeledDataset.from_csv(
        csv_path=path, metrics_type=MetricsType.OBJECTIVE, version="1.0"
    ).entries[0]
    print(repr(entry.conversation[0].message_pieces[0].original_value))
    print(repr(entry.objective))

Expected Results

Response '00123' and objective '00456'; numeric-looking text should retain its spelling.

Actual Results

Response '123' and objective '456'. No exception or warning is raised. Similarly, scientific notation and trailing decimal zeros are normalized rather than preserved.

Versions

PyRIT main at b6a18a20, Python 3.12.3, pandas 3.0.6, Linux. No external model or service is needed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions