fix: include class-based UDF instance state in hash

[ilongin]

Fixes #1903.

Class-based UDFs (Mapper / Generator / Aggregator subclasses) hashed to the same value regardless of constructor args, so the second .agg(CountAbove(3), ...) call was served the first .agg(CountAbove(0), ...)'s cached rows. No error, no version bump, wrong numbers.

Changes:

• UDFBase.hash() now mixes in _hash_state() bytes when the UDF is class-based (self._func is None). Default _hash_state() returns filtered_cloudpickle_dumps(self), so instance attributes like self.limit end up in the hash.
• ArrowGenerator and HFGenerator override _hash_state() to skip self.output_schema, which is a dynamically-created pydantic class with a random name suffix. The schema's stable field shape is already in self.output.hash(), so no signal is lost and dc.read_csv / dc.read_parquet hashes stay deterministic across calls.
• Tests: unit hash tests pin the fix and the determinism guarantee; a func test reproduces the exact issue scenario.

Function-based UDFs (plain lambdas and def functions passed to .map()) are untouched - the pickle branch only runs for class-based UDFs.

[cloudflare-workers-and-pages]

Deploying datachain with    Cloudflare Pages

Latest commit:	08fc4ba
Status:	✅  Deploy successful!
Preview URL:	https://5b439428.datachain-2g6.pages.dev
Branch Preview URL:	https://ilongin-1903-class-udf-hash.datachain-2g6.pages.dev

View logs

[codecov]

Codecov Report

❌ Patch coverage is 96.87500% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/datachain/lib/hf.py	78.57%	1 Missing and 2 partials ⚠️

📢 Thoughts on this report? Let us know!

[Copilot]

Pull request overview

Updates UDF cache identities to distinguish class instances with different constructor arguments, preventing stale cached results.

Changes:

• Adds normalized value hashing for class-based UDF constructor arguments.
• Preserves deterministic Arrow/Hugging Face and LLM identities.
• Adds unit and functional regression coverage.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/datachain/hash_utils.py	Adds normalized value hashing utilities.
src/datachain/lib/udf.py	Incorporates constructor arguments into UDF hashes.
src/datachain/lib/arrow.py	Excludes dynamic output schemas from constructor hashes.
src/datachain/lib/hf.py	Excludes dynamic output schemas from constructor hashes.
src/datachain/llm/spec.py	Reuses shared hash normalization.
tests/func/test_udf.py	Tests distinct aggregate results by constructor state.
tests/unit/lib/test_arrow.py	Tests Arrow constructor hashing.
tests/unit/lib/test_hf.py	Tests Hugging Face constructor hashing.
tests/unit/lib/test_llm.py	Removes superseded canonicalization coverage.
tests/unit/lib/test_udf.py	Tests UDF hash variation and determinism.
tests/unit/test_hash_utils.py	Tests hash-value normalization.
tests/unit/test_query_steps_hash.py	Updates expected query hashes.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.