Skip to content

Added unnest - #1822

Draft
ilongin wants to merge 5 commits into
mainfrom
ilongin/1820-unnest
Draft

Added unnest#1822
ilongin wants to merge 5 commits into
mainfrom
ilongin/1820-unnest

Conversation

@ilongin

@ilongin ilongin commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Adds DataChain.unnest(col) - expands a list column into rows, one row per element. Sibling columns are repeated. Equivalent to SQL UNNEST, Spark explode, pandas .explode(). Empty lists and None drop the source row (inner semantics).

Naming

The industry-standard name for this is explode, but datachain already has an explode() that does something different (JSON-string-to-columns). To get to the standard name without breaking users, the plan is:

  1. Ship unnest now (this PR).
  2. In a follow-up, rename the existing explode to something descriptive (e.g. expand_json) and keep explode as a deprecated alias that warns.
  3. In a later release, once explode is no longer the JSON helper, alias unnest as explode and deprecate unnest.
  4. After a deprecation window, drop the old names.

This avoids a single confusing rename and lets users migrate one step at a time.

@ilongin
ilongin marked this pull request as draft June 18, 2026 12:30
@ilongin ilongin linked an issue Jun 18, 2026 that may be closed by this pull request
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 18, 2026

Copy link
Copy Markdown

Deploying datachain with  Cloudflare Pages  Cloudflare Pages

Latest commit: d2e953f
Status: ✅  Deploy successful!
Preview URL: https://18bd0b1e.datachain-2g6.pages.dev
Branch Preview URL: https://ilongin-1820-unnest.datachain-2g6.pages.dev

View logs

@codecov

codecov Bot commented Jun 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.54839% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/datachain/lib/dc/datachain.py 93.54% 1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@dmpetrov

Copy link
Copy Markdown
Contributor

@ilongin it might be a little bit premature implementing this. I expect a lot of unset-challenges with the coming Union[] #1824 that's required for proper LLM resposes decoding.

I'd wait for Union first.

Also, as we discussed before, it’s better not to pick up issues that haven’t been prioritized yet - or ask about it (I'm not sure if you had some other discussions about this issue).

@ilongin
ilongin marked this pull request as ready for review June 22, 2026 14:44
@ilongin
ilongin marked this pull request as draft June 30, 2026 04:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add unnest() to explode a list column into rows

3 participants