Skip to content

fix: fix inconsistent metadata when executing multiple statements#40

Merged
jackye1995 merged 1 commit intolance-format:mainfrom
majin1102:fix-unconsistency
Aug 6, 2025
Merged

fix: fix inconsistent metadata when executing multiple statements#40
jackye1995 merged 1 commit intolance-format:mainfrom
majin1102:fix-unconsistency

Conversation

@majin1102
Copy link
Contributor

fix #16

@github-actions github-actions bot added the enhancement New feature or request label Jul 23, 2025
@majin1102
Copy link
Contributor Author

majin1102 commented Jul 23, 2025

The reason here is the dataset cache in executors is out of date. So adding a scanId in the cache key could fix the issue.

I think eventually we need to eliminate the dataset dependency in reader tasks. But this may be a long story.

@majin1102 majin1102 changed the title feat: fix inconsistent metadata when executing multiple statements fix: fix inconsistent metadata when executing multiple statements Jul 23, 2025
@github-actions github-actions bot added the bug Something isn't working label Jul 23, 2025
@majin1102
Copy link
Contributor Author

@yanghua Could you help triggering ci again?
It seems timeout. But local environment works

@yanghua
Copy link

yanghua commented Jul 30, 2025

@majin1102 Have triggered.

Copy link
Contributor

@jackye1995 jackye1995 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, I was noticing the issue and about to fix it. Looks good to me!

@jackye1995 jackye1995 merged commit b668af8 into lance-format:main Aug 6, 2025
14 of 16 checks passed
jiaoew1991 pushed a commit to jiaoew1991/lance-spark that referenced this pull request Sep 16, 2025
jiaoew1991 pushed a commit to jiaoew1991/lance-spark that referenced this pull request Dec 1, 2025
jiaoew1991 pushed a commit to jiaoew1991/lance-spark that referenced this pull request Feb 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sparksql select throws error after a select and insert sql

3 participants