Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue #459: Deprecate StringHistogram for backward compatibility #485

Merged
merged 15 commits into from
Dec 3, 2024

Conversation

osopardo1
Copy link
Member

@osopardo1 osopardo1 commented Nov 22, 2024

Description

Fixes #459

Type of change

Bug Fix to ensure backward compatibility within new versions.

Checklist:

Here is the list of things you should do before submitting this pull request:

  • New feature / bug fix has been committed following the Contribution guide.
  • Add logging to the code following the Contribution guide.
  • Add comments to the code (make it easier for the community!).
  • Change the documentation.
  • Add tests.
  • Your branch is updated to the main branch (dependent changes have been merged).

How Has This Been Tested? (Optional)

This will be tested with an old table written with histogram transformation and version 0.7.0 of Qbeast Spark.

@osopardo1 osopardo1 requested a review from Jiaweihu08 November 26, 2024 07:23
@osopardo1 osopardo1 marked this pull request as ready for review November 26, 2024 07:23
@osopardo1
Copy link
Member Author

osopardo1 commented Nov 28, 2024

After discussion, we agreed on:

  • Allow writing with StringHistogramTransformer and StringHistogramTransformation.
  • Use superseded and merge methods to change between StringHIstogram and CDF for the moment.
  • Add a method to change from histogram to cdf metadata.

I would update the PR in the upcoming days.

@osopardo1 osopardo1 requested a review from Jiaweihu08 November 29, 2024 13:04
@osopardo1
Copy link
Member Author

I would merge with main and then we can proceed to the squash 🐒

# Conflicts:
#	src/test/scala/io/qbeast/spark/index/model/transformer/TransformerIndexingTest.scala
Copy link
Member

@Jiaweihu08 Jiaweihu08 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job!

@Jiaweihu08 Jiaweihu08 merged commit d64f0a6 into Qbeast-io:main Dec 3, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rollback and deprecate StringHistogramTransformation
2 participants