Zyp Treatments: A slightly tailored transformation subsystem #46
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
... when deselecting them: pytest -m "not integration"
amotl
changed the title
amotl
force-pushed
the
zyp-treatments
branch
2 times, most recently
from
1859746
to
54bdb4d
Compare
Because CrateDB can not store lists of varying objects, try to normalize them, currently biased towards strings and floats.
2 tasks
amotl
requested review from
hammerhead and
seut
and removed request for
zolbatar,
juanpardo,
lservini and
wierdvanderhaar
amotl
force-pushed
the
zyp-treatments
branch
from
54bdb4d
to
7ae37a7
Compare
wierdvanderhaar
approved these changes
Sep 9, 2024
|
Thanks.
Now that fragments have been slotted in here, they don't need to be part of crate/cratedb-toolkit#255, so there became more room for software tests and features over there, if that comforts you a bit? ;] |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
About
Reading real-world data from MongoDB sometimes needs a few adjustments. Zyp Treatments propose to implement another possible variant to cope with them. They can be used independently of any runtime environment, and their concepts and ingredients can be tweaked based on new requirements as we go.
Details
Contrary to Zyp Transformations, rules of Zyp Treatments currently are not anchored to a certain node, for example using JSONPointer. Instead, they apply on all elements and levels of a document. Treatment rules are defined using Python code.
Usage
zyp-treatment-all.yamlis an example file representing a Zyp project that includes definitions of special treatments for a specific collection.References
/cc @juanpardo, @wierdvanderhaar, @hlcianfagna, @lservini, @zolbatar