-
Notifications
You must be signed in to change notification settings - Fork 11
Conversation
Should this be a joint spec with json line? Support already exist, and just needs to be exposed |
I personally advocate for a separation of two distinct specs as we can use the merged pull requests to reflect what the engine is capable of as a single source of truth (like a spec dictionary?). Nonetheless, we need to take a decision about this statement. Do we use it as a task tracker or do we use it as a support for mapping the whole Meilisearch capabilities at a given point in time? I've added a commit on the last specification guide proposal to elaborate a specification lifecycle (github automation can't do all this steps, so it will need a lot of manual work I think). What do you mean by saying "expose"? Thinking about this one, I'm wondering if we need to expose it as an SDK's method to pass CSV file through http. It would be nice to have for sure. Especially to propose an interface for the future dashboard or DX but maybe we can just use it as a discovery tool to measure usage and payload size modifications on the CLI level. It still permits for MeiliSearch to iterate in the future. |
What I mean by expose is that this feature is already implemented and supported by the engine, and is disabled for use by the user. I get your point about isolation of specs, I'm just wondering if that since this is already implemented, and is basically the same thing, we should maybe ship the two at once, and avoid unecessary overhead. As for the sdk, what they decide to implement usually follows what MeiliSearch exposes in it's api. |
Just to follow up with you @curquiza, I will update the tech part as soon as I've updated the PR related to the change in spec definition. |
As mentioned on meilisearch/transplant#142, tech part needs to be updated to mention that. |
As mentioned in meilisearch/transplant#142 it will be a candidate for v0.22 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What bout
Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: Clément Renault <clement@meilisearch.com>
184: Transfer numbers and strings facets into the appropriate facet databases r=Kerollmops a=Kerollmops This pull request is related to #152 and changes the layout of the facets values, numbers and strings are now in dedicated databases and the user no more needs to define the type of the fields. No more conversion between the two types is done, numbers (floats and integers converted to f64) go to the facet float database and strings go to the strings facet database. There is one related issue that I found regarding CSVs, the values in a CSV are always considered to be strings, [meilisearch/specifications#28](https://github.com/meilisearch/specifications/blob/d916b57d748628c3b45500e9dfa46b50dfa2ad3f/text/0028-indexing-csv.md) fixes this issue by allowing the user to define the fields types using `:` in the "CSV Formatting Rules" section. All previous tests on facets have been modified to pass again and I have also done hand-driven tests with the 115m songs dataset. Everything seems to be good! Fixes #192. Co-authored-by: Clément Renault <clement@meilisearch.com> Co-authored-by: Kerollmops <clement@meilisearch.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you :)
Co-authored-by: Clément Renault <clement@meilisearch.com>
* Initiate csv indexation support specification * update spec file name to match pull request id * update csv indexation spec * fix code examples and typos * fix typos * update spec name * Update header part to match MeiliSearch Tracking-Issues * update spec from the equivalent ndjson spec reviews * update --data sample examples * add information about giving application/json content-type or not for a json payload * Apply suggestions from code review Co-authored-by: Clément Renault <clement@meilisearch.com> * Change curl --data param to --binary-data in examples Co-authored-by: Clément Renault <clement@meilisearch.com> * updates error codes * moved behavior about missing content-type in explanation part * Apply suggestions from code review Co-authored-by: Clément Renault <clement@meilisearch.com> Co-authored-by: Clément Renault <clement@meilisearch.com>
* Initiate csv indexation support specification * update spec file name to match pull request id * update csv indexation spec * fix code examples and typos * fix typos * update spec name * Update header part to match MeiliSearch Tracking-Issues * update spec from the equivalent ndjson spec reviews * update --data sample examples * add information about giving application/json content-type or not for a json payload * Apply suggestions from code review Co-authored-by: Clément Renault <clement@meilisearch.com> * Change curl --data param to --binary-data in examples Co-authored-by: Clément Renault <clement@meilisearch.com> * updates error codes * moved behavior about missing content-type in explanation part * Apply suggestions from code review Co-authored-by: Clément Renault <clement@meilisearch.com> Co-authored-by: Clément Renault <clement@meilisearch.com>
* Initiate csv indexation support specification * update spec file name to match pull request id * update csv indexation spec * fix code examples and typos * fix typos * update spec name * Update header part to match MeiliSearch Tracking-Issues * update spec from the equivalent ndjson spec reviews * update --data sample examples * add information about giving application/json content-type or not for a json payload * Apply suggestions from code review Co-authored-by: Clément Renault <clement@meilisearch.com> * Change curl --data param to --binary-data in examples Co-authored-by: Clément Renault <clement@meilisearch.com> * updates error codes * moved behavior about missing content-type in explanation part * Apply suggestions from code review Co-authored-by: Clément Renault <clement@meilisearch.com> Co-authored-by: Clément Renault <clement@meilisearch.com>
* Initiate csv indexation support specification * update spec file name to match pull request id * update csv indexation spec * fix code examples and typos * fix typos * update spec name * Update header part to match MeiliSearch Tracking-Issues * update spec from the equivalent ndjson spec reviews * update --data sample examples * add information about giving application/json content-type or not for a json payload * Apply suggestions from code review Co-authored-by: Clément Renault <clement@meilisearch.com> * Change curl --data param to --binary-data in examples Co-authored-by: Clément Renault <clement@meilisearch.com> * updates error codes * moved behavior about missing content-type in explanation part * Apply suggestions from code review Co-authored-by: Clément Renault <clement@meilisearch.com> Co-authored-by: Clément Renault <clement@meilisearch.com>
* Indexing NDJSONs (#29) * initialize a draft for json lines indexation support specification * update filename number to match related pull-request * update specs * update link to CSV spec * update spec name * Apply typos correction from code review Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com> * fix typo * update impact on documentation part * replace file by data * add information about giving application/json content-type or not for a json payload * updates error codes, curl instructions * moved behavior about missing content-type in explanation part Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com> * Indexing CSVs (#28) * Initiate csv indexation support specification * update spec file name to match pull request id * update csv indexation spec * fix code examples and typos * fix typos * update spec name * Update header part to match MeiliSearch Tracking-Issues * update spec from the equivalent ndjson spec reviews * update --data sample examples * add information about giving application/json content-type or not for a json payload * Apply suggestions from code review Co-authored-by: Clément Renault <clement@meilisearch.com> * Change curl --data param to --binary-data in examples Co-authored-by: Clément Renault <clement@meilisearch.com> * updates error codes * moved behavior about missing content-type in explanation part * Apply suggestions from code review Co-authored-by: Clément Renault <clement@meilisearch.com> Co-authored-by: Clément Renault <clement@meilisearch.com> * Geosearch (#59) * Initialize draft specification for geo-search feature * add future possibilities * Update specification * mention errors and aspects about filterableAttributes and sortableAttributes * Add measure and finalized key changes * Add description in OpenApi * remove old falsy sentence * Add definition and explanation for error * fix rebase on develop * Specify missing edge cases (#63) * Initialize draft specification for geo-search feature * add future possibilities * Update specification * mention errors and aspects about filterableAttributes and sortableAttributes * Add measure and finalized key changes * Add description in OpenApi * remove old falsy sentence * Add definition and explanation for error * fix rebase on develop * update open-api.yml with description on _geoPoint built-in sort rule and _geo field * Apply suggestions from code review Co-authored-by: gui machiavelli <gui@meilisearch.com> * remove - char in geo-search * update invalid_geo_field error definition Co-authored-by: gui machiavelli <gui@meilisearch.com> * Patch GeoSearch specification to mention technical limit on `desc` ordering around a _geoPoint (#66) * mention decision and expected behavior for a desc ordering around a geoPoint * add desc ordering around a geoPoint as a future possibility * Patch error codes for csv and ndjson formats specs (#64) * specify error codes dedicated to payload format for post/put documents endpoints * Udpdate error codes naming * Add errors definition * update errors and cURL examples * Add alternative message for reserved keyword and update invalid_criterion error definition (#67) * add alternative message for reserved keyword and update invalid_criterion error * update error name in link field for invalid_ranking_rule error * update invalid_geo_field error message * fix typo * Add future possibilities from irevoire to geosearch spec file (#74) * add descending order capability for _geoPoint built-in sort (#77) * Add variant message to ensure _geoPoint and _geoRadius expressions are not used as a ranking rule to help the user (#78) * Mention the supported separator character (#81) * Add content-type header requirements * update bump.yml configuration Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com> Co-authored-by: Clément Renault <clement@meilisearch.com> Co-authored-by: gui machiavelli <gui@meilisearch.com>
The goal of this PR is to specify CSV file indexing feature.