Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add validation to check max int limit in train API #159

Merged
merged 1 commit into from
Oct 27, 2021

Conversation

jmazanec15
Copy link
Member

Signed-off-by: John Mazanec jmazane@amazon.com

Description

This PR checks to see if the int parameters in the training API are within integer bounds. If not, it will throw an illegal argument exception which will get propagated back to the user as a 400 error.

For validation, we use the NumberFieldMappers INTEGER type's parser. Code can be found here.

Validation:

curl -X POST "localhost:9200/_plugins/_knn/models/_train" -H 'Content-Type: application/json' -d'
{
  "training_index": "train_index",
  "training_field": "train_field",
  "dimension": 8,
  "description": "this should be allowed to be null",
  "max_training_vector_count": 1312321312313123112312123123,
  "method": {
      "name":"ivf",
      "engine":"faiss",
      "space_type": "l2",
      "parameters":{
        "nlist":16,
        "encoder":{
            "name":"pq",
            "parameters":{
                "code_size":8,
                "code_count": 2
            }
        }
      }
  }
}
'
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Value [1312321312313123112312123123] is out of range for an integer"}],"type":"illegal_argument_exception","reason":"Value [1312321312313123112312123123] is out of range for an integer"},"status":400}

Check List

  • Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: John Mazanec <jmazane@amazon.com>
@jmazanec15 jmazanec15 added the Bug Fixes Changes to a system or product designed to handle a programming bug/glitch label Oct 27, 2021
Copy link
Member

@vamshin vamshin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks

@codecov-commenter
Copy link

Codecov Report

Merging #159 (5ccc22b) into main (c14f977) will not change coverage.
The diff coverage is 33.33%.

Impacted file tree graph

@@            Coverage Diff            @@
##               main     #159   +/-   ##
=========================================
  Coverage     82.82%   82.82%           
  Complexity      801      801           
=========================================
  Files           117      117           
  Lines          3580     3580           
  Branches        339      339           
=========================================
  Hits           2965     2965           
  Misses          461      461           
  Partials        154      154           
Impacted Files Coverage Δ
...nsearch/knn/plugin/rest/RestTrainModelHandler.java 61.90% <33.33%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c14f977...5ccc22b. Read the comment docs.

@jmazanec15 jmazanec15 merged commit 769a137 into opensearch-project:main Oct 27, 2021
jmazanec15 added a commit to jmazanec15/k-NN-1 that referenced this pull request Oct 28, 2021
Signed-off-by: John Mazanec <jmazane@amazon.com>
martin-gaievski pushed a commit to martin-gaievski/k-NN that referenced this pull request Mar 7, 2022
Signed-off-by: John Mazanec <jmazane@amazon.com>
martin-gaievski pushed a commit to martin-gaievski/k-NN that referenced this pull request Mar 7, 2022
Signed-off-by: John Mazanec <jmazane@amazon.com>
Signed-off-by: Martin Gaievski <gaievski@amazon.com>
martin-gaievski pushed a commit to martin-gaievski/k-NN that referenced this pull request Mar 30, 2022
Signed-off-by: John Mazanec <jmazane@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Fixes Changes to a system or product designed to handle a programming bug/glitch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants