Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add sagemaker predict PoC to eland #2

Merged
merged 30 commits into from
Sep 9, 2022
Merged
Changes from 1 commit
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
83a6230
Merge pull request #3 from LEFTA98/data-setup-fix
LEFTA98 Jul 8, 2022
3644f84
added initial connection to predicting with sagemaker
LEFTA98 Jul 19, 2022
fd017db
added sagemaker predict api
LEFTA98 Jul 21, 2022
55b2f4f
added band-aid to fix iterating over rows
LEFTA98 Jul 25, 2022
5f2d9aa
debugging indexing issue
LEFTA98 Jul 25, 2022
5679b13
reverted indexing change for sagemaker predict
LEFTA98 Jul 28, 2022
727c0db
Merge pull request #4 from LEFTA98/sagemaker_integration
LEFTA98 Aug 2, 2022
35ccb3b
added deprecation warnings to ml module
LEFTA98 Aug 2, 2022
396d7ec
refactoring elasticsearch names to opensearch
LEFTA98 Aug 2, 2022
53df988
continued renaming opensearch variables
LEFTA98 Aug 3, 2022
e4881d8
more renaming changes
LEFTA98 Aug 4, 2022
410ad0e
first commit for ml common integration
LEFTA98 Aug 11, 2022
7f3d7dd
PoC for model upload
LEFTA98 Aug 12, 2022
a3bf832
renamed model chunk uploading path
LEFTA98 Aug 23, 2022
1b79f62
added total chunks to model upload
LEFTA98 Aug 24, 2022
18ebcaa
fixed docstring typo
LEFTA98 Aug 31, 2022
2d60689
added first iteration of custom model load supprot
LEFTA98 Sep 6, 2022
ed1f7f9
PR feedback
LEFTA98 Sep 6, 2022
564f071
implement PR feedback
LEFTA98 Sep 6, 2022
96024af
removed unsupported features
LEFTA98 Sep 6, 2022
06138ac
Merge pull request #5 from opensearch-project/custom_model_fixes
LEFTA98 Sep 6, 2022
0afe29a
renaming all instances of elastic in code
LEFTA98 Sep 8, 2022
0f5e2cb
created new dev requirements file
LEFTA98 Sep 8, 2022
ace0afa
typo fix
LEFTA98 Sep 9, 2022
15a8b16
Merge pull request #6 from opensearch-project/main-amzn
LEFTA98 Sep 9, 2022
a108086
fix merge issues
LEFTA98 Sep 9, 2022
9d35696
Merge pull request #4 from opensearch-project/ml_common_integration
LEFTA98 Sep 9, 2022
1972f81
Merge branch 'sagemaker_predict' into rename_eland_to_opensearch
LEFTA98 Sep 9, 2022
9787843
Merge pull request #3 from opensearch-project/rename_eland_to_opensearch
LEFTA98 Sep 9, 2022
8db8492
fix merge issues
LEFTA98 Sep 9, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fixed docstring typo
  • Loading branch information
LEFTA98 committed Aug 31, 2022
commit 18ebcaa735b6e98d9800ab8a37bde2647b310b00
6 changes: 3 additions & 3 deletions opensearch_py_ml/sagemaker_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

import numpy as np
from opensearch_py_ml import DataFrame
from typing import List, Optional
from typing import List, Optional, Dict, Tuple, Any
from math import ceil

from sagemaker import RealTimePredictor, Session
Expand All @@ -21,7 +21,7 @@ def make_sagemaker_prediction(endpoint_name: str,
column_order: Optional[List[str]] = None,
chunksize: int = None,
sort_index: Optional[str] = '_doc'
)-> np.array:
) -> Tuple[List[Any], Dict[Any, Any]]:
"""
Make a prediction on an opensearch_py_ml dataframe using a deployed SageMaker model endpoint.

Expand All @@ -44,7 +44,7 @@ def make_sagemaker_prediction(endpoint_name: str,

Returns
----------
np.array representing the output of the model on input data
list representing the indices, dictionary representing the output of the model on input data
"""
predictor = RealTimePredictor(endpoint=endpoint_name, sagemaker_session=sagemaker_session, content_type='text/csv')
data = data.drop(columns=target_column)
Expand Down