Sherif akoush/quickfix/get names from metadata endpoint #393
Conversation
sakoush
force-pushed
the
SherifAkoush/quickfix/get_names_from_metadata_endpoint
branch
from
c0c57da
to
540a11b
Compare
| @@ -51,6 +57,19 @@ def remote_predict( | |||
| return InferenceResponse.parse_raw(response_raw.text) | |||
|
|
|||
|
|
|||
| def remote_metadata(url: str) -> MetadataModelResponse: | |||
| """Get metadata from v2 endpoint""" | |||
| response_raw = requests.get(url) | |||
There was a problem hiding this comment.
Not that it matters much, but should we make this call asynchronous?
There was a problem hiding this comment.
thats a good point. having a look at the caller load, it has also load_explainer that we need to do async so that the entire function (i.e load) is truly async.
I am tempted however to leave it as a separate ticket?
| if metadata is not None: | ||
| if metadata.inputs: | ||
| # we only support a big single input numpy | ||
| input_name = metadata.inputs[0].name |
There was a problem hiding this comment.
Do we also need to pass on the datatype, shape and parameters from the metadata?
There was a problem hiding this comment.
the type and shape is inferred from the payload. I think we can add a warning it they dont match? I didnt want to get into converting the payload according to the metadata as part of this ticket?
| # For List[str] (e.g. AnchorText), we use StringCodec for input | ||
| input_payload_codec = StringCodec if type(input_data) == list else NumpyCodec | ||
| v2_request = InferenceRequest( | ||
| id=id_name, |
There was a problem hiding this comment.
I'm not 100% sure, but I think the executor will set this to a unique value. Therefore, should we leave the id field empty?
There was a problem hiding this comment.
I am not sure about SC, maybe this is the case. However locally it will not work with a None id (for triron case).
| ) | ||
| ], | ||
| # set outputs as empty list, this will return everything? |
There was a problem hiding this comment.
I don't think that the protocol itself defines what should happen, so I guess that the behaviour will depend on the model and model runtime.
There was a problem hiding this comment.
actually this comment is not valid anymore as we get that from the metadata.
Fetch metadata from v2 endpoint inference model and use the info to construct the internal inference request from alibi explain model.
Also in this pr: