CU-869a2kpv0 Add method for model card load off disk #111
Conversation
medcat-v2/medcat/cat.py
Outdated
| """ | ||
| # unpack if needed | ||
| if model_pack_path.endswith(".zip"): | ||
| model_pack_path = cls.attempt_unpack(model_pack_path) |
There was a problem hiding this comment.
Hey as I've also got some interest in this
Instead of unpacking the whole zip, can you just pull the json out direct?
EG I have a 1GB zip but just need 10kb json right now, would prefer not doubling the storage usage in the cloud just to show model info
There was a problem hiding this comment.
Should be able to do that. My assumption was that if you're trying to access it, it's fine to unpack as well. But I'm sure I don't need to tell you what I do when I assume...
There was a problem hiding this comment.
Added an option to do that, but disbaled by default.
There was a problem hiding this comment.
Yeah it's a fair assumption, it's a "nice to have one day, maybe never" ask from my side, probably bigger from the mlops side
For me - I want to deploy like 1000 instances of the service, but maybe I'm not sure exactly which instance has the model I want, so I could check the /info API first 1k times to find the right one. Right now from an service perspective, medcat is lazy loaded on the first process call, so would cool if /info didnt use any resources up.
I have no plan to really do this but in general it feels like it should be able to get info with minimal side effects.
| return model_card | ||
| return json.dumps(model_card, indent=2, sort_keys=False) | ||
|
|
||
| @overload |
There was a problem hiding this comment.
purpose of @overload here rather than -> Union[str, ModelCard] as you've done in the impl?
There was a problem hiding this comment.
This is for typing for the contracts. mypy needs to know that if as_dict=True, we get a ModelCard (i.e a TypedDict) and we get a str if as_dict=False .
If we didn't provide the @overload methods, mypy would always treat the return value as Union[str, ModelCard], even though the type is deterministic based on as_dict.
3 participants
Adds method to load model card off disk without loading model pack (though this may include unzipping the archive).
Also adds a few simple tests for it.
And updates the test time model pack to a newer one (the old one was in an older format that didn't have the model card).