-
Notifications
You must be signed in to change notification settings - Fork 19.6k
Improve save_model
and load_model
#19852
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
d1393ed
to
ccb14c9
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #19852 +/- ##
=======================================
Coverage 78.83% 78.84%
=======================================
Files 498 498
Lines 45864 45887 +23
Branches 8450 8455 +5
=======================================
+ Hits 36159 36181 +22
Misses 7999 7999
- Partials 1706 1707 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
a tiny suggestion: the form |
ccb14c9
to
21fa376
Compare
That's cleaner. Thanks for the suggestion. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the fix! This is great.
keras/src/saving/saving_lib.py
Outdated
@@ -202,6 +227,8 @@ def _load_model_from_fileobj(fileobj, custom_objects, compile, safe_mode): | |||
weights_store.close() | |||
if asset_store: | |||
asset_store.close() | |||
if weights_file_path: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should put the unlink
in a try/except/finally block to make sure it gets executed no matter what.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have added the try/finally blocks.
In _save_model_to_fileobj
, if an error is raised, the bad model.weights.h5
won't be written to the zip file.
In _load_model_from_fileobj
, if an error is raised, the extracted model.weights.h5
will be deleted no matter what.
Tests have been added to verify these behaviors.
Improve `save_model` and `load_model` Address comments
21fa376
to
99a8dcc
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the contribution!
Fixes #19793
save_model
will first open the .h5 (in the same working dir asfilepath
), then write it to the zipfile at the end.load_model
will first extract the .h5, then load it usingh5py.File
. The weights file will be deleted at the end.Result:
Using GPT2 from KerasNLP
save_model
save_model
load_model
load_model
The only regression is that
save_model
might be slightly slower than before.