Make batch norm mask shape error more descriptive #19829

pandyah5 · 2024-06-10T18:28:13Z

This PR fixes #19818 by adding a more descriptive message to the mask mismatch exception in batch normalization.

Before:

After (Highlighted for clarity):

google-cla · 2024-06-10T18:28:17Z

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

gbaned · 2024-06-11T03:23:24Z

Hi @pandyah5 Can you please sign CLA? Thank you!

pandyah5 · 2024-06-11T06:01:07Z

@gbaned I have signed the CLA and rescanned the PR for the check

codecov-commenter · 2024-06-11T06:11:33Z

Codecov Report

Attention: Patch coverage is 33.33333% with 2 lines in your changes missing coverage. Please review.

Project coverage is 78.84%. Comparing base (2305fad) to head (0502951).
Report is 7 commits behind head on master.

Files	Patch %	Lines
...as/src/layers/normalization/batch_normalization.py	33.33%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #19829       +/-   ##
===========================================
+ Coverage   56.52%   78.84%   +22.32%     
===========================================
  Files         498      498               
  Lines       45801    45846       +45     
  Branches     8440     8448        +8     
===========================================
+ Hits        25890    36149    +10259     
+ Misses      18330     7995    -10335     
- Partials     1581     1702      +121

Flag	Coverage Δ
keras	`78.70% <33.33%> (+22.17%)`	⬆️
keras-jax	`62.38% <33.33%> (?)`
keras-numpy	`56.63% <33.33%> (+0.11%)`	⬆️
keras-tensorflow	`63.68% <33.33%> (?)`
keras-torch	`62.36% <33.33%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

fchollet · 2024-06-11T14:35:17Z

keras/src/layers/normalization/batch_normalization.py

+            if len(mask.shape) != len(inputs.shape) - 1:
+                # Raise a value error
+                raise ValueError(
+                    "The mask provided should be one dimension less than the inputs."


Add:

"Received: mask.shape={mask.shape}, inputs.shape={inputs.shape}"

This makes the error easier to debug.

Added it! The new error message looks as follows:

fchollet

LGTM, thanks!

Fix `LayerNormalization.get_config` (keras-team#19807) Propagate kwargs through `keras.ops.isclose` (keras-team#19782) * propagate kwargs through isclose this allows passing atol and rtol * switch isclose **kwargs to explicit kwargs * reduce line lengths * fix ops.isclose signature * fix ops.IsClose compute_output_spec signature * implement isclose rtol atol equal_nan args for all backends * shorten line lengths again * revert using tf.experimental.numpy.isclose tensorflow version now uses code inspired from tf.experimental.numpy.isclose * fix lint * add docs for new parameters Faster in_top_k implementation for Jax backend (keras-team#19814) * Faster in_top_k implementation. * Fix bug in rank computation. Fix CI Fix TypeError in `Lambda.from_config` (keras-team#19827) fixing dmtree.is_nested() and parameterized tree test (keras-team#19822) Fix `keras.ops.repeat` cannot return an expected shape when `x` is a … (keras-team#19826) * Fix `keras.ops.repeat` cannot return an expected shape when `x` is a `KerasTensor` and the `axis` is `None` * Test dynamic is still dynamic after repetition * Improve error messages `Metric.variables` is now recursive. (keras-team#19830) This allows it to surface variables from metrics nested at any depth. Previously, metrics within metrics within metrics would not have their variables tracked in JAX, causing them to not be updated. Fix `get_file` when the HTTP response has no `Content-Length` header (keras-team#19833) Add `ops.switch` (keras-team#19834) * Add `ops.switch` * Update tests * Fix out-of-bound issue * Revert `torch.cond` Use `absl.testing.parameterized` for `tree_test.py`. (keras-team#19842) For consistency, use `absl.testing.parameterized` instead of `parameterized` for `tree_test.py` since that is used for all other tests. It's one less dependency. It also says `optree` or `dmtree` in each test name. Make batch norm mask shape error more descriptive (keras-team#19829) * Made batch norm mask shape error more descriptive * Added shape info in mask error message to help with degugging Fix code style doc: `ops.slice` (keras-team#19843) corrected the example code in unit_normalization.py (keras-team#19845) Added missing closing bracket and exact output value in example code after replicating the code. Adjust code example Add `training` argument to `Model.compute_loss()`. (keras-team#19840) This allows models to perform different computations during training and evaluation. For instance, some expensive to compute metrics can be skipped during training and only computed during evaluation. Note that backwards compatibility with overrides that do not have the `training` argument is maintained. Fix the compatibility issues of `Orthogonal` and `GRU` (keras-team#19844) * Add legacy `Orthogonal` class name * Add legacy `implementation` arg to `GRU` Fix inconsistent behavior of `losses.sparse_categorical_crossentropy`… (keras-team#19838) * Fix inconsistent behavior of `losses.sparse_categorical_crossentropy` with and without `ignore_class` * Test * chore(format) * Fix tests in `losses` Fix bugs with `Mean`, `Accuracy` and `BinaryAccuracy` metrics. (keras-team#19847) - `reduce_to_samplewise_values` would not reduce `sample_weights` correctly because the number of dimensions of `values` was checked. - `reduce_to_samplewise_values` needs to explicitely broadcast `sample_weights`. Before, it was implicitly broadcast in the multiplication with `values`. However, the explicit broadcast is needed for the computation of `num_samples` for the averaging to be correct. This causes a bug when `sample_weights` is of rank 2 or more and a broadcast happens when doing the multiplication. This logic existed in `tf_keras`: https://github.com/keras-team/tf-keras/blob/master/tf_keras/metrics/base_metric.py#L508 - `Accuracy` and `BinaryAccuracy` were doing a mean reduction too early, before multiplying by `sample_weights`. This matters when the rank of `sample_weights` is the same as `y_true` and `y_pred`. Add tests for `DTypePolicyMap` Fix test Update the logic of `default_policy` Improve serialization of `DTypePolicyMap` Improve `__repr__` and `__eq__` Add `custom_gradient` for the numpy backend (keras-team#19849) fix variable name when add in init function (keras-team#19853) Address comments

Introduce `DTypePolicyMap` Fix `LayerNormalization.get_config` (keras-team#19807) Propagate kwargs through `keras.ops.isclose` (keras-team#19782) * propagate kwargs through isclose this allows passing atol and rtol * switch isclose **kwargs to explicit kwargs * reduce line lengths * fix ops.isclose signature * fix ops.IsClose compute_output_spec signature * implement isclose rtol atol equal_nan args for all backends * shorten line lengths again * revert using tf.experimental.numpy.isclose tensorflow version now uses code inspired from tf.experimental.numpy.isclose * fix lint * add docs for new parameters Faster in_top_k implementation for Jax backend (keras-team#19814) * Faster in_top_k implementation. * Fix bug in rank computation. Fix CI Fix TypeError in `Lambda.from_config` (keras-team#19827) fixing dmtree.is_nested() and parameterized tree test (keras-team#19822) Fix `keras.ops.repeat` cannot return an expected shape when `x` is a … (keras-team#19826) * Fix `keras.ops.repeat` cannot return an expected shape when `x` is a `KerasTensor` and the `axis` is `None` * Test dynamic is still dynamic after repetition * Improve error messages `Metric.variables` is now recursive. (keras-team#19830) This allows it to surface variables from metrics nested at any depth. Previously, metrics within metrics within metrics would not have their variables tracked in JAX, causing them to not be updated. Fix `get_file` when the HTTP response has no `Content-Length` header (keras-team#19833) Add `ops.switch` (keras-team#19834) * Add `ops.switch` * Update tests * Fix out-of-bound issue * Revert `torch.cond` Use `absl.testing.parameterized` for `tree_test.py`. (keras-team#19842) For consistency, use `absl.testing.parameterized` instead of `parameterized` for `tree_test.py` since that is used for all other tests. It's one less dependency. It also says `optree` or `dmtree` in each test name. Make batch norm mask shape error more descriptive (keras-team#19829) * Made batch norm mask shape error more descriptive * Added shape info in mask error message to help with degugging Fix code style doc: `ops.slice` (keras-team#19843) corrected the example code in unit_normalization.py (keras-team#19845) Added missing closing bracket and exact output value in example code after replicating the code. Adjust code example Add `training` argument to `Model.compute_loss()`. (keras-team#19840) This allows models to perform different computations during training and evaluation. For instance, some expensive to compute metrics can be skipped during training and only computed during evaluation. Note that backwards compatibility with overrides that do not have the `training` argument is maintained. Fix the compatibility issues of `Orthogonal` and `GRU` (keras-team#19844) * Add legacy `Orthogonal` class name * Add legacy `implementation` arg to `GRU` Fix inconsistent behavior of `losses.sparse_categorical_crossentropy`… (keras-team#19838) * Fix inconsistent behavior of `losses.sparse_categorical_crossentropy` with and without `ignore_class` * Test * chore(format) * Fix tests in `losses` Fix bugs with `Mean`, `Accuracy` and `BinaryAccuracy` metrics. (keras-team#19847) - `reduce_to_samplewise_values` would not reduce `sample_weights` correctly because the number of dimensions of `values` was checked. - `reduce_to_samplewise_values` needs to explicitely broadcast `sample_weights`. Before, it was implicitly broadcast in the multiplication with `values`. However, the explicit broadcast is needed for the computation of `num_samples` for the averaging to be correct. This causes a bug when `sample_weights` is of rank 2 or more and a broadcast happens when doing the multiplication. This logic existed in `tf_keras`: https://github.com/keras-team/tf-keras/blob/master/tf_keras/metrics/base_metric.py#L508 - `Accuracy` and `BinaryAccuracy` were doing a mean reduction too early, before multiplying by `sample_weights`. This matters when the rank of `sample_weights` is the same as `y_true` and `y_pred`. Add tests for `DTypePolicyMap` Fix test Update the logic of `default_policy` Improve serialization of `DTypePolicyMap` Improve `__repr__` and `__eq__` Add `custom_gradient` for the numpy backend (keras-team#19849) fix variable name when add in init function (keras-team#19853) Address comments Update docstrings

Made batch norm mask shape error more descriptive

54e2729

google-ml-butler bot added the size:XS label Jun 10, 2024

google-ml-butler bot assigned gbaned Jun 10, 2024

gbaned requested a review from qlzh727 June 11, 2024 03:24

google-ml-butler bot added the awaiting review label Jun 11, 2024

gbaned added stat:awaiting response from contributor and removed awaiting review labels Jun 11, 2024

google-ml-butler bot removed the stat:awaiting response from contributor label Jun 11, 2024

fchollet reviewed Jun 11, 2024

View reviewed changes

qlzh727 removed their request for review June 11, 2024 15:59

Added shape info in mask error message to help with degugging

0502951

fchollet approved these changes Jun 11, 2024

View reviewed changes

google-ml-butler bot added kokoro:force-run ready to pull Ready to be merged into the codebase labels Jun 11, 2024

kokoro-team removed the kokoro:force-run label Jun 11, 2024

fchollet merged commit a8aac97 into keras-team:master Jun 11, 2024

google-ml-butler bot removed the ready to pull Ready to be merged into the codebase label Jun 11, 2024

drasmuss mentioned this pull request Jun 11, 2024

Error in masked BatchNormalization with > 3 dimensions #19818

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Make batch norm mask shape error more descriptive #19829

Make batch norm mask shape error more descriptive #19829

Uh oh!

pandyah5 commented Jun 10, 2024 •

edited

Loading

Uh oh!

google-cla bot commented Jun 10, 2024

Uh oh!

gbaned commented Jun 11, 2024

Uh oh!

pandyah5 commented Jun 11, 2024

Uh oh!

codecov-commenter commented Jun 11, 2024 •

edited

Loading

Uh oh!

fchollet Jun 11, 2024

Uh oh!

pandyah5 Jun 11, 2024

Uh oh!

fchollet left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Make batch norm mask shape error more descriptive #19829

Make batch norm mask shape error more descriptive #19829

Uh oh!

Conversation

pandyah5 commented Jun 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

google-cla bot commented Jun 10, 2024

Uh oh!

gbaned commented Jun 11, 2024

Uh oh!

pandyah5 commented Jun 11, 2024

Uh oh!

codecov-commenter commented Jun 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

fchollet Jun 11, 2024

Choose a reason for hiding this comment

Uh oh!

pandyah5 Jun 11, 2024

Choose a reason for hiding this comment

Uh oh!

fchollet left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

pandyah5 commented Jun 10, 2024 •

edited

Loading

codecov-commenter commented Jun 11, 2024 •

edited

Loading