Fix: UpSampling2D bilinear set_image_data_format(channels_first) bug #21456
Conversation
…assing it in ops.image.resize, which has been defined as channels_first if keras.backend.set_image_data_format has been called on channelsfirst in user's code. This creates a bug, a passed in tensor [16, 3, 224, 224] will return as shape [16, 448, 224, 448] instead of [16, 3, 448, 448]. Setting the data_format as the expected channels_last fixes that issue.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #21456 +/- ##
==========================================
- Coverage 82.72% 82.68% -0.04%
==========================================
Files 565 565
Lines 55247 55304 +57
Branches 8613 8626 +13
==========================================
+ Hits 45702 45728 +26
- Misses 7433 7461 +28
- Partials 2112 2115 +3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| num_samples = 2 | ||
| stack_size = 2 | ||
| input_num_row = 11 | ||
| input_num_col = 12 | ||
|
|
||
| if use_set_image_data_format: | ||
| set_image_data_format(data_format) |
There was a problem hiding this comment.
You should not set this globally in a unit test, instead pass the argument when you create the UpSampling2D layer.
There was a problem hiding this comment.
Unfortunately, that's the only way the error happens, as I've stated here
#21401 (comment)
There are no issues if I simply do
keras.layers.UpSampling2D(size=(2, 2), interpolation="bilinear", data_format="channels_first")(x)
I'm okay with just removing that line from the test, but it won't be testing my specific issue. Let me know if that's our best solution.
|
/gemini review |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request fixes a bug in UpSampling2D with bilinear interpolation when image_data_format is set globally to channels_first. The fix correctly forces data_format='channels_last' when calling ops.image.resize, as the input is always transposed to this format beforehand. The changes to the unit tests are good, especially adding coverage for the global setting and fixing the interpolation argument. Ensure test isolation by restoring the global image_data_format after the test runs to prevent potential side effects on other tests.
| if use_set_image_data_format: | ||
| set_image_data_format(data_format) |
There was a problem hiding this comment.
Modifying the global state image_data_format can lead to flaky tests. To ensure test isolation, save the original image_data_format before changing it and restore it after the test execution using a try...finally block.
original_data_format = backend.image_data_format()
if use_set_image_data_format:
set_image_data_format(data_format)
try:
if data_format == "channels_first":
inputs = np.random.rand(
num_samples, stack_size, input_num_row, input_num_col
)
else:
inputs = np.random.rand(
num_samples, input_num_row, input_num_col, stack_size
)
self.run_layer_test(
layers.UpSampling2D,
init_kwargs={
"size": (2, 2),
"data_format": data_format,
"interpolation": "bilinear",
},
input_shape=inputs.shape,
)
layer = layers.UpSampling2D(
size=(length_row, length_col),
data_format=data_format,
interpolation="bilinear",
)
layer.build(inputs.shape)
np_output = layer(inputs=backend.Variable(inputs))
if data_format == "channels_first":
self.assertEqual(np_output.shape[2], length_row * input_num_row)
self.assertEqual(np_output.shape[3], length_col * input_num_col)
else:
self.assertEqual(np_output.shape[1], length_row * input_num_row)
self.assertEqual(np_output.shape[2], length_col * input_num_col)
finally:
if use_set_image_data_format:
set_image_data_format(original_data_format)
The current approach transforms the tensor to channels_last, before passing it in ops.image.resize, which has been defined as channels_first if keras.backend.set_image_data_format has been called on channelsfirst in the user's code. This creates a bug, a passed in tensor [16, 3, 224, 224] will return as shape [16, 448, 224, 448] instead of [16, 3, 448, 448]. Setting the data_format as the expected channels_last fixes that issue.
I changed the unit tests to integrate keras.backend.set_image_data_format, also I noticed
layer = layers.UpSampling2D(
size=(length_row, length_col),
data_format=data_format,
)
Inside test_upsampling_2d_bilinear, wasn't using interpolation="bilinear", so I corrected that too
Closes #21401.