Skip to content

Conversation

@hashamyounis9
Copy link
Contributor

@hashamyounis9 hashamyounis9 commented Apr 10, 2025

/claim #2743

Product Description

In commcare-android, the image displaying mechanism for captured images was not respecting the orientation of the image.

Technical Summary

This ensures that resized images display with the correct orientation, consistent with how they were captured by the device.

Note

I checked that all the Exif properties we retained in #3014 are getting retained correctly and the underlying encryption also gets applied correctly, I also tested the images uploaded to server for further confirmation.

Feature Flag

Not behind any feature flag.

Safety Assurance

Safety story

  • The change was tested locally using images captured in different orientations (portrait, landscape, and upside down).
  • Verified that all resized images now display with the correct orientation after the fix.
  • The rotation is only applied when required, avoiding unnecessary overhead.
  • The code includes error handling for cases where EXIF data may be missing or unreadable.
  • This change only affects the image resizing utility and does not modify existing stored images or data.

Automated test coverage

QA Plan

  • Manual testing has been performed with a variety of image orientations.
  • QA can verify by capturing photos in various orientations and ensuring resized versions display upright.

Labels and Review

  • Do we need to enhance the manual QA test coverage? ✅ Yes, the "QA Note" label is set correctly.
  • Does the PR introduce any major changes worth communicating ? If yes, the "Release Note" label is set and a "Release Note" is specified in PR description.
  • Risk label is set correctly
  • The set of people pinged as reviewers is appropriate for the level of risk of the change

@damagatchi
Copy link

Can one of the admins verify this patch?

@coderabbitai
Copy link

coderabbitai bot commented Apr 10, 2025

📝 Walkthrough

Walkthrough

The changes update the image scaling functionality in MediaUtil.java to handle EXIF orientation properly. After attempting to inflate the bitmap, the code now performs a null check and returns null if loading fails. The updated logic retrieves the image's EXIF orientation using ExifInterface and logs an error if this operation fails. A new helper method, rotateBitmap, has been added to rotate the bitmap based on the obtained orientation. If a rotation is applied, the original bitmap is recycled to manage memory effectively. Finally, the method returns the rotated bitmap instead of the original, ensuring that the final scaled output respects the correct image orientation.

Sequence Diagram(s)

sequenceDiagram
    participant Caller
    participant MediaUtil
    participant ExifInterface
    participant Bitmap

    Caller->>MediaUtil: scaleDownImage(imagePath)
    MediaUtil->>Bitmap: loadBitmap(imagePath)
    alt Bitmap is null
        MediaUtil->>Caller: return null
    else
        MediaUtil->>ExifInterface: read EXIF orientation(imagePath)
        ExifInterface-->>MediaUtil: return orientation
        MediaUtil->>MediaUtil: rotateBitmap(bitmap, orientation)
        alt Bitmap is rotated
            MediaUtil->>Bitmap: recycle original bitmap
        end
        MediaUtil->>Caller: return rotated bitmap
    end
Loading

Possibly related issues

Possibly related PRs

Suggested labels

skip-integration-tests, 🙋 Bounty claim

Suggested reviewers

  • shubham1g5
✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai plan to trigger planning for file edits and PR creation.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
app/src/org/commcare/utils/MediaUtil.java (2)

379-387: Appropriate EXIF metadata handling with error logging.

The code correctly retrieves the orientation information with a proper default value and includes error handling for cases where EXIF data cannot be read.

Consider using the class constant TAG instead of hardcoded "ImageScaling" for consistent logging:

-    Log.e("ImageScaling", "Unable to read EXIF data: " + e.getMessage());
+    Log.e(TAG, "Unable to read EXIF data: " + e.getMessage());

410-446: Well-implemented bitmap rotation helper method.

The rotateBitmap method handles all standard EXIF orientation values correctly with appropriate transformations for each case. The implementation is efficient with early returns for null bitmaps and normal orientation.

Consider adding OutOfMemoryError handling similar to other bitmap operations in this class:

 private static Bitmap rotateBitmap(Bitmap bitmap, int orientation) {
     if (bitmap == null || orientation == ExifInterface.ORIENTATION_NORMAL) {
         return bitmap;
     }

     Matrix matrix = new Matrix();
     switch (orientation) {
         case ExifInterface.ORIENTATION_ROTATE_90:
             matrix.postRotate(90);
             break;
         // cases omitted for brevity
         default:
             return bitmap;
     }

+    try {
         return Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(), bitmap.getHeight(), matrix, true);
+    } catch (OutOfMemoryError e) {
+        Log.d(TAG, "Ran out of memory attempting to rotate bitmap");
+        return bitmap;
+    }
 }
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 07b4543 and 15287e3.

📒 Files selected for processing (1)
  • app/src/org/commcare/utils/MediaUtil.java (2 hunks)
🔇 Additional comments (4)
app/src/org/commcare/utils/MediaUtil.java (4)

7-9: Good addition of necessary imports.

The addition of Matrix and ExifInterface are required for the new image rotation functionality being introduced.


375-377: Good defensive programming with null check.

This null check prevents potential NullPointerExceptions when bitmap loading fails, which improves error handling.


388-392: Good memory management when rotating bitmaps.

The code properly recycles the original bitmap if a new bitmap was created during rotation, which prevents memory leaks.


394-407: Correctly returning rotated bitmap.

The return statements have been appropriately updated to use the rotated bitmap instead of the original bitmap.

@shubham1g5
Copy link
Contributor

@damagatchi ok to test

@shubham1g5
Copy link
Contributor

@hashamyounis9 Would you be able to complete our CLA agreement if you have not already.

Copy link
Contributor

@shubham1g5 shubham1g5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me, just made one minor request reg. exception handling.

ExifInterface exif = new ExifInterface(imageFilepath);
orientation = exif.getAttributeInt(ExifInterface.TAG_ORIENTATION, ExifInterface.ORIENTATION_NORMAL);
} catch (IOException e) {
Log.e(TAG, "Unable to read EXIF data: " + e.getMessage());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we log this using Logger.exception instead.

try {
return Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(), bitmap.getHeight(), matrix, true);
} catch (OutOfMemoryError e) {
Log.d(TAG, "Ran out of memory attempting to rotate bitmap");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be great if we can use Logger.exception here and in the other OutOfMemoryError catch abvove in the calling method.

@shubham1g5
Copy link
Contributor

@hashamyounis9 Seems like we have failing unit tests -

10:46:35 org.commcare.android.tests.ImageInflationTest > testScaleDownDueToBoth_densityDominant FAILED
10:46:35     java.lang.NullPointerException at ImageInflationTest.java:47
10:46:36 
10:46:36 org.commcare.android.tests.ImageInflationTest > testDoNoScaling FAILED
10:46:36     java.lang.NullPointerException at ImageInflationTest.java:47
10:46:36 
10:46:36 org.commcare.android.tests.ImageInflationTest > testScaleDownDueToContainer_noDensityEffect FAILED
10:46:36     java.lang.NullPointerException at ImageInflationTest.java:47
10:46:36 
10:46:36 org.commcare.android.tests.ImageInflationTest > testScaleDownDueToBoth_containerDominant FAILED
10:46:36     java.lang.NullPointerException at ImageInflationTest.java:47
10:46:36 
10:46:36 org.commcare.android.tests.ImageInflationTest > testScaleDownDueToDensity FAILED
10:46:36     java.lang.NullPointerException at ImageInflationTest.java:47
10:46:36 
10:46:36 org.commcare.android.tests.ImageInflationTest > testInflationWithoutDensity_shouldChange FAILED
10:46:36     java.lang.NullPointerException at ImageInflationTest.java:38
10:46:36 
10:46:36 org.commcare.android.tests.ImageInflationTest > testInflationWithoutDensity_noChange FAILED
10:46:36     java.lang.NullPointerException at ImageInflationTest.java:38

@hashamyounis9
Copy link
Contributor Author

hashamyounis9 commented Apr 11, 2025

@hashamyounis9 Would you be able to complete our CLA agreement if you have not already.

do I have to sign the CLA separately for each contribution or just once? coz I did sign the CLA during my last
contribution..

@shubham1g5
Copy link
Contributor

@hashamyounis9 Just once is fine, so if you have done it already, no need to do it again.

@hashamyounis9
Copy link
Contributor Author

@shubham1g5

Is the build fail caused by my commit?

can you provide me with the error we are getting?

@shubham1g5
Copy link
Contributor

@hashamyounis9
Copy link
Contributor Author

@shubham1g5

are we still having the same unit tests to fail?

@shubham1g5
Copy link
Contributor

@hashamyounis9 yes, are you not able to replicate failure locally ?

@hashamyounis9
Copy link
Contributor Author

hashamyounis9 commented Apr 14, 2025

@hashamyounis9 yes, are you not able to replicate failure locally ?

nope, always works for me

@shubham1g5
Copy link
Contributor

I am not sure why they are passing for you and failing on Jenkins, but here is a more detailed error from Jenkins if it helps -

testDoNoScaling
java.lang.NullPointerException: Cannot invoke "String.indexOf(int)" because "attrStr" is null
	at android.media.ExifInterface.loadAttributes(ExifInterface.java:211)
	at android.media.ExifInterface.__constructor__(ExifInterface.java:124)
	at android.media.ExifInterface.<init>(ExifInterface.java)
	at org.commcare.utils.MediaUtil.scaleDownToTargetOrContainer(MediaUtil.java:377)
	at org.commcare.utils.MediaUtil.getBitmapScaledForNativeDensity(MediaUtil.java:168)
	at org.commcare.android.tests.ImageInflationTest.testCorrectInflationWithDensity(ImageInflationTest.java:47)

@hashamyounis9
Copy link
Contributor Author

I am not sure why they are passing for you and failing on Jenkins, but here is a more detailed error from Jenkins if it helps -

testDoNoScaling
java.lang.NullPointerException: Cannot invoke "String.indexOf(int)" because "attrStr" is null
	at android.media.ExifInterface.loadAttributes(ExifInterface.java:211)
	at android.media.ExifInterface.__constructor__(ExifInterface.java:124)
	at android.media.ExifInterface.<init>(ExifInterface.java)
	at org.commcare.utils.MediaUtil.scaleDownToTargetOrContainer(MediaUtil.java:377)
	at org.commcare.utils.MediaUtil.getBitmapScaledForNativeDensity(MediaUtil.java:168)
	at org.commcare.android.tests.ImageInflationTest.testCorrectInflationWithDensity(ImageInflationTest.java:47)

@shubham1g5

so I guess it does help as I was able to find out that image that we use for testing which lives inside directory app/unit-tests/resources/images named as 100x100.png does not contain the EXIF properties as I check the image using this tool, so when the EXIF properties are being read in the line org.commcare.utils.MediaUtil.scaleDownToTargetOrContainer(MediaUtil.java:377) it causes an NPE, I think we need to update the image being used for image inflation test to have the EXIF properties

@shubham1g5
Copy link
Contributor

@hashamyounis9 Why would not the test fail for you if that was the error, also even after the change, the error in test failure remains to be present on Jenkins.

@hashamyounis9
Copy link
Contributor Author

@shubham1g5

So as we discussed here that PNGs do not contain EXIF properties, their bitmap itself gets stored rotated rather than calculating rotation at view-time like JPGs do, but my code was trying to read exif irrespective of the image extension, now it avoids the rotation logic if image is a PNG resulting in passing the unit tests!!

Let me know if any other changes required...

Bitmap b = inflateImageSafe(imageFilepath, approximateScaleDownFactor).first;

int orientation = ExifInterface.ORIENTATION_NORMAL;
if (!imageFilepath.endsWith("png")) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we can practically maintain a list of what image formats have Exif data vs not and think our strategy here should be to try and get exif data safely for all file formats, but if that's not available simply don't do any exif manipulations and have the core functionality to display the image work without any crashes.

@hashamyounis9
Copy link
Contributor Author

hashamyounis9 commented Apr 21, 2025

@shubham1g5

5562446 tries to read EXIF properties from the image irrespective of image extension and if it is able to do that only then performs rotation on the bitmap to view it properly in the app. Otherwise it avoids rotating Bitmap and displays the bitmap as it is from the disk.

Let me know if any changes required

Copy link
Contributor

@shubham1g5 shubham1g5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some minor comments, but looks good as a whole.

Comment on lines 378 to 382
} catch (IOException e) {
Logger.exception("Unable to read image file from disk: ", e);
} catch (Exception e) {
Logger.exception("Unable to read exif data from image file: ", e);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can remove the catch (IOException e) { from here and just have a generic catch as we are not handling those 2 very differently.

return Bitmap.createScaledBitmap(rotatedBitmap, newWidth, newHeight, false);
} catch (OutOfMemoryError e) {
Log.d(TAG, "Ran out of memory attempting to scale image at: " + imageFilepath);
Logger.exception("Ran out of memory attempting to scale image at: ", e);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing imageFilepath from message.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like we don't need it anymore.

@hashamyounis9
Copy link
Contributor Author

@shubham1g5

made the changes as asked!

Copy link
Contributor

@shubham1g5 shubham1g5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. looks good, waiting for build to complete to merge.

@shubham1g5 shubham1g5 merged commit 7c25d05 into dimagi:master Apr 21, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants