Skip to content

Conversation

@utkarsh14
Copy link
Collaborator

@utkarsh14 utkarsh14 commented Jul 16, 2025

Locations : Set error messages similar to targets/enumerators end point

Ticket

Fixes: UX fixes

Description, Motivation and Context

Changed locations endpoint to use record_errors and summary_by_error_type in error messages, this links to frontend changes where we are adding an error warning table for locations.

How Has This Been Tested?

Local

Checklist:

This checklist is a useful reminder of small things that can easily be forgotten.
Put an x in all the items that apply and remove any items that are not relevant to this PR.

  • My code follows the style guidelines and standard practices for this project
  • I have reviewed my own code to ensure good quality
  • I have tested the functionality of my code to ensure it works as intended
  • I have resolved merge conflicts
  • I have written good commit messages
  • I have created migration scripts from the latest dev changes. This will prevent migration script version conflicts (if applicable)
  • I have scrutinized and edited the migration script with reference to changes Alembic won't auto-detect (if applicable). Note that this includes manually adding the creation of new CHECK constraints.
  • I have updated the automated tests (if applicable)
  • I have updated the README file (if applicable)
  • I have updated the OpenAPI documentation (if applicable)

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors how nullability and bulk-editability are represented in the targets module and standardizes error reporting in the locations endpoints by adopting a record_errors structure with summary_by_error_type.

  • Replaces the bulk_editable column/flag with allow_null_values across migrations, model, controllers, tests, and API docs.
  • Updates location file validations and error handlers to emit a record_errors object containing a summary_by_error_type list of detailed error entries.
  • Adjusts unit tests and OpenAPI documentation to align with the new error response and configuration schema.

Reviewed Changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tests/unit/test_targets.py Updated tests to assert allow_null_values instead of bulk_editable.
tests/unit/test_locations.py Modified tests to verify the new record_errors.summary_by_error_type structure.
migrations/versions/b0c4574bd52e_.py Added allow_null_values column and removed bulk_editable in migration.
app/blueprints/targets/models.py Replaced bulk_editable column with allow_null_values.
app/blueprints/targets/controllers.py Swapped references to bulk_editable for allow_null_values; adjusted bulk-update logic.
app/blueprints/locations/utils.py Refactored validation to accumulate record_errors.summary_by_error_type entries.
app/blueprints/locations/controllers.py Transitioned error handlers to produce the new record_errors format.
app/blueprints/docs/surveystream.yml Updated OpenAPI spec to use allow_null_values and reflect new response schema.
Comments suppressed due to low confidence (1)

app/blueprints/targets/controllers.py:1013

  • Removing the check on bulk_editable means all columns except target_id are now treated as bulk-editable. Verify that this aligns with intended behavior or reintroduce a dedicated flag to control which fields can be bulk updated.
        if column.column_name != "target_id":

Comment on lines 493 to 494
]
for error_message in e.geo_level_hierarchy_errors
Copy link

Copilot AI Jul 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comprehension is applied to the outer dict rather than inside the summary_by_error_type list. It should be a list comprehension within the list literal to avoid overwriting the same key on each iteration.

Suggested change
]
for error_message in e.geo_level_hierarchy_errors
for error_message in e.geo_level_hierarchy_errors
],

Copilot uses AI. Check for mistakes.
with op.batch_alter_table("target_column_config", schema="webapp") as batch_op:
batch_op.add_column(
sa.Column(
"allow_null_values", sa.Boolean(), server_default="true", nullable=False
Copy link

Copilot AI Jul 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Consider using server_default=sa.text('true') instead of a raw string to ensure the default boolean is emitted correctly in the SQL DDL.

Suggested change
"allow_null_values", sa.Boolean(), server_default="true", nullable=False
"allow_null_values", sa.Boolean(), server_default=sa.text('true'), nullable=False

Copilot uses AI. Check for mistakes.
Comment on lines +723 to +725
except Exception as e:
db.session.rollback()
return jsonify(message=str(e)), 500
Copy link

Copilot AI Jul 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Catching a bare Exception can mask unexpected issues and make debugging harder. Consider catching more specific exceptions or logging before rethrowing.

Suggested change
except Exception as e:
db.session.rollback()
return jsonify(message=str(e)), 500
except ValueError as e: # Example of a specific exception
db.session.rollback()
return jsonify(message=f"Value error occurred: {str(e)}"), 400
except Exception as e:
db.session.rollback()
# Log the unexpected exception for debugging purposes
app.logger.error(f"Unexpected error: {str(e)}", exc_info=True)
raise

Copilot uses AI. Check for mistakes.
@jeenut27 jeenut27 self-requested a review July 17, 2025 11:54
Copy link
Collaborator

@jeenut27 jeenut27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left one comment. Rest looks good! Thanks

expected_response = {
"success": False,
"record_errors": {
"summary_by_error_type": [
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@utkarsh14 - if you compare the error message here vs that of targets you will see targets has extra invalid_records and summary keys. I believe invalid_records is needed for the download output which was discussed. And summary is used for summary stats on UI. ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@utkarsh14 utkarsh14 requested a review from jeenut27 July 23, 2025 04:36
Copy link
Collaborator

@jeenut27 jeenut27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good. Thanks!

@utkarsh14 utkarsh14 merged commit 3325971 into dev Jul 23, 2025
5 checks passed
@utkarsh14 utkarsh14 deleted the validation_fixes branch July 25, 2025 08:14
@sentry
Copy link

sentry bot commented Jul 26, 2025

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

Did you find this useful? React with a 👍 or 👎

utkarsh14 added a commit that referenced this pull request Aug 1, 2025
* HOTFIX: Fix choice name linking with question name (#290)

* upgrading gevent version

* Add debugging statement for notifications test

* Locations : Set error messages similar to targets/enumerators end point (#321)

* Locations : Set error messages similar to targets/enumerators end point

* Add allow_null_values to target column config

* Add allow null values column in Enumerators column config

* Add summary and invalid records to location upload errors

* Replace null values by blanks (#322)

---------

Co-authored-by: Griffin Muteti <griffinmuteti31@gmail.com>
Co-authored-by: Jayprakash <0freerunning@gmail.com>
Co-authored-by: Jeenu Thomas <jeenu.thomas@idinsight.org>
Co-authored-by: jeenut27 <jeenuthomas27@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants