Fix DataError Due to String Length Exceeding Column Constraint in AssetAuditLog

[BelhsanHmida]

Description

This PR introduces changes to prevent string length exceeding column size in the AssetAuditLog by implementing updating audit_log for each asset or attribute change individually and truncation for long strings (exceeding 255).

• Added the truncate_string function to handle string truncation, ensuring compliance with the 255-character limit on the event field in the AssetAuditLog table by adding ... in middle of string.

• Updated the AssetAuditLog class to ensure each asset update or feature change is logged as a separate entry. This prevents exceeding the column size limit by breaking up large events into individual records.

Look & Feel

changing an asset attribute From:

{  "capacity_in_mw": 0.5, 
   "min_soc_in_mwh": 0.05, 
   "max_soc_in_mwh": 0.45, 
   "sensors_to_show": [{"title": "Test Title 1", "sensors": [1]}, {"title": "Test Title 2", "sensors": [3,2]}, {"title": "Test Title 3", "sensors": [2,5]}, {"title": "Test Title 4", "sensors": [1]} ,{"title": "Test Title 5", "sensors": [1]}]}

To:

{  "capacity_in_mw": 0.05, 
   "min_soc_in_mwh": 0.005, 
   "max_soc_in_mwh": 0.045, 
   "sensors_to_show": [{"title": "New Test Title 1", "sensors": [1]}, {"title": "Test Title 2", "sensors": [3,2]}, {"title": "Test Title 3", "sensors": [2,5]}, {"title": "Test Title 4", "sensors": [1]} ,{"title": "Test Title 5", "sensors": [1]}]}

would result in 5 audit_log instead of 1 long one that would exceed column length:

• Updated asset 'toy-battery': 3 fields: Attr: capacity_in_mw, From: 0.5, To: 0.05
• Updated asset 'toy-battery': 3 fields: Attr: max_soc_in_mwh, From: 0.45, To: 0.045
• Updated asset 'toy-battery': 3 fields: Attr: min_soc_in_mwh, From: 0.05, To: 5
• Updated asset 'toy-battery-1': 3 fields: Attr: sensors_to_show, From: [{'title': 'Test Title 1', 'sensors': [1]}, {'title': ' ... e': 'Test Title 3', 'sensors': [2, 5]}, {'title': 'Test Title 4', 'sensors': [1]}, {'title': 'Test Title 5', 'sensors': [1]}] (This event was truncated by adding ' ... ' in middle because event was over 255 limit)

How to test

1. Reproduce the Issue:

   • Modify the sensors_to_show field with JSON data such as:

     {
       "sensors_to_show": [[129, 125, 119], [129, 125, 119], [129, 125, 119], [129, 125, 119], [129, 125, 119], [129, 125, 119], [129, 125, 119]]
     }

   • Alternatively, use a long title in sensors_to_show to trigger the error.

2. Verify Fix:

   • Test adding or updating asset pages with lengthy data or titles and confirm that no DataError occurs.
   • Ensure that each update is logged in the audit log properly and are truncated appropriately and do not exceed the column constraints.

Further Improvements

• Review and potentially adjust truncation logic if further edge cases are identified.
• Increase audit_log column size to minimize truncation and information loss.

Related Items

closes #1136

---

• I agree to contribute to the project under Apache 2 License.
• To the best of my knowledge, the proposed patch is not based on code under GPL or other license that is incompatible with FlexMeasures.

[nhoening]

It tested well!
I only have documentation improvements, let's do this.

[nhoening]

This is adding lots of code that is hard to read, with the goal to make one message for potentially many changes at once. One more complexity problem can be seen in my one comment.

Idea: We do it simpler. Each updated field or attribute gets its own audit log, by AssetAuditLog.add_record_for_attribute_update() or AssetAuditLog.add_record().

We buy simplicity with sometimes more than one log entry. That will not happen too often I believe. And the asset audit log will be more readable.

Each message is probably short enough, but these two helper methods can still make sure by truncating if needed.

[nhoening]

Can you bring the PR description up to date?

[nhoening]

Thanks, it (still) works.
Can you quickly add a changelog entry and add one semi-colon for me?

[nhoening]

Thanks!