Alphagov synch May 2026#48
Draft
rodrigoAbril wants to merge 685 commits into
Draft
Conversation
…perations - dao_fetch_live_services_data added bulk session and retry attempts
…_data_by_service operations - fetch_monthly_notification_statuses_per_service added bulk session and retry attempts
…s-108 notifications-utils: 107.0.2 -> 108.0.0
This is the second of the two queries that the `check-for-services-with-high-failure-rates-or-sending-to-tv-numbers` nightly task runs. It doesn't need to have the latest data.
…te-call-to-use-kwargs update call to validate_template to use keyword arguments instead of positional
78 is the max that Document Download API will allow. Before https://github.com/alphagov/notifications-admin/pull/5795/files we were creating entries with a `retention_period` of 90. This fixes those so we don’t cause 500s if someone tries to send an email with them.
…n-functions-replica refactor: platform admin endpoints to use read replica
This is the result of deleting our `requirements*.txt` files, running `make freeze-requirements` and removing anything that might need more testing or where we might want to check the changelog in more detail.
Fix template email files with too-long retention
…k-replica Move "check_for_services_with_high_failure_rates_or_sending_to_tv_numbers" task to read replica
…rsion into a `DISTINCT ON` statement
The "check-for-services-with-high-failure-rates-or-sending-to-tv-numbers" task had logic which meant that it checks for services with a high percentage of permanent failures OR services that have sent more than 500 texts to TV numbers. If service(s) with high failure rates are found, the check for sending to TV numbers didn't run. This might have been deliberate when creating the task - services which are sending to TV numbers are also likely to have high permanent failure rates. However, it's possible we were sometimes missing services which were sending to TV numbers because the code in the "elif" wasn't reached. This changes the check to check for both high failure rates and services sending to TV numbers. Services might now appear in both categories, but this alert very rarely triggers and having a service appear in both should not be an issue.
…et-template-email-file re-write query for getting template_email_files by template_id and version into a `DISTINCT ON` statement
…already been archived using the `email_files` attribute can fail when trying to only get files that haven't been archived. The Template model object points to the file objects in the `template_email_files` table with a given id, regardless of whether they are later archived etc. or not. We should instead run a fresh query after archiving the template so that we only archive files that aren't already archived.
…or-archived-template-bug fixed update template logic so that we do not return files that have already been archived
…tes-task Adjust the logic of the task to check for high failure rates
Bump most sub-dependencies
We have been asked to include the following headers X-Content-Type-Options: nosniff Nosniff stops the browser trying to guess the type of a downloaded file, which can prevent XSS attacks. Strict-Transport-Security: max-age=31536000; includeSubDomains HSTS instructs the web browser that the server will only communicate https. This prevents downgrade attacks that might leak data X-Frame-Options: DENY / X-Frame-Options: SAMEORIGIN Stops the application being loaded from a different domain, so there is no chance of accidental click jacking. X-Permitted-Cross-Domain-Policies: none This instructs adobe clients not to load any policies that allow improper cross domain access. Cache-control: Cache-control: no-store This makes sure that requests with potentially sensitive data are not cached. X-Xss-Protection: 1; mode=block To be removed as it is a legacy command that can cause issues. See : https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/X-XSS-Protection#security_considerations
…atabase queries and implement retryable_query decorator
…port-generation DB Replica - CSV Report Generation
ITHC fix headers
…it-fair-queue refactor: pin Kombu to commit containing fair queue changes
Bcrypt 5.0.0 raises an error (rather than silently truncating) when given a long password: ``` ValueError: password cannot be longer than 72 bytes, truncate manually if necessary (e.g. my_password[:72]) ``` This commit follows the advice to manually truncate first. We need to do this (rather than move to a new hashing algorithm) to preserve compatibility with the hashed passwords we already have stored in our database.
We have addressed the breaking change in previous commits
We want to let these upgrade naturally whenever anyone re-freezes the requirements.
…pe use the read replica It does this by making fetch_service_data_retention_for_all_services_by_notification_type take a session. And then passing in the read replica session Added a test to make sure this works too
…a-ui-permission Remove `send_files_via_ui` service permission
…ltimate-admin-user Update the logic that determines when a user is allowed to be removed from a service or archived
We want to check if changing a user's permissions for a service would result in the service having too few users with the `manage_settings` permission, so this adds in a function that can be used to check. The minimum number of users with `manage_settings` is 2 users for live services and trial services which have requested to go live, and 1 for trial services without a go-live request.
This uses the new `users_permissions_can_be_changed` function to check if changing a user's permissions would result in too few users for the service having the `manage_settings` permission. If so, we raise an error that the admin app can catch and then show a suitable message.
We need a newer version of uv to get the `--exclude-newer` flag
## 115.0.1 * Don’t set `require-hashes` to `true` when running `uv pip install` etc. * Generate hashes by default when running `uv pip compile` ## 115.0.0 * Adds shared `uv.toml` with `require-hashes = true`. Run `make freeze-requirements` to re-generate `requirements.txt` with hashes ## 114.1.0 * `RecipientCSV`: add some micro-optimizations for functions called from inner loops, use `__slots__` for `Cell`. The latter of these is technically a breaking change, but it's extremely unlikely consumers are mutating arbitrary attributes on `Cell` objects. ## 114.0.0 * **Breaking change**: Make `SerialisedModelCollection` a proper `Iterable`. To achieve this, model instance construction from "item" dicts has been moved to a new dedicated method `_get_model_instance_from_item`. Subclasses that previously customised model instance construction by overriding `__getitem__` will need to be updated to move this to `_get_model_instance_from_item` (and as a result may be able to remove their custom `__getitem__` implementation. ## 113.6.2 * Configure gunicorn with `control_socket_disable` as `True` by default *** Complete changes: alphagov/notifications-utils@113.6.1...115.0.1
Bump utils to 115.0.1
…" partial index `delivered` notifications make up over 95% of the `notifications` table. when we need to find notifications of *any* of the other statuses, this index will allow the query planner to scan in in created_at order, skipping the vast majority of the rows, provided we're also filtering by `service_id` and `notification_type`. judging from the current contents of the `notifications` table, this should barely add 20% to the size of the index over the existing "failed statuses" partial index. the original motivation for this is speeding up `get_notifications` queries for the `sent` status.
…elivered-index `Notification` model: replace "failed" partial index with "nondelivered" partial index
…s-apt-key-keyserver-alternatives Dockerfile: try multiple keyservers when attempting to fetch postgres apt repo key
…ermissions Raise error if changing a user's permissions would result in a service have too few users with `manage_settings`
for send_email api requests When a request to send an email has sanitise_content_for argument, we check the content of specified placeholders for potentially malicious content. During the user research, service users requested to be notified when this is activated, so that they could investigate. This PR sends the information about the content we changed, and how it looks before and after. The information is included in the response. "sanitised_content" key is included in the response for all send_email requests (so it's less error-prone for our service users), but t is not included for send_sms and send_letter requests, as this is an email-nly feature (because it's a response to the Markdown injection vulnerability, which is mainly an issue for email templates).
…anitise-their-content Include sanitised content info in send_email V2 endpoint api response
…ce-when-we-sanitise-their-content Revert "Include sanitised content info in send_email V2 endpoint api response"
for send_email api requests When a request to send an email has sanitise_content_for argument, we check the content of specified placeholders for potentially malicious content. During the user research, service users requested to be notified when this is activated, so that they could investigate. This PR sends the information about the content we changed, and how it looks before and after. The information is included in the response. "sanitised_content" key is included in the response for all send_email requests (so it's less error-prone for our service users), but t is not included for send_sms and send_letter requests, as this is an email-nly feature (because it's a response to the Markdown injection vulnerability, which is mainly an issue for email templates).
It’s better if the date on a templated letter is the date when the user
pressed _Send_ rather than when it was processed by us.
This is because:
- it will reflect what they see in the preview at the time of pressing
send
- it won’t change if we have to recreate the PDFs for any reason
In order to do this the API needs to tell template preview when the
letter was created.
***
Depends on:
- [ ] https://github.com/alphagov/notifications-template-preview/pull/973/files
…etter Send `created_at` date when creating letter for print
Changes the instrumentation of `notification.deliver.duration` so that it does not fall back to the current time if `delivery_dt` is unset, and adds a new metric `notification.callback.duration` based on the timestamp at the moment we received the delivery receipt. This makes the delivery receipt metric more honest while giving us a backup metric if we decide we don't trust the delivery timestamps as reported by a provider.
More OTel metrics
…when-we-sanitise-their-content Redeploy: Include sanitised content info in response for send_email api requests
this should avoid us fetching the cartesian product of the number of organisations the user's in, the number of services those organisations have, and the number of services the user's directly in - which can get quite big (50k+) for some users. expecting this to make the worst case scenario for this view significantly faster.
…-selectinload `get_organisations_and_services_for_user`: use `selectinload` loading
rodrigoAbril
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
14 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Alphagov synch May 2026