Warehouse: l10n skeleton

[woodruffw]

TODO:

• Changes to the .mo files should cause the web container/service to restart in development mode
• Accept-Language is not working, for unclear reasons
• Figure out how to defer request so that we can use _() in form fields

Closes #6455

[woodruffw]

To follow up on the "deferred request" issue:

This is our current localization method:

def localize(message, **kwargs):
    request = get_current_request()
    return request.localizer.translate(_translation_factory(message, **kwargs))

which we expose as the traditional _()-style identifier for string tagging:

from warehouse.i18n import localize as _

# ...

request.session.flash(
    _(
        "Email address ${email_address} verified. ${confirm_message}.",
        mapping={"email_address": email.email, "confirm_message": confirm_message},
    ),
    queue="success",
)

This works well for the bodies of views and forms, since there's an active request pending that get_current_request can yank from the threadlocal state.

Unfortunately, this doesn't work in form field declarations, like so:

class TOTPValueMixin:

    totp_value = wtforms.StringField(
        validators=[
            wtforms.validators.DataRequired(),
            wtforms.validators.Regexp(
                rf"^[0-9]{{{TOTP_LENGTH}}}$",
                message=_(f"TOTP code must be {TOTP_LENGTH} digits."),
            ),
        ]
    )

...since _() is called before a request is active.

[mmerickel]

Yeah this is the exact issue I was expecting to see - makes sense.

The workflow with the translationstring package (you probably know this, but just to frame it):

1. Tag every translatable string as a TranslationString.

2. Eventually (you determine when) run those strings through the localizer. Usually it's clear when you should do this because it's the same time that you need to interpolate the values into the string. It's not as clear when the string has no interpolation placeholders (like most strings in this pull request so far).

Your current approach combines both steps into 1 via the localize(foo, args) function. This approach falls apart when you need to mark strings as translatable at module scope (with no active request).

Ok cool, problem stated - my solution: I would suggest defining _ = TranslationStringFactory('messages') and using warehouse.i18n.localize only in functions (as it won't work at module scope). A purist could even remove the usage of _translation_factory from localize. This basically doesn't change anything you have done so far except renaming _ to localize and wrapping the arg to localize in _. For example localize(_('You did evil things.')).

For module-scope you need to find a good location to take a translationstring marked with _ and run it through the localizer. This is the biggest wart in Pyramid's approach to localization imo, but it kinda is what it is unless you define a magic version of translationstring that uses the localizer in __str__ and just assuming that you will never use __str__ outside of request scope.

For wtforms (I don't have much experience with this library so my suggestion is likely to be vague here) it probably means at the spot where you handle errors, you receive the error messages and you then run them through localize(error_msg). If the error messages need interpolated values then it needs to be done in the actual error functions (similar to what you are already doing).

I hope this helps and I'm happy to spitball on some other options.

[ewdurbin]

The only outstanding question I have is how we will handle setting the locale when rendering messages in email templates.

I don't know that it must be addressed immediately, but given that once we launch translation services the strings will be translatable we might want to consider that.

[woodruffw]

The only outstanding question I have is how we will handle setting the locale when rendering messages in email templates.

Hmm, do we have access to the request that prompted the email? If so, we could use our normal locale negotiation. Otherwise, we could probably add a locale field to the user model and set it as the highest priority in the negotiator, when available.

[ewdurbin]

Resolution to email question: Email templates are rendered in the request before being enqueued, so the Locale of the request determines the locale of the template rendering!