Broaden type annotation for verbose_name(_plural) to accept lazystr.

[PIG208]

I have made things!

There might be other use cases of lazystr that need to be considered. It needs to be verified that Promise objects are actually expected for the callee. Right now we are following a conservative approach to first mark ones that do commonly accept Promise.

Related issues

• Closes Error while passing gettext_lazy to arguments expecting strings #1137
• Refs Error while passing gettext_lazy to arguments expecting strings #1137

[sobolevn]

Is there any way to make str and Promise compatible?

[PIG208]

str and Promise are not compatible in terms of non-callable attribute access, isinstance check, etc. Making them incompatible is the whole point of typing the return types of these helpers with a separate Promise class for Promise-like objects. The interchangeable use cases arise in user-facing text such as Field.help_text or Field.verbose_name, where laziness is intended to be preserved, which we do want to explicitly type as a Union of the two types.

I did a quick search on GitHub and found that people use gettext_lazy with ValidationError, label/help_text/verbose_name/name argument of Field.__init__. These use cases should be pretty manageable, and we can add support to other ones. Moreover, in places where str is expected, the caller should force the _StrPromise into str with force_str or str.

[flaeppe]

I'm not following on why we need to type Promise at all? Isn't it an implementation detail? That just proxies whatever type/value you put in to it.

I mean, as a user of lazy/promise etc, you'll never need to know or directly consider that it's a Promise and e.g. not a str, since the string itself is just proxied. So you as a user can handle it as a regular string and the Promise falls under the radar. Or am I missing something?

---

Though the Union option suggested here looks a bit never ending, as it'll likely pop up quite a lot of cases where it'll have to be Union[str, Promise] instead of just str. While, typing and runtime wise, it would've been just fine to have seen str.

In addition to that, that's just to cover gettext_lazy, I mean, hypothetically there could be some getint_lazy which would then have to become Union[int, Promise], etc, and on it goes.

I think what I'm saying is that there have to be some Promise[str], that can be accepted as str, it doesn't look viable doing Union[str, Promise] everywhere. And in addition to that, pushing it to users of django-stubs to also start writing Union[str, Promise] in their own code.

[sobolevn]

Yeah, I agree with @flaeppe
This change seems quite invasive.

[PIG208]

I agree that this is probably invasive in terms of the functions that might be affected within Django. If that's the issue we can try to implement method+function hooks ensuring that Django's functions/methods that accept str should also accept _StrPromise, assuming that Django itself is well aware of the Promise objects and always correctly handles them.

However, it might be an overkill since it is unclear whether the Promise objects are commonly used Django APIs other than the above mentioned ones. Practically, I still prefer the current approach and wait and see if it is necessary to switch to a plugin implementation.

I mean, as a user of lazy/promise etc, you'll never need to know or directly consider that it's a Promise and e.g. not a str, since the string itself is just proxied. So you as a user can handle it as a regular string and the Promise falls under the radar. Or am I missing something?

When using lazy_str externally, it is more important to let the user know that this lazy object is not actually an instance of str. isinstance(gettext_lazy("foo"), str) for example returns False, which is something the callee in an external application wouldn't expect. There are more things that make Promise different from str in runtime.

In fact, the documentation explicitly states the difference between a str and Promise (referred as "lazy object" in the text) and that it does not work with arbitrary code. This actually has caused bugs downstream, so I think it is worth it to make this explicit.

pushing it to users of django-stubs to also start writing Union[str, Promise] in their own code.

We have already done something similar by typing the HttpResponse objects returned by the test client as _WSGIHttpResponse. This encourages the users to type the test responses as this fictional type we created in their helper functions.

Passing _StrPromise to APIs that expect str can lead to unexpected issues. django-stubs won't block users from using these APIs because they can always force the Promise object into str before calling.

In addition to that, that's just to cover gettext_lazy, I mean, hypothetically there could be some getint_lazy which would then have to become Union[int, Promise], etc, and on it goes.

We did consider making a generic Promise[T], but this use case is less prevalent compared to the work required to implement it.

[andersk]

As an example of what can go wrong if we aren’t truthful to mypy, consider

def maybe_decode(s: str | bytes) -> str:
    if isinstance(s, str):
        return s
    else:
        return s.decode()

maybe_decode(gettext_lazy("promise"))

If we incorrectly told mypy that gettext_lazy returns str or a subtype of str, then mypy finds no problem with this code, but this would be a false negative—it will crash at runtime.

There would also be false positives, such an incorrect “error: Statement is unreachable” on this else block with mypy --warn-unreachable:

def force(s: str | Promise) -> str:
    if isinstance(s, str):
        return s
    else:
        return str(s)

force(gettext_lazy("foo"))

Another reason that the user needs to care about the difference between Promise and str is that a Promise from gettext_lazy evaluates to different strings depending on the language of the current request, or within a with translation.override(language) block. It really is a different API, not just an implementation detail.

[flaeppe]

I can't come up with any better idea really. But I'm curious if these changes could result in users prematurely evaluating lazy strings.

But I guess those cases could be mitigated by patching the stubs.

Suppose we'll notice in time

[flaeppe]

Maybe _StrPromise should be public? Since it'll be exposed to users, right?

So that one can do str | StrPromise if passing around/accepting gettext_lazy values

Suggested change

	_StrOrPromise = Union[str, _StrPromise]
	_StrOrPromise = Union[str, StrPromise]

[PIG208]

_StrPromise -> StrPromise, _StrOrPromise -> StrOrPromise in the latest commit.

[intgr]

If we expect users to import and use StrPromise and StrOrPromise, they should go in django_stubs_ext instead.

Otherwise using them would be annoying, import would need if TYPE_CHECKING and the types must be quoted always.

[PIG208]

The PR is updated with aliases made available in django_stubs_ext.

[andersk]

This isn’t how re-exports work in mypy.

But I think your initial inclination to use leading underscores in django-stubs/utils/functional.pyi was correct, because these don’t exist in django.utils.functional at runtime. We should use underscores in django-stubs and re-export with no underscores here in django_stubs_ext.

[flaeppe]

Suggested change

	StrOrPromise = _StrOrPromise
	StrPromise = _StrPromise

[flaeppe]

Suggested change

	from django.utils.functional import Promise
	from django.utils.functional import Promise as StrPromise

[sobolevn]

LGTM, any other comments?

Thanks for working on this!

[flaeppe]

All good from my end

[PIG208]

Thanks. If any of you have time, #1072 and #1037 are also awaiting review. These are stub fixes that are relatively straightforward.