Question: producing https addresses using url_for

[cglacet]

I'm looking for a way to produce static file routes with https prefixes instead of the default http. For example, say I have the following template:

<script src="{{ url_for('static', path='file.js') }}"></script>

I would like it to produce:

<script src="https://my-app.herokuapp.com/static/file.js"></script>

I don't find information about this on the documentation, so I was wondering if it was possible? I can't use the <script src="/static/file.js"></script> form because I want to use the generated html on a different domain.

In case it makes a difference: I'm using fastAPI and the app is running on heroku.

[devsetgo]

Did you ever get an answer on this? I am having a similar problem using jwilder/nginx-proxy as it redirects automatically to https, but the references are to http://domain.name instead of https://domain.name.

The difference is I am using Starlette and not FastAPI.

[tiangolo]

Note: If in a hurry, skip to the end.

The way this information is passed to the app is via the ASGI spec itself. The server (Uvicorn, Hypercorn, or any other) is the one that tells the app if it is running on HTTPS or HTTP.

If you provide the HTTPS certificates to the ASGI server (e.g. Uvicorn) it will know it is running on HTTPS and pass that information to the framework (Starlette, FastAPI, or anything else).

But it's very common to have a "TLS Proxy" on top of it. That would be something that has the HTTPS certificates, handles the connection, and passes the pure HTTP to the thing running behind (in this case, Uvicorn, running your Starlette/FastAPI app). Examples of programs that can run as TLS Proxies include Nginx, HAProxy, Traefik (I recommend Traefik 😉 ). The same would be done by Heroku or jwilder's Docker image (based on Nginx).

But these TLS Proxies (and actually many other layers and servers) create some HTTP headers to let the thing that runs after them know that they are handling HTTPS for them.

But by default, none of the intermediate parts (Nginx, Traefik, Uvicorn) receive and accept those HTTPS headers from outside, as that would be a security risk. But if you know that the specific part (e.g. Uvicorn) is behind a TLS Proxy, you can normally configure/override it to receive those HTTP headers about the HTTPS connection.

In Uvicorn, the command parameter is --proxy-headers.

[devsetgo]

@tiangolo - Thank you for the response and really like FastAPI by the way. This solves the problem. I also temporarily solved it by adding to my template until I make the changes in Nginx or switch to Traefik (been meaning to try it).

In Flask the URL would be served up relative , but in Starlette it is served up absolute. Would it not be better for it to be relative vs absolute?

Maybe it should just be an option in how static files are severed. Something like this.
app = Router(routes=[
Mount('/static', app=StaticFiles(directory='static', absolute=False), name="static"),
])

If I am correct in how I think Starlette is operating it would be good to update the static file documentation to include that it is absolute path and not relative. Plus how to get Uvicorn use --proxy-headers.

[tiangolo]

@devsetgo if you want to use relative URLs, I think you can just write them as is in your template, you then don't need to use url_for.

I think the main thing url_for does is add the corresponding http or https and the domain that was requested (that is not known in the code before).

But with relative URLs you don't need those, so I think you can just use the explicit relative URL.

[lovelydinosaur]

Using --proxy-headers is the right way to resolve this.

However it might be nice to not have to worry about this, and for us to just serve up relative urls by default, since there's less potential for misconfiguration that way.

We could still provide an alternative for absolute URLs, but we could more throughly document how to ensure it's properly configured.

Also relevant is Kludex/uvicorn#369, since that'd mean that we're getting the configuration correct automatically more often.

[i4x]

@tiangolo why the FastAPI doesn't work correctly, even when --proxy-headers is passed to uvicorn directly? The request.url, when imported as described, doesn't return the right scheme. (Sorry for saying this on Starlette's side, but relative URLs are not enough. Starlette works great, and has no issue here.)

[lovelydinosaur]

@i4x Could you make sure to update to the latest version of uvicorn? What platform is the service deployed to? Can you output request.headers and request.scope so we can see exactly what's being passed through?

[lovelydinosaur]

Note that the latest version of uvicorn now matches gunicorn behaviour - by default it will accept any X-Forwarded-For and X-Forwarded-Scheme headers if the client IP is in --forwarded_allow_ips (Defaults to the $FORWARDED_ALLOW_IPS environment variable, or else "127.0.0.1")

[i4x]

The version tested in the parent was uvicorn==0.10.2, that's installed automatically, when I install fastapi==0.42.0. After update to uvicorn-0.10.3, the behavior didn't change. I'm using Caddy as proxy (and it works well with uvicorn==0.9.1), and what's in request.headers under uvicorn==0.10.2, is:

{"host":"mydomain.net","user-agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9","cache-control":"max-age=0","cookie":"SESS99def2ca8b51de44904e6acf20ead392=8t5neatlj9ffn43933toggif16","sec-fetch-mode":"navigate","sec-fetch-site":"none","sec-fetch-user":"?1","upgrade-insecure-requests":"1","x-forwarded-for":"192.168.1.254","x-forwarded-proto":"https","x-real-ip":"192.168.1.254"}

The request.scope under uvicorn==0.10.2, was:

{'type': 'http', 'http_version': '1.1', 'server': ('192.168.1.103', 1717), 'client': ('192.168.1.185', 42582), 'scheme': 'http', 'method': 'GET', 'root_path': '', 'path': '/items/1', 'raw_path': b'/items/1', 'query_string': b'', 'headers': [(b'host', b'mydomain.net'), (b'user-agent', b'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36'), (b'accept', b'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3'), (b'accept-encoding', b'gzip, deflate, br'), (b'accept-language', b'en-US,en;q=0.9'), (b'cache-control', b'max-age=0'), (b'cookie', b'SESS99def2ca8b51de44904e6acf20ead392=8t5neatlj9ffn43933toggif16'), (b'sec-fetch-mode', b'navigate'), (b'sec-fetch-site', b'none'), (b'sec-fetch-user', b'?1'), (b'upgrade-insecure-requests', b'1'), (b'x-forwarded-for', b'192.168.1.254'), (b'x-forwarded-proto', b'https'), (b'x-real-ip', b'192.168.1.254')], 'fastapi_astack': <contextlib.AsyncExitStack object at 0x7fbdd5f5b6d0>, 'app': <fastapi.applications.FastAPI object at 0x7fbdd8f7a410>, 'router': <fastapi.routing.APIRouter object at 0x7fbdd8f7a610>, 'endpoint': <function read_root at 0x7fbdd5f53cb0>, 'path_params': {'item_id': '1'}}

---

Downgrading the default uvicorn installed by FastAPI to uvicorn==0.9.1 resolved the issue. The request.headers under uvicorn==0.9.1 was:

{"host":"mydomain.net","user-agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9","cache-control":"max-age=0","cookie":"SESS99def2ca8b51de44904e6acf20ead392=8t5neatlj9ffn43933toggif16","sec-fetch-mode":"navigate","sec-fetch-site":"none","sec-fetch-user":"?1","upgrade-insecure-requests":"1","x-forwarded-for":"192.168.1.254","x-forwarded-proto":"https","x-real-ip":"192.168.1.254"}

The request.scope under uvicorn==0.9.1 was:

{'type': 'http', 'http_version': '1.1', 'server': ('192.168.1.103', 1717), 'client': ('192.168.1.254', 0), 'scheme': 'https', 'method': 'GET', 'root_path': '', 'path': '/items/1', 'raw_path': b'/items/1', 'query_string': b'', 'headers': [(b'host', b'mydomain.net'), (b'user-agent', b'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36'), (b'accept', b'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3'), (b'accept-encoding', b'gzip, deflate, br'), (b'accept-language', b'en-US,en;q=0.9'), (b'cache-control', b'max-age=0'), (b'cookie', b'SESS99def2ca8b51de44904e6acf20ead392=8t5neatlj9ffn43933toggif16'), (b'sec-fetch-mode', b'navigate'), (b'sec-fetch-site', b'none'), (b'sec-fetch-user', b'?1'), (b'upgrade-insecure-requests', b'1'), (b'x-forwarded-for', b'192.168.1.254'), (b'x-forwarded-proto', b'https'), (b'x-real-ip', b'192.168.1.254')], 'fastapi_astack': <contextlib.AsyncExitStack object at 0x7fa8cb9c3e90>, 'app': <fastapi.applications.FastAPI object at 0x7fa8cee52250>, 'router': <fastapi.routing.APIRouter object at 0x7fa8ce531550>, 'endpoint': <function read_root at 0x7fa8cb9c6320>, 'path_params': {'item_id': '1'}}

[i4x]

Note that the latest version of uvicorn now matches gunicorn behaviour - by default it will accept any X-Forwarded-For and X-Forwarded-Scheme headers if the client IP is in --forwarded_allow_ips (Defaults to the $FORWARDED_ALLOW_IPS environment variable, or else "127.0.0.1")

Wonderful. So, under uvicorn==0.10.2, in the systemd, adding Environment="FORWARDED_ALLOW_IPS=192.168.1.185" resolved the problem for me, and now the request.url has correct scheme. :) Thanks, @tomchristie

[lovelydinosaur]

Wonderful! Thanks for the feedback. ✨