Consistent handling of environment variables

[Rich-Harris]

Describe the problem

SvelteKit doesn't do anything special with environment variables; it delegates to Vite. This is good for the sorts of environment variables Vite deals in — VITE_ prefixed values that are replaced at build time — but less good for other kinds.

Environment variables can be static (known at build time) or dynamic (not known until run time), and private (only accessible to code that runs on the server) or public (accessible to code that runs on the server or the client):

	static	dynamic
public	✅ import.meta.env.VITE_*	❌ (1)
private	⚠️ can be done with plugins (2)	⚠️ process.env.* (3)

1. To be fair, I'm not sure what sorts of things would go in this quadrant, though I would be eager to hear examples
2. You can add something like plugin-replace to selectively replace things at build time, but this is ad hoc, undiscoverable, and involves duplication between .env files and config
3. process.env is only available in Node-based environments (e.g. adapter-node, adapter-netlify and adapter-vercel — though once the Vercel adapter starts building Edge Functions that will no longer be the case)

It would be great if we had a consistent and idiomatic way of dealing with all of these.

Describe the proposed solution

I don't actually have any proposals to make, I just didn't want #4293 (comment) to get lost. Whatever solution we land on should

• work with .env/.env.* files
• enable dead code elimination etc at build time
• allow autocompletion/auto-imports (e.g. we could generate typings from your .env file)
• feel consistent across the four quadrants, but with clearly understandable distinctions between them (e.g. $app/env/public vs $app/env/private or whatever)

Alternatives considered

No response

Importance

would make my life easier

Additional Information

No response

[bluwy]

For dynamic-public env vars, I had a usecase for it and lightly touched on it before. But the gist is that a frontend would connect to a backend URL, which can be public, but the backend URL could change for e.g. testing purposes, or it's not known in build time. But that said, it should be a userland concern, and isn't something SvelteKit/Vite has to handle IMO.

[tsongas]

I like https://github.com/kerimdzhanov/dotenv-flow.

[bfanger]

The current recommendation (according to the FAQ) is to use a getSession hook

But using import { session } from "$app/stores"; get(session).MY_VAR outside of a svelte component doesn't work, which limits its use.
(It uses getContext in the subscribe, this is by-design, and desired because on the server you really want to isolate these stores per request)

Idea, part 1 (AdapterEnv)

import { env } from "$app/adapter";

console.log(env.TRACKING_ID);

The values inside the env object will be provided by the adapter, in case of adapter-node and the dev server it will be the process.env after dotenv, on adapter-deno it will be Deno.env.toObject()

The values can be used to store secrets such as connections string and are therefor only are available on the server.
This import is not available on the client.

The "$app/adapter" import would also allow adapters to expose emulation for features in development like Cloudflare KV, but that's another topic

Idea, part 2 (env Hook)

To expose variables to client code we could add a getPublicEnv hook.

in hooks.js:

export function getPublicEnv(env: AdapterEnv) {
  return {
    API_ENDPOINT: env.API_ENDPOINT,
    TRACKING_ID: env.TRACKING_ID,
  };
}

The getPublicEnv hook is executed only once at startup, after that the values are set and can be imported from anywhere. (no context needed)

Idea, part 3 (generated exports)

To access the public environment variables you'd write:

import { API_ENDPOINT } from "$app/env";

Using $app/env here allows the server to also access the values from the getPublicEnv hook.
The getPublicEnv must therefore return an object with the same keys in all environments.

Summary

• Work with .env/.env.* files - ✅ Possible but is up to the adapter to implement
• Enable dead code elimination etc at build time - ✅ Still possible by using the VITE_ variables
• Allow autocompletion/auto-imports ✅ generated app/env based on the output of the hook.
• Feel consistent across the four quadrants ❌ Adding getServerEnv, getPublicBuildEnv and getServerBuildEnv hooks might make the api more uniform, but doesn't add much clarity imho.

[Rich-Harris]

Another dimension along which we're currently inconsistent: #3040. Vite loads .env files in dev and build, but using the example .env file...

DB_PASSWORD=foobar
VITE_SOME_KEY=123

...VITE_SOME_KEY is replaced in both cases, but DB_PASSWORD only has meaning in dev. Part of me thinks this is all a bit confusing and we'd ideally bypass Vite's automatic .env loading, but that might annoy people.

[elliott-with-the-longest-name-on-github]

@Rich-Harris, I'm hugely passionate about this one. Maybe it's because I was talking about it and frustrated by it even back in June of last year... I encourage you to scroll through that conversation for context about how confusing this is to someone new to the ecosystem -- the fact that Vite statically replaces process.env.foo but NOT process.env['foo'] is not straightforward. I wasted several days battling with how to get Vite to play nicely with my environment variables, and overall, the number of kludgy workarounds I have to use is just too high. Kit needs to address this, like you said, completely and consistently. Thanks for opening this issue for discussion.

Some notes, in no particular order:

I'm a TypeScript-first user, so generating types based on the .env file sounds neat, but I'd much rather be able to declare my expected environment variable shapes similarly to how I declare my Session type. Managing .env files across teams/users is tough, because they're not usually committed to the repository -- meaning their shape is documented in the README or not at all. I'd much rather have them documented through code.

As far as API goes, if I could have my way, it'd look like this:

import { env } from '$app/buildtime';
import { privateEnv } from '$app/buildtime';
import { env } from '$app/runtime';
import { privateEnv } from '$app/runtime';

Whatever solution we build should be able to guarantee that no properties on the privateEnv object can be built into client-exposed code or exposed to the client at runtime. I think in the case of environment variables, we should not reinvent the wheel. Let userland rename the variables if they want (import { env as buildtimePublicEnv } from '$app/buildtime';), and make the object returned a plain old JavaScript object. As far as the typing stuff, the default would be (just spitballing):

declare namespace App {
  interface Environment {
    buildtime: {
      env: Record<string, string>,
      privateEnv: Record<string, string>,
    }
    runtime: {
      env: Record<string, string>,
      privateEnv: Record<string, string>,
    }
  }

As a bonus, future feature, we may be able to have Kit validate that the expected keys appear in .env. If they're buildtime variables, it could throw during build if any are missing. If they're runtime variables, it could throw on initialization.

This isn't meant to be a complete proposal, per se, but more just to try to shuffle the conversation in a direction I think might be beneficial. Heck, it'd certainly be beneficial to me.

Once the team can decide on a full design spec, I'd be happy to help implement this.

[luisgabrielm]

Whatever solution we build should be able to guarantee that no properties on the privateEnv object can be built into client-exposed code or exposed to the client at runtime

@tcc-sejohnson I strongly agree with being clear about which variables are being exposed in the client.

I started coding in the backend, so the security concerns were rarely about mistakenly exposing a private-key to the client. And, whenever I worked with the frontend I was careful to always use a proxy API or something similar.

Since sveltekit has both client side and server side code it does make me very worried that I’ll expose something by accident. If it’s super clear what’s being used where I think that’s a major feature of the framework.

[elliott-with-the-longest-name-on-github]

Hey @Rich-Harris,

Like I said, I'm pretty passionate about this one -- so here's a full design proposal to get people talking.

To restate the points Rich laid out in the original comment, a solution needs to:

• work with .env/.env.* files
• enable dead code elimination etc at build time
• allow autocompletion/auto-imports (e.g. we could generate typings from your .env file)
• feel consistent across the four quadrants, but with clearly understandable distinctions between them (e.g. $app/env/public vs $app/env/private or whatever)

After thinking about it for some time, I think it should also:

• be composable to some degree
• guarantee from a Kit perspective that private variables are not exposed to the client
• provide the ability to validate that the environment config is valid at startup/buildtime
• provide a relatively easy-to-follow execution path

Let's talk about those.

Be composable to some degree

The immediate use-case I can think of is when my app needs to know an environment-dependent URL for an api:

• local: http://localhost:7071
• dev: https://asdfhjkl.azurewebsites.net
• staging: https://staging.api.mywebsite.com
• prod: https://api.mywebsite.com

Whenever I need to reference this URL within my code, I often need to change behavior based on whether I'm operating in HTTP/HTTPS. Sometimes I need to know the port. Sometimes I just need the host, etc. Sure, I can just new URL(process.env['API_URL']), but that's annoying -- I have to do it every time I need the variable. I'd much rather structure my environment config as:

API_PROTOCOL=http
API_HOST=localhost
API_PORT=7071
API_URL={API_PROTOCOL}://{API_HOST}{API_PORT ? ':' + API_PORT : ''}

There are other instances where I'd like composability (or just the ability to compose a JavaScript object out of a string config value without having to do so at every single line that uses the config value). More on this in the design section. 😺

Guarantee no client exposure of private variables

Pretty simple. Obviously, Kit can't guarantee you don't just create a GET endpoint that returns MY_PRIVATE_VARIABLE to the frontend, but it should be able to assert that accessing a private variable from the client is invalid.

Provide the ability to validate config at startup/buildtime

I've wasted more time than I probably should by forgetting to include a new environment variable in .env and then having to track down why my credentialed API call isn't working. ☹️ It'd be nice to be able to assertively validate that the environment is what is expected at buildtime and runtime.

Provide a relatively easy-to-follow execution path

There are a lot of complicated ways to implement something like this. Let's not do one of them.

Design

I propose the addition of a few new exports from the $app route, one new root file, startup.{js|ts}, one new argument to the CLI, environment, and four new types to the App namespace. Users can define environment variables through a .env | .env.{environment} file or through environment variables present on the system at buildtime (or at runtime, if supported by whichever adapter). Environment variables will override .env variables. At both startup and buildtime, the variables will be resolved to a JavaScript Record<string, string> and passed to startup.{js|ts}, where users can validate and compose their env (more on this later). From there, environment variables will be available as import { env, privateEnv } from '$app/{buildtime|runtime}'.

With that summary, let's follow the "execution flow" and describe in more detail how this would work.

.env | .env.environment

Same design as everyone is used to for .env files. KEY=value, supports strings. .env.{environment} is loaded when svelte-kit {do-something} --environment={environment}. (Technically, we could reuse the Vite mode here, but given that we're building our own environment config, I would think we would want to separate it conceptually from Vite as much as possible.)

In order to support our four quadrants, we'll need some sort of Vite-like prefixing scheme. I'm not particularly attached to any one scheme, so I'm open for suggestions, but for the sake of this proposal, we'll say variables prefixed with ___ are ___:

• BLDPRV, buildtime private
• BLDPUB, buildtime public
• RUNPRV, runtime private
• RUNPUB, runtime public

We could also potentially allow simply PRV or PUB to indicate that the variable should be serialized during buildtime but also be available during runtime. The more specific RUNPRV/BLDPRV would take precedence.

startup.{js|ts}

Why isn't this file named env.{js|ts}? Well, because, we can set ourselves up to kill two birds with one stone here. (Or we can just kill one bird and call it .env.{js|ts}. Up for debate.)

This file exports four methods with the following defaults:

export const resolvePublicRuntimeEnv = (publicEnv: Record<string, string>): App.PublicRuntimeEnv => publicEnv;
export const resolvePrivateRuntimeEnv = (privateEnv: Record<string, string>): App.PrivateRuntimeEnv => privateEnv;
export const resolvePublicBuildtimeEnv = (publicEnv: Record<string, string>): App.PublicBuildtimeEnv => publicEnv;
export const resolvePrivateBuildtimeEnv = (privateEnv: Record<string, string>): App.PrivateBuildtimeEnv => privateEnv;

Each of these receives the resolved variables from .env and the environment. Users can validate the values, compose them as they wish, and return the resolved environment.

New App types

You'll notice I referenced types from the App namespace above. I propose the following be added:

declare namespace App {
  interface PublicRuntimeEnv extends Record<string, string>;

  interface PrivateRuntimeEnv extends Record<string, string>;

  interface PublicBuildtimeEnv extends Record<string, string>;

  interface PrivateBuildtimeEnv extends Record<string, string>;
}

Like the other App types, these can be set by the user.

Edit: The types should default to Record<string, string>, but users should be able to set them to anything extending Record<string, any> (and maybe we should allow numbers as indexers as well?).

User API

The objects returned from resolveXxxXxxEnv will be available to the developer throughout the app via import { env, privateEnv } from '$app/{buildtime|runtime}'. Importing privateEnv client-side (regardless of whether any of its properties are accessed) will result in an exception.

A full example, using my previous example

So, using the previous example:

// .env
BLDPUB_API_PROTOCOL=http
BLDPUB_API_HOST=localhost
BLDPUB_API_PORT=7071

// app.d.ts
declare namespace App {
  interface PublicBuildtimeEnv {
    apiUrl: URL;
  }
}

// startup.ts
const throwError = (missingKey: string) => throw Error(`Could not find ${missingKey} in .env or environment variables.`);

export const resolvePublicBuildtimeEnv = (env) => {
  if (!env.BLDPUB_API_PROTOCOL) {
    throwError('BLDPUB_API_PROTOCOL');
  }
  if (!env.BLDPUB_API_HOST) {
    throwError('BLDPUB_API_HOST');
  }

  const port = env.BLDPUB_API_PORT ? `:${env.BLDPUB_API_PORT}` : '';
  const apiUrl = new URL(`${env.BLDPUB_API_PROTOCOL}://${env.BLDPUB_API_HOST}${port}`);
  return { apiUrl }
}

// anywhere else in my code
<script lang="ts">
  import { env } from '$app/buildtime`
</script>

// typing provided by app.d.ts!
// 'http://localhost:7071' serialized into code at buildtime
<p>{env.url}</p>

Does this meet the requirements?

• work with .env/.env.* files
  • Yes
• enable dead code elimination etc at build time
  • Maybe? I'm not super familiar with how this works.
• allow autocompletion/auto-imports (e.g. we could generate typings from your .env file)
  • Absolutely.
• feel consistent across the four quadrants, but with clearly understandable distinctions between them (e.g. $app/env/public vs $app/env/private or whatever)
  • I think so, but up to the community to decide.

(My added requirements)

• be composable to some degree
  • Yes, supports resolving arbitrary environment config from Record<string, string>
• guarantee from a Kit perspective that private variables are not exposed to the client
  • Yes
• provide the ability to validate that the environment config is valid at startup/buildtime
  • Yes
• provide a relatively easy-to-follow execution path
  • To me, the .env => startup.ts => import { env, privateEnv } from '$app/{buildtime|runtime}' pipeline is pretty darn simple.

Quick note about startup.{js|ts}

Way back when, I opened #1753 to document "startup" behavior in hooks.js. Essentially, if you've got server-side startup code, running it in the root of hooks.js and waiting for it to finish in handle before serving any requests is the only way to do it.

To me, this behavior feels really bad. hooks.js doesn't feel like the place it should be done. The name isn't right, all of the other things hooks have to do have nothing to do with startup, etc. Also, having to await startup() in handle is nothing but dead code after startup is complete, but it still has to be run for every incoming server request. Ew. Introducing this startup.js file would naturally lead to a fifth export, startup, which would receive the resolved config and run (blocking) before the "server starts" (the exact behavior would likely be adapter-specific?). I'm just taking the opportunity to introduce that idea and scaffold it while we're looking at this.

Please rip this apart, provide your controversial opinions, and hate on my design. If you notice any typos or anything that seems to stupid to be an actual suggestion, let me know so that I can correct it.

😺

[shiftlabs1]

@Rich-Harris can we borrrow a leaf here https://docs.astro.build/en/guides/environment-variables/ instead of letting this drag for so long . import.meta.env.WHATEVER works on Server and client side in Astro. BUT only environment variables prefixed with PUBLIC_ are available client side .note they also available server side so that code running on client and server have the same mode of extracting environment variables. I am porting a small app to astro edge functions given how long this is taking to get stamped