Feature: Support multi-line secrets

[shellscape]

There have been multiple issues opened about the lack of newline support for values in the past, some going back a few years. #215 took a stab at this specifically for certificates in env files. And yes, in most cases simply providing a path to a cert as a value does the trick. But not in all situations.

The rather large environment I'm working in involves packaging up an application into a single bundle and shipping that off to AWS via serverless. Now, we could massage things to allow us to also ship this private key to the Lambda function, but then legal would go bananas and tell us that we're not in compliance and there'd be a whole political poopstorm internally. And no one wants that. What it boils down to is that we cannot ship a cert to the app's running destination.

Here's a sample key:value for reference:

PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----
gobblygoop
-----END RSA PRIVATE KEY-----"

Note that this is not specific to certs, but to all multiline values:

MULTILINE='hello
goodby
allo'

For both, only the first line of the multiline value will be read, even though it's valid to wrap multiline values in quotes. They appear in process.env as:

PRIVATE_KEY: '"-----BEGIN RSA PRIVATE KEY-----',
MULTILINE: "'hello",

We could do what many have resorted to, and that's replacing every newline with '\n' in the cert in values in question. But holy crap does that get stupid long and it's extremely prone to errors.

This issue hits home especially hard since (on macs at least) if you place these values into a file and source the file, the system reads those values correctly. I haven't found a compelling argument against dotenv supporting the same.

What's holding dotenv back from being able to support this, when *nix flavors seem to be able to from the shell?

[lornajane]

I would also really appreciate support for multiline values, specifically so that we can encourage users to avoid pushing files containing their private keys into source control for cloud deployment.

As a side note: Adding the \n seems to be a bit error-prone for many users, I've moved to recommending base64 encoding and using the result as an environment variable instead - mentioned in case that helps anyone else!

[andreialecu]

Note that the ruby version of dotenv does support this:

https://github.com/bkeepers/dotenv#multi-line-values

Would be nice to have it here as well.

This seems to conflict with #376 however.

[andreialecu]

I have opened a PR at #486 to address this. In order to work around the conflict with #376 it is implemented behind an option: { multiline: "line-breaks" } which switches to the new behavior.

[motdotla]

This is on the roadmap now. Thank you for the great PR.

[axmachina]

In the mean time, can accomplish this with rewire:

.env

list=```[
  one,
  two,
  three
]```

jsonlist=```[
  "one", // comment
  "two", # another comment
  "three",
]```

code

const rewire = require("rewire");
const dotenv = rewire("dotenv");

// original parse function
const dotenvParse = dotenv.__get__("parse");

/**
 * @param {String} src .env contents
 * @param {import('dotenv').DotenvParseOptions} opts 
 */
dotenv.__set__("parse", (src, opts) => {
  // fix multi-line params to single line
  const commentRegex = /(\/\/|\#).*/g;
  // using negative look-around will match any string, or line without a line break, not containing the (sub)string "```".
  const multiValueRegex = /\=\s*```(((?!```).)*)```/gs;
  const fixed = src.replaceAll(multiValueRegex,(m, value) => {
    return "="+value.replaceAll(commentRegex,"").replaceAll(newlineRegex," ");
  });
  return dotenvParse(fixed, opts);
});

const dotenvResult = dotenv.config({path});

// restore original dotenv parser
dotenv.__set__("parse", dotenvParse);

[weihao]

I agree with this post. We don't want to reformat the key files with newlines, so support for multiples lines is needed.

#28
The issue has been around since 2014...