Allow env vars to be checked in for the server deploy

[paulmelnikow]

What do you think of this approach to handling config on the servers? This is intended as a way to turn on the work we did in #1725 on the VPS without SSH access.

Also ref: #1723 #1568 #1742 (comment)

Todo:

• Test on staging
• Update deploy task
• Update gitignores?

[shields-ci]

	Warnings
⚠️	This PR modified package.json, but not package-lock.json - Perhaps you need to run npm install?
⚠️	This PR modified the server but none of the service tests. That's okay so long as it's refactoring existing code.
⚠️	This PR modified helper functions in lib/ but not accompanying tests. That's okay so long as it's refactoring existing code.

	Messages
📖	✨ Thanks for your contribution to Shields, @paulmelnikow!

Generated by 🚫 dangerJS

[chris48s]

I don't have strong opinions on this because It impacts on a process that you and @espadrine run, so I'll defer to @espadrine 's review.

[PyvesB]

It's been over a week since this pull request was opened. In the meantime, could we not simply hard-code a non 0 value for the BADGE_MAX_AGE_SECONDS variable added in #1725? This would only be a temporary measure and we can simply revert it once this pull request moves forward. 😉

[paulmelnikow]

That seems reasonable to me.

Since we're so often having serious issues, maybe we should start with 60 seconds?

[Nyholm]

Why not 300? or 600?

I rather see a badge with stale data then no badge at all.

[espadrine]

We should federate env vars and secrets vars instead. That way, all env vars that we want to check in can be stored in secrets.json. No need for multiple files with poorly-defined differences, where people won't know which file contains which flag.

[paulmelnikow]

This is coming up again because we need to turn on a feature flag for #2069.

Since we don't have so much configuration, I'd tend to go in the opposite direction, and put everything in the environment. Environment vars are the most common way of handling this. For self-hosting users on Heroku or Now, configuring through env vars will be much easier because they can be set persistently from the command line or a UI, and don't require a deploy commit.

Occasionally for complex systems I've used JSON in environment variables, though I don't see any need for it here. We can use one env var per setting.

Though I'm not sure quite what the alternate proposal is, I don't think we should create a solution that requires the servers to talk to each other in order to configure themselves. Immutability isn't just trendy. A system with fewer moving parts is more predictable, and there's less code to write. I made a similar argument in #1848 (comment).

[paulmelnikow]

Have labeled this a blocker because of #2069.

[platan]

It looks good to me. This PR gives us another way to configure server, which is a good idea in my opinion. This looks similar to externalized configuration in Spring Boot, where you can configure application using several options.
dotenv does not override existing environment variables (doc).
Conflicts must be resolved and currently 6.2.0 is the newest version of dotenv.
@paulmelnikow how exactly do you want to set environment variables on our servers?

[paulmelnikow]

I opened #2621 with a proposal to use node-config which would let us check in the non-secret part of the config in the main repo. We can use dotenv with a deploy commit to set NODE_CONFIG_ENV in the current / legacy production deploy, and handle the secrets via checked-in local-shields-io-public.yml or additional checked-in environment variables.

[paulmelnikow]

Closing in favor of #2626.