Reorganization of ParseServerOptions

[dblythy]

Is your feature request related to a problem? Please describe.
As Parse Server continues to evolve, so do the Parse Server options.

Describe the solution you'd like
There are certain options that in my view should be grouped together for neater organization, such as:

User Options:
-accountLockout
-enableAnonymousUsers
-expireInactiveSessions
-passwordPolicy
-preventLoginWithUnverifiedEmail
-revokeSessionOnPasswordReset
-sessionLength
-userSensitiveFields

Database Options:
-allowClientClassCreation
-allowCustomObjectId
-collectionPrefix
-databaseOptions
-objectIdSize
-protectedFields

REST Options:
-allowHeaders
-allowOrigin
-enableExpressErrorHandler
-middleware

Adapters:
-analyticsAdapter
-auth
-cacheAdapter
-databaseAdapter
-emailAdapter
-filesAdapter
-loggerAdapter
-push

Keys:
-clientKey
-databaseURI
-dotNetKey
-encryptionKey
-fileKey
-javascriptKey
-masterKey
-readOnlyMasterKey
-restAPIKey
-webhookKey

Server Info:
-appId
-appName
-cloud
-cluster
-customPages
-directAccess
-host
-jsonLogs
-masterKeyIps
-maxLimit
-mountPath
-port
-publicServerURL
-serverURL

Cache:
-cacheMaxSize
-cacheTTL
-enableSingleSchemaCache
-schemaCacheTTL

Email Options:
-emailVerifyTokenReuseIfValid
-emailVerifyTokenValidityDuration
-verifyUserEmails

Graph QL:
-graphQLPath
-graphQLSchema
-mountGraphQL
-mountPlayground
-playgroundPath

Idempotency:
-idempotencyOptions

Live Query:
-liveQuery
-liveQueryServerOptions
-startLiveQueryServer

Logs:
-logLevel
-logsFolder
-maxLogFiles
-silent
-verbose

Files:
-maxUploadSize
-preserveFileName

Push:
-scheduledPush

Server Events:
-serverCloseComplete
-serverStartComplete

I don't think every option requires categorization, but I think it could be handy to have all similar options tied together and cleaner in the docs, such as:

email : {
  verifyUserEmails: true,
  emailVerifyTokenValidityDuration: 2 * 60 * 60,
  preventLoginWithUnverified: false,
  resetTokenReuseIfValid: false
}

Considerations:

-How to handle options that could be under 2 categories, such as emailAdapter, which could be under adapters and email options
-This change would either need to be backwards compatible, or be a breaking change requiring changes to existing index.js.

The goal of this would be to ensure consistency and maintainability for ParseServerOptions, and to prevent them from continuing to bloat.

Please let me know your thoughts.

[mtrezza]

Thanks for suggesting.

This would certainly be an improvement in terms of configuration structure and oversight.

However, as discussed in the past, this would have to be look at holistically as it would be a breaking change affecting virtually every Parse Server deployment. The change efforts required can vary greatly depending on the specific environment, automation complexity and setup in which Parse Server is deployed.

Internally, changed would have to be made in:

• Parse Server config definition
• test cases
• docs

Ergo, the frequency of such changes should be kept as low as possible and any fundamental overhaul of the Parse Server configuration should be well crafted for the foreseeable future. Such a change may as well justify - or be part of - a Parse Server major version increment.

All this goes to say that just regrouping parameters may not be all that should be considered here, but also:

• establish and document a naming syntax for new parameters
• rename existing parameters according to the syntax
• evaluate current parameters and their related features as to whether they should be combined, deprecated or refactored

[dblythy]

Fair comments. It’s my view that the problem will continue to get worse, however I agree it’s hard to justify such a breaking change for such a nit.

Personally I think cacheOptions, GraphQLoptions and keys are probably the only ones that need better organisation.

[mtrezza]

I agree it’s hard to justify such a breaking change for such a nit

I think it justifiable, just needs to be looked at holistically. I think it should be taken on.

I would advice to first discuss the intend changed here in detail before creating a PR because this will require so many changes that the PR will likely need several rebases/merges and face merge conflicts.

I suggest this process:

• define new syntax for parameters (node and env var naming, description text)
• identify any outdated/refactorable parameters
• suggest new config structure (+ env vars)
• refactor description text for each parameter
• present final config structure for community feedback
• prepare docs change PRs
• create parse server and SDK PRs
• merge docs PRs

[Moumouls]

i think the Parse Server Options need this kind of re work, and group of options proposed by @dblythy at first glance seems not bad at all.

But the breaking change is massive for parse developers and also for all of our internal code (tests, etc...)
I think this kind of rework need a deprecated strategy.

During one major version we need to support current options and the new one.

Also since changes could be huge and lead to massive PR. I would propose also a development process:

0. Prepare a code base to support old Options and new Options

1. Create a new-parse-server-options branch on parse-server

2. Define an option group to support (ex: Database Options)

3. Work on it (and continue to support current option, i think group name will not conflict)

4. Fix tests

5. Send a PR from working branch to new-parse-server-options

Do it again for each option group.
This way, we can contribute progressively to support the new Parse Server Options.

Finally when new-parse-server-options is ready, we can open the final PR to master and start showing some deprecated warnings when old ParseServerOptions are used, then old ParseServerOptions could be removed in next major version

What do you think about that ?

[mtrezza]

I'm supportive of slowly phasing out the old options while still supporting them alongside with the new options, if that does not complicate the code too much or introduce other costs.

This change has no functional implication, it is merely aesthetics, so the costs and risks for however this is done should be kept as low as possible.

Another strategy to consider may be introducing a config version parameter as part of the configuration. The parameter can default to the old config, but developers can set it to the new config and use the new structure. Internally, this only requires a translation method that translates the old config into the new config. It does not require multiple PRs or to change all tests at once. It is similar to AWS policy versioning but for Parse Server configuration:

{
   configVersion: 20210101,
   ...
}

We may benefit from this feature in the future, so the efforts are not for just a one time change.

[dblythy]

So with the config version idea, Config.validate would translate according to the configVersion?

e.g,

configVersion 20210101 converts config.keys.masterKey to config.masterKey, so that the rest of the code base can still access config.masterKey

[mtrezza]

configVersion 20210101 converts config.keys.masterKey to config.masterKey, so that the rest of the code base can still access config.masterKey

Yes, along these lines, but that's just an idea, maybe that complexity is not needed after all.

[dblythy]

I would imagine in terms of lines changed it would be the simplist change (rather than changing every single mention of config.masterKey), as all the current lines would stay the same. There would just need to be a function for transposing the keys depending on the version - which I don't think the logic is too complicated - and test cases depending on the configVersion. It would also be easy to make backwards compatible.

I'd imagine ENV variables would have to stay the same as they are, which I don't think will be too big of a deal.

[mtrezza]

Well, that's something we'd have to figure out. It could create more confusion if there is a discrepancy between the configuration structure on how to access the config internally. I think internally, the config should be adapted to the new structure, because it is more developer friendly, and the old config should be transposed to the internal new config. But again, this approach was just an idea, the effort may not be necessary.

[mtrezza]

Keys:
-clientKey
-databaseURI
-dotNetKey
-encryptionKey
-fileKey
-javascriptKey
-masterKey
-readOnlyMasterKey
-restAPIKey
-webhookKey

Just noticed, I don't think databaseURI fits in keys; also keys that belong to a distinct features such as fileKey or encryptionKey may fit better in that distinct feature group.

[sadortun]

@mtrezza After working with https://www.npmjs.com/package/convict i think it could be a really good way to expose a configuration interface as it allow to define a configuration schema, and validation.

It also allow binding to ENV, and defining prod/dev/test configs