Standardize config files

[cob-web-corner]

Hello,

This is a rather large ask but would make things much easier for parsing, IaC, clustering, dynamic setup, provisioning from config files, Kubernetes config map, docker configuration, etc. Currently the config files located in /etc/dns are of an arbitrary nature. It would be nice to have these in some sort of standardized format that could be statically typed and parsed to classes (i.e. yaml/json). It would additionally significantly reduce your config file writers/readers and map to classes that are already defined.

Some of the apps are already standardized (dnsApp.config for example)

[ShreyasZare]

Thanks for the post. I have had this thought already but the task will need a lot of changes since entire code base is designed to serialize/de-serialize data in binary formats. From development perspective, the readers/writers are not much of an issue since changes are incremental. The original idea to have binary format was to discourage manual editing of the config so that the HTTP API is preferred instead. That was opinionated design decision so that the DNS server can do proper validation with HTTP API and also apply those changes immediately without need to reload/restart like with usual daemons.

[cob-web-corner]

I have had this thought already but the task will need a lot of changes since entire code base is designed to serialize/de-serialize data in binary formats

Yeah when I took a quick look through the code on the config read/writer side it definitely seemed like it would be an effort

That was opinionated design decision so that the DNS server can do proper validation with HTTP API and also apply those changes immediately without need to reload/restart like with usual daemons.

In this case those classes that do the validation could be abstracted to also work when loading the config files

Totally understood overall it would be a large effort. My worker around right now is to load a scratch instance, inject everything through the API and then grab the generated config files. It's an init container that runs, a python script runs everything in a yaml file through the API and then the config gets generated that I can copy/backup/use for other purposes

[ShreyasZare]

In this case those classes that do the validation could be abstracted to also work when loading the config files

With such a design, if there is validation failure then the server fails to start. This is what the current design is trying to prevent by discouraging manual config edits.

My worker around right now is to load a scratch instance, inject everything through the API and then grab the generated config files. It's an init container that runs, a python script runs everything in a yaml file through the API and then the config gets generated that I can copy/backup/use for other purposes

If you can describe your use-case in details then it will help me understand it better so that I can think of some solution to make it easier.

[cob-web-corner]

With such a design, if there is validation failure then the server fails to start. This is what the current design is trying to prevent by discouraging manual config edits.

Such is the case with any software really, but you cannot guard rail the stove because someone might burn their hand

If you can describe your use-case in details then it will help me understand it better so that I can think of some solution to make it easier.

In general the use case is deploying a ready to go system with it's configuration, basically IaC with provisioning after. Think dynamic failovers not from backups, deploying the server using pipelines, running it in kubernetes, etc

[ShreyasZare]

With such a design, if there is validation failure then the server fails to start. This is what the current design is trying to prevent by discouraging manual config edits.

Such is the case with any software really, but you cannot guard rail the stove because someone might burn their hand

That's true. The concern here was due to DNS being critical service, when it fails, it affects the entire system. So, this was to make sure that the DNS server always starts with correct config.

If you can describe your use-case in details then it will help me understand it better so that I can think of some solution to make it easier.

In general the use case is deploying a ready to go system with it's configuration, basically IaC with provisioning after. Think dynamic failovers not from backups, deploying the server using pipelines, running it in kubernetes, etc

Thanks for the description. Will see if something can be done to make it better.

[cob-web-corner]

With such a design, if there is validation failure then the server fails to start. This is what the current design is trying to prevent by discouraging manual config edits.

Such is the case with any software really, but you cannot guard rail the stove because someone might burn their hand

That's true. The concern here was due to DNS being critical service, when it fails, it affects the entire system. So, this was to make sure that the DNS server always starts with correct config.

I understand the idea

If you can describe your use-case in details then it will help me understand it better so that I can think of some solution to make it easier.

In general the use case is deploying a ready to go system with it's configuration, basically IaC with provisioning after. Think dynamic failovers not from backups, deploying the server using pipelines, running it in kubernetes, etc

Thanks for the description. Will see if something can be done to make it better.

Alright thanks for considering it.

An outside the box idea would be a terraform provider, but that's a whole project/effort on its own. Terraform providers basically wrap existing APIs

[IngmarStein]

I'd like to add one more argument in favor of human-readable config files: many people like to put configuration files under version control (e.g. as an audit trail, for changelogs, or easy rollbacks). This is arguably a lot more useful with human-readable diffs.

[timhae]

I realize it would probably take quite some time to implement this so I have a suggestion: what do you think of a "sidecar application" that translates a config file into api calls? This would give you the flexibility to fine tune the config format and make changes under the hood that implement the config file one setting at a time. This could also live in this repo and can be written in any language, also c# if you prefer that.

[ShreyasZare]

I realize it would probably take quite some time to implement this so I have a suggestion: what do you think of a "sidecar application" that translates a config file into api calls? This would give you the flexibility to fine tune the config format and make changes under the hood that implement the config file one setting at a time. This could also live in this repo and can be written in any language, also c# if you prefer that.

Thanks for the suggestion. Such a tool would be redundant. This is since you can just install the DNS server on any laptop, use the GUI to set a desired config and then use the Backup Settings option to export it for use.

[timhae]

The purpose of that would be to allow you not having to do any configuration through the webui but instead have a potentially scm tracked file as mentioned earlier in this issue and other issues regarding configuration files. If you aren't interested, please let me know then I will just implement that downstream in nixpkgs/nixos

[ShreyasZare]

The purpose of that would be to allow you not having to do any configuration through the webui but instead have a potentially scm tracked file as mentioned earlier in this issue and other issues regarding configuration files.

If you are looking for some kind of text config to API call then the API can be updated to accept the settings in json format, the same format the get settings api returns. You can then keep the json locally and track it.

If you aren't interested, please let me know then I will just implement that downstream in nixpkgs/nixos

I can update the API as described above when I have some time available. Creating and maintaining a set of separate tools for the conversion will not be feasible for me.

[timhae]

You mean this endpoint? If it would accept json instead of parameters that would get me 90% there. If you want I can also prepare a PR. Thanks for your time and effort :)

[ShreyasZare]

You mean this endpoint? If it would accept json instead of parameters that would get me 90% there. If you want I can also prepare a PR. Thanks for your time and effort :)

Yes, the Set DNS Settings can be changed to accept json for POST requests. The json can be exact same format as that in the Get DNS Settings call.