auth: domain transactions

[miodvallat]

Short description

It has been noticed for a while, that the current transaction model used in the authoritative server only covers records (and comments). Which means that attempts to create and populate a domain from invalid data would first, create an empty domain, then start a transaction to fill it, then abort it due to the data being invalid, leaving an empty domain.

This PR tries to make the domain creation part of the transaction, so that aborting it will correctly leave no trace of the new domain.

It is better reviewed on a per-commit basis.

The reason why domains are not part of transactions, is that transactions are tied to a backend domain id, which can't be known unless the domain exist. The first commits perform plumbing, first to make domain creation calls return a filled DomainInfo struct (i.e. perform an immediate getDomainInfo call which would have been issued by the caller soon anyway).

Then we can add the ability to these create domain routines to start a transaction at domain creation time as well. This depends upon backend support, which is why a new capability, CAP_DOMAIN_TRANSACTION, is added to report that ability.

At the moment only the SQL backends and the Bind backend (in secondary mode) support this. I am still wrestling with the LMDB backend, with no success yet.

Checklist

I have:

• read the CONTRIBUTING.md document
• read and accepted the Developer Certificate of Origin document, including the AI Policy, and added a "Signed-off-by" to my commits
• compiled this code
• tested this code
• included documentation (including possible behaviour changes)
• documented the code
• added or modified regression test(s)
• added or modified unit test(s)

[coveralls]

Coverage Report for CI Build 30531048200

Warning

Build has drifted: This PR's base is out of sync with its target branch, so coverage data may include unrelated changes.
Quick fix: rebase this PR. Learn more →

Coverage decreased (-0.02%) to 71.177%

Details

• Coverage decreased (-0.02%) from the base build.
• Patch coverage: 128 uncovered changes across 6 files (121 of 249 lines covered, 48.59%).
• 7162 coverage regressions across 85 files.

Uncovered Changes

File	Changed	Covered	%
modules/bindbackend/bindbackend2.cc	153	49	32.03%
modules/remotebackend/remotebackend.cc	10	3	30.0%
modules/lmdbbackend/lmdbbackend.cc	28	22	78.57%
pdns/backends/gsql/gsqlbackend.cc	26	20	76.92%
pdns/ws-auth.cc	13	10	76.92%
pdns/auth-secondarycommunicator.cc	10	8	80.0%
Total (13 files)	249	121	48.59%

Coverage Regressions

7162 previously-covered lines in 85 files lost coverage.

Top 10 Files by Coverage Loss	Lines Losing Coverage	Coverage
pdns/recursordist/pdns_recursor.cc	1107	46.21%
pdns/recursordist/lua-recursor4.cc	681	8.69%
pdns/recursordist/rec-main.cc	581	50.85%
pdns/recursordist/rec-tcp.cc	450	35.62%
pdns/recursordist/rec_channel_rec.cc	407	19.82%
pdns/recursordist/rec-lua-conf.cc	352	19.84%
pdns/recursordist/lwres.cc	349	38.03%
pdns/recursordist/syncres.cc	346	77.71%
pdns/recursordist/rec-rust-lib/cxxsupport.cc	343	58.03%
pdns/recursordist/rpzloader.cc	291	43.45%

---

Coverage Stats

Relevant Lines:	173065
Covered Lines:	135049
Line Coverage:	78.03%
Relevant Branches:	82800
Covered Branches:	47068
Branch Coverage:	56.85%
Branches in Coverage %:	Yes
Coverage Strength:	6935037.41 hits per line

---

💛 - Coveralls

[miodvallat]

While there is still hope LMDB can benefit from this, this requires quite a lot of work in its bowels, so if it eventually happens, this will be in a different PR.

Which means this PR can be considered complete (except for bugs) and ready for review now.

[miodvallat]

For the record (might benefit our future selves), regarding the LMDB backend:

Domain transactions can't be done in the current state of things.

LMDB requires that, at any time, there is only one active write transaction. That transaction is associated to a unique MDB_env. Every MDB_env in turn is associated to a single file.

In order to be able to make domain transactions work, we would thus need domains and records to be stored in the same file. Which isn't the case, as domains are stored in the main file, while records are split across the various shard files.

But then, LMDB also allows to nest transactions, so that they can be commited (or aborted) in the reverse order of their creation; so we can theoretically implement domain transactions by making the "records" transaction used to fill the domain contents, a child of the "domain" transaction creating it in the first place.

This just requires heavy lifting of the code, which is probably better done in a separate PR while this one gets attention, dust settles, etc.