Fix DNSSEC10 for name servers sharing the same IPs

[tgreenx]

Purpose

This PR fixes a bug in DNSSEC10 where DS10_EXPECTED_NSEC_NSEC3_MISSING is wrongly outputted for name servers sharing the same IPs. For this type of name servers it should now also output, for all message tags, all name servers name/IP pairs.

Context

Fixes #1454
Updated test scenarios from zonemaster/zonemaster#1383

Changes

• Update implementation
• Update unit tests and data files

How to test this PR

Unit tests are updated and should pass.

[matsduf]

When testing the scenarios with this PR installed, the new scenarios pass, but is it correct?

$ zonemaster-cli --show-testcase --level INFO --test dnssec10 --hints ../../COMMON/hintfile --raw GOOD-NSEC-2.dnssec10.xa 
   0.00 INFO     Unspecified    GLOBAL_VERSION  version=v7.1.0
   0.07 INFO     DNSSEC10       DS10_HAS_NSEC  ns_list=ns1a.good-nsec-2.dnssec10.xa/127.15.10.31;ns1a.good-nsec-2.dnssec10.xa/fda1:b2:c3:0:127:15:10:31

The zone is delegated to three NS, ns1a, ns1b and ns1c, but only the first one is listed under DS10_HAS_NSEC. I expect all three to be listed.

[tgreenx]

When testing the scenarios with this PR installed, the new scenarios pass, but is it correct?

$ zonemaster-cli --show-testcase --level INFO --test dnssec10 --hints ../../COMMON/hintfile --raw GOOD-NSEC-2.dnssec10.xa 
   0.00 INFO     Unspecified    GLOBAL_VERSION  version=v7.1.0
   0.07 INFO     DNSSEC10       DS10_HAS_NSEC  ns_list=ns1a.good-nsec-2.dnssec10.xa/127.15.10.31;ns1a.good-nsec-2.dnssec10.xa/fda1:b2:c3:0:127:15:10:31

The zone is delegated to three NS, ns1a, ns1b and ns1c, but only the first one is listed under DS10_HAS_NSEC. I expect all three to be listed.

Hmm I don't think its feasible unless we stop skipping the sending queries already queried IPs, i.e. remove line 3159:
if exists $ip_already_processed{$ns->address->short};
And that's the usual way of sending queries in test cases.

[matsduf]

The zone is delegated to three NS, ns1a, ns1b and ns1c, but only the first one is listed under DS10_HAS_NSEC. I expect all three to be listed.

Hmm I don't think its feasible unless we stop skipping the sending queries already queried IPs, i.e. remove line 3159: if exists $ip_already_processed{$ns->address->short}; And that's the usual way of sending queries in test cases.

In the scenario the delegation has three NS with the same IP. As we have seen it can exist, but more likely is that delegation and zone use/refer to the same IP, but with different names. It is less helpful if some names are not mentioned.

Wouldn't the following model work?

1. Collect all NS as NAME/IP from delegation and zone.
2. Normalize NAME and IP, respectively.
3. Create an ARRAY of all unique NAME/IP pairs.
4. Create a HASH of the elements from ARRAY where IP is the key, and the value is an array of the unique NAMEs for that IP.
5. Run the test based on the keys of HASH (IP) and save only IP in the sets.
6. When the messages are created, construct {ns} ({ns_list}) by doing a lookup into HASH and fetch the name. Construct one {ns} for each name for that IP.

[matsduf]

Two more scenarios have been added in zonemaster/zonemaster#1383, GOOD-NSEC-3 and GOOD-NSEC3-2. Both should output as e.g. GOOD-NSEC-1. Just as with GOOD-NSEC-2 and GOOD-NSEC3-2 there are extra names with the same IP, one name in delegation and the other in zone.

[matsduf]

When testing the scenarios with this PR installed, the new scenarios pass, but is it correct?

$ zonemaster-cli --show-testcase --level INFO --test dnssec10 --hints ../../COMMON/hintfile --raw GOOD-NSEC-2.dnssec10.xa 
   0.00 INFO     Unspecified    GLOBAL_VERSION  version=v7.1.0
   0.07 INFO     DNSSEC10       DS10_HAS_NSEC  ns_list=ns1a.good-nsec-2.dnssec10.xa/127.15.10.31;ns1a.good-nsec-2.dnssec10.xa/fda1:b2:c3:0:127:15:10:31

The zone is delegated to three NS, ns1a, ns1b and ns1c, but only the first one is listed under DS10_HAS_NSEC. I expect all three to be listed.

This is a different bug compared to what this PR is to resolve. The bug in this comment will be lifted into an issue instead.

[matsduf]

This is fine.

[tgreenx]

When testing the scenarios with this PR installed, the new scenarios pass, but is it correct?

$ zonemaster-cli --show-testcase --level INFO --test dnssec10 --hints ../../COMMON/hintfile --raw GOOD-NSEC-2.dnssec10.xa 
   0.00 INFO     Unspecified    GLOBAL_VERSION  version=v7.1.0
   0.07 INFO     DNSSEC10       DS10_HAS_NSEC  ns_list=ns1a.good-nsec-2.dnssec10.xa/127.15.10.31;ns1a.good-nsec-2.dnssec10.xa/fda1:b2:c3:0:127:15:10:31

The zone is delegated to three NS, ns1a, ns1b and ns1c, but only the first one is listed under DS10_HAS_NSEC. I expect all three to be listed.

This is a different bug compared to what this PR is to resolve. The bug in this comment will be lifted into an issue instead.

This should now be fixed too with commit 7690d4e :

$ zonemaster-cli --show-testcase --level INFO --test dnssec10 --hints ../zonemaster/test-zone-data/COMMON/hintfile --raw GOOD-NSEC-2.dnssec10.xa
   0.00 INFO     Unspecified    GLOBAL_VERSION  version=v7.1.0
   0.06 INFO     DNSSEC10       DS10_HAS_NSEC  ns_list=ns1a.good-nsec-2.dnssec10.xa/127.15.10.31;ns1a.good-nsec-2.dnssec10.xa/fda1:b2:c3:0:127:15:10:31;ns1b.good-nsec-2.dnssec10.xa/127.15.10.31;ns1b.good-nsec-2.dnssec10.xa/fda1:b2:c3:0:127:15:10:31;ns1c.good-nsec-2.dnssec10.xa/127.15.10.31;ns1c.good-nsec-2.dnssec10.xa/fda1:b2:c3:0:127:15:10:31

[mattias-p]

This looks reasonable to me. I believe I found a slightly simplifying refactoring.

[matsduf]

When testing the scenarios with this PR installed, the new scenarios pass, but is it correct?

$ zonemaster-cli --show-testcase --level INFO --test dnssec10 --hints ../../COMMON/hintfile --raw GOOD-NSEC-2.dnssec10.xa 
   0.00 INFO     Unspecified    GLOBAL_VERSION  version=v7.1.0
   0.07 INFO     DNSSEC10       DS10_HAS_NSEC  ns_list=ns1a.good-nsec-2.dnssec10.xa/127.15.10.31;ns1a.good-nsec-2.dnssec10.xa/fda1:b2:c3:0:127:15:10:31

The zone is delegated to three NS, ns1a, ns1b and ns1c, but only the first one is listed under DS10_HAS_NSEC. I expect all three to be listed.

This is a different bug compared to what this PR is to resolve. The bug in this comment will be lifted into an issue instead.

With 7690d4e I get a correct result with all NS listed.

[tgreenx]

I applied Mattias's suggestions, please re-approve.

[tgreenx]

v2025.1 Release testing

From #1454 (comment):

Ran test (Ubuntu 22.04) Looks fine.

$ zonemaster-cli --show-testcase --level info --test dnssec10 info-slovenija.si

Seconds Level    Testcase       Message
======= ======== ============== =======
   0.00 INFO     Unspecified    Using version v6.0.0 of the Zonemaster engine.
   1.03 INFO     DNSSEC10       The zone has NSEC records. Fetched from the nameservers with IP addresses "193.9.21.61".