Add CNAME followage in Recursor#1288
Conversation
2450985 to
5018a87
Compare
68c5e67 to
99e8759
Compare
I will be clearer to say that the answer section of the response does not contain any record with the same type as the query type. And that the answer section contains one CNAME with the same owner name as the query name. It may contain additional CNAME records of all CNAME records form a single CNAME chain.
Not multipel CNAME records with the same owner name.
What does that mean? |
99e8759 to
6a32b4c
Compare
Sure, I will make an update to provide better wording according to your suggestions. |
share/profile.json
Outdated
| "CNAME_LOOP_INNER" : "DEBUG", | ||
| "CNAME_LOOP_OUTER" : "DEBUG", | ||
| "CNAME_MULTIPLE_FOR_NAME" : "DEBUG", |
There was a problem hiding this comment.
Can these three be defined or described in Recursor.pm?
There was a problem hiding this comment.
What do you mean by defined/described? Do you mean adding descriptive message ids such as in Zonemaster::Engine::Translator, or expanding the POD documentation of the associated function that output them (i.e. Zonemaster::Engine::Recursor::recurse())?
There was a problem hiding this comment.
I do not think it is necessary to have a message ID on those, which would put a burden on the translators. Definition or description in POD would be fine.
6a32b4c to
2cab9b6
Compare
Done. Description of this PR and commit's log message are now updated. |
|
Zone files: |
|
What is the purpose of mixing case in and others? |
Generally it is to test case-insensitivity of the implementation. But here it is also about testing that several CNAME records with the same owner name (with different cases) lead to an error, i.e. that the implementation outputs |
Then, shouldn't there be a test where the two CNAME records have identical owner names in a case sensitive comparison. too? |
t/recursor-A.t
Outdated
| # Good CNAMEs | ||
| my $p = Zonemaster::Engine->recurse( 'alias.good-cname.recursor.xa' ); | ||
| isa_ok( $p, 'Zonemaster::Engine::Packet' ); | ||
| is( scalar( $p->answer ), 1, 'one answer record' ); |
There was a problem hiding this comment.
I suggest "one DNS record in answer section" if that is what it means.
t/recursor-A.t
Outdated
| $p = Zonemaster::Engine->recurse( 'chain.good-cname.recursor.xa' ); | ||
| isa_ok( $p, 'Zonemaster::Engine::Packet' ); | ||
| is( scalar( $p->answer ), 1, 'one answer record' ); | ||
| is( name( ($p->answer)[0]->owner ), 'target.good-cname.recursor.xa', 'RR name ok' ); |
There was a problem hiding this comment.
The answer section will contain four DNS records, as far as I can see:
;; ANSWER SECTION:
chain.good-cname.recursor.core.xa. 3600 IN CNAME chain0.good-cname.recursor.core.xa.
chain0.good-cname.recursor.core.xa. 3600 IN CNAME chain1.good-cname.recursor.core.xa.
chain1.good-cname.recursor.core.xa. 3600 IN CNAME target.good-cname.recursor.core.xa.
target.good-cname.recursor.core.xa. 3600 IN A 127.0.0.1
There was a problem hiding this comment.
This is indeed the answer section of the first response, on A query for name chain.good-cname.recursor.xa.. But the $p object only contains the final response, which is, after CNAME followage, from the A query of the CNAME target (target.good-cname.recursor.xa.):
$ perl -MZonemaster::Engine -E 'Zonemaster::Engine::Recursor->remove_fake_addresses("."); Zonemaster::Engine->add_fake_delegation( ( "." => { ibns01.labs.prive.nic.fr => [ "10.1.72.23" ] } ) ); my $p = Zonemaster::Engine->recurse("chain.good-cname.recursor.xa", "A"); for my $entry ( @{ Zonemaster::Engine->logger->entries } ) { say "\n", $entry if index($entry->tag, "CNAME") != -1 }; say $p->string;'
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 57427
;; flags: qr aa ; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; target.good-cname.recursor.xa. IN A
;; ANSWER SECTION:
target.good-cname.recursor.xa. 3600 IN A 192.0.2.1
;; AUTHORITY SECTION:
;; ADDITIONAL SECTION:
;; Query time: 17 msec
;; SERVER: 10.1.72.23
;; WHEN: Mon Nov 6 11:57:35 2023
;; MSG SIZE rcvd: 63
There was a problem hiding this comment.
So if there are three A records in target.good-cname.recursor.xa then that would match is( scalar( $p->answer ), 3, 'three answer records' );
There was a problem hiding this comment.
Yes. But note that more generally scalar( $p->answer ) will count the number of (any) ressource records in the answer section of a packet. So if there are RRs with owner name different than QNAME they would be counted too. If you only want to count RRs with specific owner names or type, then you can use scalar( $p->get_records_for_name('A', 'target.good-cname.recursor.xa', 'answer')).
But for test zones, since you write the zone's content, I guess scalar( $p->answer ) is fine.
There was a problem hiding this comment.
I get two records in the response and the unit test fails:
$ perl -MZonemaster::Engine -E 'Zonemaster::Engine::Recursor->remove_fake_addresses("."); Zonemaster::Engine->add_fake_delegation( ( "." => { ns1 => [ "127.1.0.1" ] } ) ); my $p = Zonemaster::Engine->recurse("good-cname-1.cname.recursor.engine.xa", "A"); for my $entry ( @{ Zonemaster::Engine->logger->entries } ) { say "\n", $entry if index($entry->tag, "CNAME") != -1 }; say $p->string;'
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 45265
;; flags: qr aa ; QUERY: 1, ANSWER: 2, AUTHORITY: 1, ADDITIONAL: 0
;; QUESTION SECTION:
;; good-cname-1.cname.recursor.engine.xa. IN A
;; ANSWER SECTION:
good-cname-1.cname.recursor.engine.xa. 3600 IN CNAME good-cname-1-target.cname.recursor.engine.xa.
good-cname-1-target.cname.recursor.engine.xa. 3600 IN A 127.0.0.1
;; AUTHORITY SECTION:
cname.recursor.engine.xa. 3600 IN NS ns1.cname.recursor.engine.xa.
;; ADDITIONAL SECTION:
;; Query time: 0 msec
;; EDNS: version 0; flags: ; udp: 512
;; SERVER: fda1:b2:c3:0:127:30:1:31
;; WHEN: Wed Nov 15 09:40:41 2023
;; MSG SIZE rcvd: 287
|
@tgreenx, How should the unit tests react on NODATA and NXDOMAIN responses, respectively? NODATA: NXDOMAIN: And how will the code handle broken chain (and here with a duplicate CNAME record)? broken-chain.recursor.core.xa. --> broken-chain2.recursor.core.xa. --> broken-chain3.recursor.core.xa. --> NULL NULL --> broken-chain-target.recursor.core.xa. 100 IN A 127.0.0.1 |
|
@tgreenx, could we complete this together with Test zones for the CNAME function in Recursor.pm in Engine? |
Yes this should be finished in time. |
This commit makes Engine able to follow CNAMEs when doing recursive lookups. Currently CNAMEs will be followed when all of the following are true: - the response has RCODE "NoError" - the answer section of the response does not contain records of the queried type, but does contain at least one CNAME record for the query name - the answer section of the response does not contain multiple CNAME records with the same owner name - the final target of the CNAME record(s) chain has not been followed before - there are no records of the queried type with owner name as the final target of the CNAME record(s) Three system, debug level messages are created: 'CNAME_LOOP_INNER', 'CNAME_LOOP_OUTER' and 'CNAME_MULTIPLE_FOR_NAME'. Some test cases have been modified to account for this new behavior. Unitary tests have also been updated.
- Move CNAME resolution to a dedicated internal method 'Zonemaster::Engine::Recursor::_resolve_cname()' - Various refactoring (renaming of variables, removal of unneeded code, etc) - Update Test Cases code that relates to CNAME - Add documentation for 'Zonemaster::Engine::Recursor::_resolve_cname()' and 'Zonemaster::Engine::Recursor::_recurse()' - Update unit tests and unit tests data
- Add constants CNAME_MAX_RECORDS and CNAME_MAX_CHAIN_LENGTH - Add message tags CNAME_START, CNAME_RECORDS_TOO_MANY, CNAME_RECORDS_CHAIN_BROKEN, CNAME_CHAIN_TOO_LONG, CNAME_FOLLOWED_IB, CNAME_FOLLOWED_OOB, CNAME_NO_MATCH - Rename message tag CNAME_MULTIPLE_FOR_NAME to CNAME_RECORDS_MULTIPLE_FOR_NAME - Add stopping conditions based on CNAME_MAX_RECORDS and CNAME_MAX_CHAIN_LENGTH - Check that CNAME target is out of zone before making a new recursive lookup for that name - Document further Zonemaster::Engine::Recursor::_recurse() - Update unit tests
- Lower value of constant CNAME_MAX_RECORDS from 10 to 9 - Remove duplicates CNAME RRs - Add message tag CNAME_RECORDS_DUPLICATES - Adjust logging level of some message tags - Refactoring - Update documentation - Update unit tests
- Rename CNAME_FOLLOWED_IB to CNAME_FOLLOWED_IN_ZONE and CNAME_FOLLOWED_OOB to CNAME_FOLLOWED_OUT_OF_ZONE - Update documentation
5756f49 to
2cff836
Compare
|
@matsduf please re-review. I addressed your latest comment, and rebased on latest develop. |
lib/Zonemaster/Engine/Recursor.pm
Outdated
| # Remove duplicate CNAME RRs | ||
| my ( %duplicate_cname_rrs, @original_rrs ); | ||
| for my $rr ( @cname_rrs ) { | ||
| my $rr_hash = $rr->class . '/' . $rr->ttl . '/CNAME/' . $rr->owner . '/' . $rr->cname; |
There was a problem hiding this comment.
Suppose a response contains two identical CNAME records that only differ by TTL. Should we consider them as duplicates? I’d say yes.
And how about two CNAME records that only differ by case in the owner name, CNAME name, or both? I think they should be considered duplicate too.
lib/Zonemaster/Engine/Recursor.pm
Outdated
| } | ||
|
|
||
| # CNAME owner name is target, or target has already been seen in this response, or owner name cannot be a target | ||
| if ( lc( $rr_owner ) eq lc( $rr_target ) or exists $seen_targets{lc( $rr_target )} or grep( /^$rr_target$/, keys %forbidden_targets ) ) { |
There was a problem hiding this comment.
The grep can have odd side-effects if $rr_target contains names with special characters in them. It was also missing a conversion to lowercase. It’s better to do:
grep { lc( $_ ) eq $rr_target } ( keys %forbidden_targets )There was a problem hiding this comment.
I think you meant grep { $_ eq lc( $rr_target ) } ( keys %forbidden_targets ) instead.
Fixed.
|
|
||
| $seen_targets{lc( $rr_target )} = 1; | ||
| $forbidden_targets{lc( $rr_owner )} = 1; | ||
| $cnames{$rr_owner} = $rr_target; |
There was a problem hiding this comment.
We should do case-insensitive comparisons when checking whether we’ve already queried the name before (and thus, detect loops). But we should also be case-preserving: if foo.example is an alias to BAR.example, we should query BAR.example, not bar.example, even though both of these QNAMEs should in theory get us the same RRset.
|
Is case preservation really needed here? I cannot see that is wrong to query for |
RFC 1034 does, in fact, state that:
My understanding is that when you “receive” a CNAME, if the resolver decides to follow it, the follow-up query’s QNAME should retain the case exactly as it was received in the previous response. So in a nutshell: it’s better to preserve case here. If not, I feel like we are adding an assumption to |
- Omit TTL and names case in resource record duplicate comparison - Fix condition
tgreenx
left a comment
There was a problem hiding this comment.
@marc-vanderwal @matsduf @mattias-p Comments have been addressed, please re-review.
lib/Zonemaster/Engine/Recursor.pm
Outdated
| # Remove duplicate CNAME RRs | ||
| my ( %duplicate_cname_rrs, @original_rrs ); | ||
| for my $rr ( @cname_rrs ) { | ||
| my $rr_hash = $rr->class . '/' . $rr->ttl . '/CNAME/' . $rr->owner . '/' . $rr->cname; |
lib/Zonemaster/Engine/Recursor.pm
Outdated
| } | ||
|
|
||
| # CNAME owner name is target, or target has already been seen in this response, or owner name cannot be a target | ||
| if ( lc( $rr_owner ) eq lc( $rr_target ) or exists $seen_targets{lc( $rr_target )} or grep( /^$rr_target$/, keys %forbidden_targets ) ) { |
There was a problem hiding this comment.
I think you meant grep { $_ eq lc( $rr_target ) } ( keys %forbidden_targets ) instead.
Fixed.
|
I think this can be merged now. |
|
Release testing for 2024.2: no issues found. |
Purpose
This PR makes the recursive lookup functionality of Zonemaster able to follow CNAME redirections.
Attempts to follow CNAMEs will be made when all of the following are true:
Context
Necessary for #1257 (#1257 (comment))
Test Zones specification: zonemaster/zonemaster#1220
Changes
lib/Zonemaster/Engine/Recursor.pm)CNAME_START,CNAME_RECORDS_DUPLICATES,CNAME_CHAIN_TOO_LONG,CNAME_LOOP_INNER,CNAME_LOOP_OUTER,CNAME_NO_MATCH,CNAME_RECORDS_CHAIN_BROKEN,CNAME_MULTIPLE_FOR_NAMEandCNAME_RECORDS_TOO_MANYCNAME_MAX_CHAIN_LENGTHandCNAME_MAX_RECORDSHow to test this PR
Unit tests are updated and should pass (based on zonemaster/zonemaster#1220).
Manual testing:
You can can test any zone by providing it to
Zonemaster::Engine->recurse(), e.g. with any of the zones from zonemaster/zonemaster#1220. See below: