Skip to content

Zonemaster CLI aborts testing immediately on CRITICAL messages. #394

@mattias-p

Description

@mattias-p

When Engine emits a CRITICAL message, CLI used to continue testing unless --stop-level has been explicitly set, resulting in output like this:

$ zonemaster-cli --no-ipv6 --no-time --raw mail.protection.outlook.com
CRITICAL B02_NO_WORKING_NS  domain="mail.protection.outlook.com"
ERROR    B02_UNEXPECTED_RCODE  ns=ns2-proddns.glbdns.protection.outlook.com/104.47.72.81; rcode=FORMERR
ERROR    B02_UNEXPECTED_RCODE  ns=ns1-proddns.glbdns.protection.outlook.com/104.47.34.49; rcode=FORMERR
CRITICAL CANNOT_CONTINUE  domain=mail.protection.outlook.com

However #371 inadvertently changed this behavior so that test case execution terminates immediately if a CRITICAL message is emitted.

CRITICAL B02_NO_WORKING_NS  domain="mail.protection.outlook.com"
Exited early: Saw message at level CRITICAL

The new behavior makes it harder to develop test cases that emit CRITICAL messages.

This was first reported in the PR thread of #389.

Metadata

Metadata

Assignees

No one assigned

    Labels

    T-BugType: Bug in software or error in test case description

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions