ℹ️ Info: This script can not be used on its own but requires the base installation. See main README for details.
This script sends notifications about host UP and DOWN events. In comparison
to just netwatch (/tool/netwatch) and its up-script and down-script
this script implements a simple state machine and dependency model. Host
down events are triggered only if the host is down for several checks and
optional parent host is not down to avoid false alerts.
Just install the script:
$ScriptInstallUpdate netwatch-notify;
Then add a scheduler to run it periodically:
/system/scheduler/add interval=1m name=netwatch-notify on-event="/system/script/run netwatch-notify;" start-time=startup;
The hosts to be checked have to be added to netwatch with specific comment:
/tool/netwatch/add comment="notify, name=example.com" host=[ :resolve "example.com" ];
Also notification settings are required for e-mail, matrix, ntfy and/or telegram.
It is possible to run an up hook command (up-hook) or down hook command
(down-hook) when a notification is triggered. This has to be added in
comment, note that some characters need extra escaping:
/tool/netwatch/add comment=("notify, name=device, down-hook=/interface/ethernet \\{ disable \\\"en2\\\"; enable \\\"en2\\\"; \\}") host=10.0.0.20;
Also there is a pre-down-hook that fires at two thirds of failed checks
required for the notification. The idea is to fix the issue before a
notification is sent.
Getting the escaping right may be troublesome. Please consider adding a
script in /system/script, then running that from hook.
The count threshold (default is 5 checks) is configurable as well:
/tool/netwatch/add comment="notify, name=example.com, count=10" host=104.18.144.11;
If the host is behind another checked host add a dependency, this will suppress notification if the parent host is down:
/tool/netwatch/add comment="notify, name=gateway" host=93.184.216.1;
/tool/netwatch/add comment="notify, name=example.com, parent=gateway" host=93.184.216.34;
Note that every configured parent in a chain increases the check count threshold by one.
The host address can be updated dynamically. Give extra parameter resolve
with a resolvable name:
/tool/netwatch/add comment="notify, name=example.com, resolve=example.com" host=0.0;
This supports multiple A records for a name just fine, even a CNAME to those. An update happens only if no more record with the configured host address is found.
The address family is preserved, so if you want AAAA records (for IPv6) use this:
/tool/netwatch/add comment="notify, name=example.com, resolve=example.com" host=::;
Also suppressing the notification on host down is possible with parameter
no-down-notification. This may be desired for devices that are usually
powered off, but accessibility is of interest.
/tool/netwatch/add comment="notify, name=printer, no-down-notification" host=10.0.0.30;
Go and get your coffee ☕️ before sending the print job.
A message is writting to log after three failed attemts to resolve a host.
However this can cause some noise for hosts that are expected to have
failures, for example when the name is dynamically added by
dhcp-to-dns. This can be suppressed:
/tool/netwatch/add comment="notify, name=client, resolve=client.dhcp.example.com, no-resolve-fail" host=10.0.0.0;
For some extra information it is possible to add a text note. This is included verbatim into the notification.
/tool/netwatch/add comment="notify, name=example, note=Do not touch!" host=10.0.0.31;
It is possible to add a link in notification, that is added below the formatted notification text.
/tool/netwatch/add comment="notify, name=example.com, resolve=example.com, link=https://example.com/" host=0.0;
Sometimes it is sufficient if one of a number of hosts is available. You can
make netwatch-notify check for that by adding several items with same
name. Note that count has to be multiplied to keep the actual time.
/tool/netwatch/add comment="notify, name=service, count=10" host=10.0.0.10;
/tool/netwatch/add comment="notify, name=service, count=10" host=10.0.0.20;
Sometimes you can not check your gateway for internet connectivity, for
example when it does not respond to pings or has a dynamic address. You could
check 1.1.1.1 (Cloudflare DNS), 9.9.9.9 (Quad-nine DNS), 8.8.8.8
(Google DNS) or any other reliable address that indicates internet
connectivity.
/tool/netwatch/add comment="notify, name=internet" host=1.1.1.1;
A target like this suits well to be parent for other checks.
/tool/netwatch/add comment="notify, name=example.com, parent=internet" host=93.184.216.34;
Having several ISPs for redundancy a failed link may go unnoticed without proper monitoring. You can use routing-mark to monitor specific connections. Create a route and firewall mangle rule.
/routing/table/add fib name=via-isp1;
/ip/route/add distance=1 gateway=isp1 routing-table=via-isp1;
/ip/firewall/mangle/add action=mark-routing chain=output new-routing-mark=via-isp1 dst-address=1.0.0.1 passthrough=yes;
Finally monitor the address with netwatch-notify.
/tool/netwatch/add comment="notify, name=quad-one via isp1" host=1.0.0.1;
Note that all traffic to the given address is routed that way. In case of
link failure this address is not available, so use something reliable but
non-essential. In this example the address 1.0.0.1 is used, the same service
(Cloudflare DNS) is available at 1.1.1.1.
Netwatch entries can be created to work with both - this script and netwatch-dns. Just give options for both:
/tool/netwatch/add comment="doh, notify, name=cloudflare-dns" host=1.1.1.1;