There isn't an option to deactivate the notification once one has finished the downtime. Therefore the notification needs to be delete it instead.
One way to fix the issue would be to set an hour for the notification to expire, but it can be a problem with the time stamp in the server. Might be simpler to just had an active/inactive flag that can be changes as needed.