Slack notifications on workflows failures

[czeslavo]

Problem Statement

Currently, it's possible we will overlook regular workflows failures if they're not part of the CI pipeline (e.g. build.yaml that happens right after merging to main and nightly.yaml that builds nightly images every night).

The goal of this issue is to setup Slack notifications (or anything else that would ping the team) if the mentioned workflows are failing.

Proposed Solution

On every failure of the mentioned workflows, post a message to team-k8s-notifications Slack channel.

Acceptance Criteria

• When build.yaml or nightly.yaml workflow fails, the team gets notified via some agreed channel