Downtime not scheduled for services on child hosts

[mdetrano]

Using the web interface to schedule downtime for a host, if I select the option to schedule downtime for "child" hosts (dependent hosts), and check the box to include all services, only the services for the "parent" host have the downtime set.

Expected Behavior

All services for all child hosts also have downtime programmed, when the "All Services" box is checked.

Current Behavior

The child hosts themselves will have the downtime, but not their services.

Possible Solution

Steps to Reproduce (for bugs)

1. Setup a host with dependency configured for another host.
2. Select the "parent" host in the web interface, click Schedule Downtime link.
3. Check box for "All services"
4. Select one of the options to schedule downtime for all child hosts in the "Child Hosts" dropdown
5. Click Schedule Downtime and check the new Downtime objects listed in the "Downtimes" overview. Services on the "child" hosts will not be present.

Context

When scheduling downtime for network devices, it can be time-consuming to select all the hosts beyond that device in order to ensure that their services are in downtime during the maintenance period.

Your Environment

• Icinga Web 2 version and modules (System - About): 2.4.1
• Version used (icinga2 --version): r2.7.0-1
• Operating System and version: CentOS 7
• Enabled features (icinga2 feature list): api checker command ido-mysql livestatus mainlog
• Config validation (icinga2 daemon -C):
• If you run multiple Icinga 2 instances, the zones.conf file (or icinga2 object list --type Endpoint and icinga2 object list --type Zone) from all affected nodes.

[lippserd]

Hi,

Thanks for the report. I think we'll implement this feature but I can't tell you when.

Best regards,
Eric

[mdetrano]

Just a note...

I created a Pull Request to the Icinga2 project, which, if accepted, would help implement this on the Icinga2 side:

Icinga/icinga2#6244

The patch extends the API and External commands for scheduling downtime a bit, making it easy to include services on the host, and dependent hosts as well.

[xenuser]

@lippserd This bug still seems to be open with Icinga2 version 2.12.5-1. We will try to double check this, though. Do you know if in the meantime someone worked on this issue or if someone still has this on his radar?

In the mean time, I would like to suggest that the official docs (e.g. here https://icinga.com/docs/icinga-2/latest/doc/12-icinga2-api/#schedule-host-downtimes-with-all-services ) are updated and that it is mentioned that downtimes will not be set for services on child hosts.

[xenuser]

@lippserd It looks like we can reproduce this with current Icinga2 2.12.x versions, but no longer with r2.13.1-1.
Could you please confirm that someone fixed this (maybe on accident)?
Could you also show us the relevant commit?
All I find is this one here:
Icinga/icinga2@07145d2

[nilmerg]

The current behavior described in the OP didn't change. (Not with my Icinga 2 master build from three weeks ago at least)

The only thing that changed in this regard and what you've linked there is that child downtimes (the ones resulting from checking the All services option) now get removed automatically when their parent downtime is removed.

[NicolasGoeddel]

I did some investigation on 2.13.1-1:
I only have host-to-host dependencies in my demo setup like this:

icinga2-agent01
  |- icinga2-agent01-vm01
  |    |- icinga2-agent01-sub01
  |- icinga2-agent01-vm02

Then I make an API call with type=Host, all_services=true and child_options=DowntimeNonTriggeredChildren.
In my opinion the schedule-downtime action should now set a downtime for all of these hosts:

• icinga2-agent01
• icinga2-agent01-vm01
• icinga2-agent01-vm01-sub01
• icinga2-agent01-vm02

and all of these services:

• icinga2-agent01!disk /
• icinga2-agent01!disk
• icinga2-agent01!ssh
• icinga2-agent01!ping4
• icinga2-agent01-vm01!ping4
• icinga2-agent01-vm01!ssh
• icinga2-agent01-vm01-sub01!ssh
• icinga2-agent01-vm01-sub01!ping4
• icinga2-agent01-vm02!ping4
• icinga2-agent01-vm02!ssh

Instead the API returns this result:

[
  {
    "child_downtimes": [
      {
        "legacy_id": 6,
        "name": "icinga2-agent01-vm02!b157ce87-d208-4ab0-9602-a97945698952",
        "service_downtimes": [
          {
            "legacy_id": 7,
            "name": "icinga2-agent01!disk!d944cee0-db19-4b98-8099-2eab6d334569"
          },
          {
            "legacy_id": 8,
            "name": "icinga2-agent01!disk /!189a93f1-6ad2-45b5-ba0e-60f5457c354c"
          },
          {
            "legacy_id": 9,
            "name": "icinga2-agent01!ping4!88eca130-f32f-496c-993f-cebc4e866816"
          },
          {
            "legacy_id": 10,
            "name": "icinga2-agent01!ssh!22ab11c1-204e-440e-b07e-61e8453c8285"
          }
        ]
      },
      {
        "legacy_id": 11,
        "name": "icinga2-agent01-vm01!9d0f16b2-d60f-4fa5-b05e-2794d53cfcdf",
        "service_downtimes": [
          {
            "legacy_id": 12,
            "name": "icinga2-agent01!disk!891e91c3-15f3-49a2-8bd5-d7cdc99046a9"
          },
          {
            "legacy_id": 13,
            "name": "icinga2-agent01!disk /!f698b51f-b2a5-46f1-9467-ffa189f33877"
          },
          {
            "legacy_id": 14,
            "name": "icinga2-agent01!ping4!9a658829-24b1-4ef5-acaf-d8fe2a46926d"
          },
          {
            "legacy_id": 15,
            "name": "icinga2-agent01!ssh!2f6e47c0-9e5a-4cc1-a2e0-206a42e3e41d"
          }
        ]
      },
      {
        "legacy_id": 16,
        "name": "icinga2-agent01-vm01-sub01!0053f54d-b01c-43a2-b169-8d647ef01068",
        "service_downtimes": [
          {
            "legacy_id": 17,
            "name": "icinga2-agent01!disk!6d7c72a4-6ac3-47aa-b199-3a977c46d250"
          },
          {
            "legacy_id": 18,
            "name": "icinga2-agent01!disk /!6c79d0a5-d211-4a40-9f1b-3a2a23e56538"
          },
          {
            "legacy_id": 19,
            "name": "icinga2-agent01!ping4!307e8fae-8772-4030-8809-f83a49c8827c"
          },
          {
            "legacy_id": 20,
            "name": "icinga2-agent01!ssh!3bc3b47c-0bc1-4134-b3aa-149c9bf4d56e"
          }
        ]
      }
    ],
    "code": 200,
    "legacy_id": 1,
    "name": "icinga2-agent01!b03116e8-29b1-41f0-9103-60f2df5fb02b",
    "service_downtimes": [
      {
        "legacy_id": 2,
        "name": "icinga2-agent01!disk!ca1a1003-8b89-490c-82e4-636910bb1c72"
      },
      {
        "legacy_id": 3,
        "name": "icinga2-agent01!disk /!a9502669-068f-4a1b-b925-fbd0dbac048b"
      },
      {
        "legacy_id": 4,
        "name": "icinga2-agent01!ping4!756af82c-de20-43aa-a281-cc59debfd436"
      },
      {
        "legacy_id": 5,
        "name": "icinga2-agent01!ssh!da652009-a16f-4521-948e-1f5cecb57f83"
      }
    ],
    "status": "Successfully scheduled downtime \"icinga2-agent01!b03116e8-29b1-41f0-9103-60f2df5fb02b\" for object \"icinga2-agent01\"."
  }
]

As you can see it creates a downtime for every host, but on the service level it creates duplicated downtimes. Here is a more easy to read list:

• icinga2-agent01!b03116e8-29b1-41f0-9103-60f2df5fb02b (active)
• icinga2-agent01-vm01!9d0f16b2-d60f-4fa5-b05e-2794d53cfcdf (active)
• icinga2-agent01-vm01-sub01!0053f54d-b01c-43a2-b169-8d647ef01068 (active)
• icinga2-agent01-vm02!b157ce87-d208-4ab0-9602-a97945698952 (active)
• icinga2-agent01!disk /!a9502669-068f-4a1b-b925-fbd0dbac048b (active)
• icinga2-agent01!disk /!189a93f1-6ad2-45b5-ba0e-60f5457c354c (active)
• icinga2-agent01!disk /!f698b51f-b2a5-46f1-9467-ffa189f33877 (active)
• icinga2-agent01!disk /!6c79d0a5-d211-4a40-9f1b-3a2a23e56538 (active)
• icinga2-agent01!disk!ca1a1003-8b89-490c-82e4-636910bb1c72 (active)
• icinga2-agent01!disk!d944cee0-db19-4b98-8099-2eab6d334569 (active)
• icinga2-agent01!disk!891e91c3-15f3-49a2-8bd5-d7cdc99046a9 (active)
• icinga2-agent01!disk!6d7c72a4-6ac3-47aa-b199-3a977c46d250 (active)
• icinga2-agent01!ssh!da652009-a16f-4521-948e-1f5cecb57f83 (active)
• icinga2-agent01!ssh!22ab11c1-204e-440e-b07e-61e8453c8285 (active)
• icinga2-agent01!ssh!2f6e47c0-9e5a-4cc1-a2e0-206a42e3e41d (active)
• icinga2-agent01!ssh!3bc3b47c-0bc1-4134-b3aa-149c9bf4d56e (active)
• icinga2-agent01!ping4!756af82c-de20-43aa-a281-cc59debfd436 (active)
• icinga2-agent01!ping4!88eca130-f32f-496c-993f-cebc4e866816 (active)
• icinga2-agent01!ping4!9a658829-24b1-4ef5-acaf-d8fe2a46926d (active)
• icinga2-agent01!ping4!307e8fae-8772-4030-8809-f83a49c8827c (active)

The services of the child hosts are not listed here, instead every service of the parent host got set a downtime several times. I don't know why it is 4 times, maybe because there are 4 hosts involved in total? Maybe a for-loop issue somewhere in the API code? I am just guessing.

[nilmerg]

The services of the child hosts are not listed here

That's what this issue is about.

instead every service of the parent host got set a downtime several times.

That's a totally different issue, please open a bug over at the icinga2 tracker.

[NicolasGoeddel]

Here is your fix for both of the issues:

https://github.com/Icinga/icinga2/blob/95cdc00ad4e1ab788794322c92c54cd3f5c8c2bf/lib/icinga/apiactions.cpp#L461

Just change the line to this:

				for (const Service::Ptr& hostService : child->GetServices()) {
					Log(LogNotice, "ApiActions")
						<< "Creating downtime for service " << hostService->GetName() << " on child host " << child->GetName();

Currently you are iterating over the services of host. But you need to iterate over the services of every child. I guess this is a copy/paste error from line 407.

After that change there should be no redundant downtimes anymore.

[bobapple]

ref/IP/35773

[julianbrost]

I think that this ticket can be closed here as this is now covered by Icinga/icinga2#8989 and will be handled there. When this ticket was created, scheduling for all services was handled by Icinga Web 2 as Icinga 2 did not provide that feature at all, now it does but has a bug that will be fixed over there.