[Reporting] Schedule Reports with Task Manager

[tsullivan]

Stakeholder teams in bold

1. Remove ESQueue from the Reporting libs [Reporting]
2. Create TM tasks for the various kinds of scheduled reports, (starting with Canvas PDF) [Reporting]
3. The Reporting screenshot observable needs to be in scope of the task, have an easy interface to run a job, and get run when the task fires [Reporting]
4. Add UI controls to the visualization container to schedule regular reports [App Teams]

Making the screenshot observable more visible should have side benefits of improving testability.

[elasticmachine]

Pinging @elastic/kibana-reporting-services (Team:Reporting Services)

[kobelb]

@elastic/kibana-alerting-services Does the task manager have any type of "per task type" concurrency control? We've been focusing on making task-manager capable of being more concurrent to support Alerting's needs, but if we spawn multiple Reporting jobs per instance of Kibana, we're likely to run out of memory.

[mikecote]

@kobelb Not at this time. There was a way around this with numWorker until it got removed in #45504. It was possible to set numWorker on a task definition to equal the number of max workers and would ensure the task only runs while nothing else is running.

We have an issue for alerting / task manager to properly limit concurrency (#54916) but no efforts started yet.

[tsullivan]

In ESQueue currently, we avoid claiming more than 1 job at a time, and I think we hold off on polling for new jobs while a current job is executing. We might need to find a way to retain that kind of behavior by handling for it in our task run function.

The plan is for Reporting to register only 1 task for all of its reporting needs.

[mikecote]

I believe #54916 is the solution you're looking for. It would allow alerts to still run concurrently while a reporting task definition can limit to only one reporting task run at a time per Kibana instance.

Is such feature what you're looking for? If so, what release is scheduled reports targeting?

[alexfrancoeur]

I don't want to speak for @tsullivan and @joelgriffith but I was hoping we'd be able to deliver scheduled reports (on task manager, and simplified UI creation, management UI's) in 7.x 😄 . We haven't fully defined MVP yet though.

[elasticmachine]

Pinging @elastic/kibana-app-services (Team:AppServices)