[Scheduled Reports] Task data required for scheduling, and mapping additions needed for the historical index #75610
Description
In the Reporting data, there are fields that describe the state of an item in a queue. This allows the index itself to act as a distributed task queue of the items that it stores.
With Reporting moving from the ESQueue sub-module to Task Manager, what impact will that have on the mapping of the fields in the reporting index? What needs to be added and will any need to go away?
Data belonging in Parameters and State of the Task Instance
For the "Create Recurring Report" form, fields of data belongs to Task Manager as the schedule definition, and are all fields of data in the scheduled task instance's parameters.
For the table of schedules these rows will come from a search against the Task Manager system index to find all scheduled tasks that match the reporting:execute task type. The returned set of "task instances" will include params and state for each task.
Parameters come from user input at the time the task was first scheduled.
State comes from the accumulated output of previous task runs.
Both of those data objects are available in the run context available to our task definition's run function.
Task Parameter Data:
- Schedule name (report name)
- URL of the page (PNG/PDF) or saved object ID/search source (CSV)
- File type (PNG/PDF/CSV)
- time range in the time picker
- username
- encrypted request headers, or an API token that allows recurring auth on behalf of the user that created the schedule
- Interval expression (cron)
- Array of actions
Task State Data:
- Last report timestamp
- Last status string
Alternative to using task state for latest status
There is also an Event Log that can preserve information about previous task runs. The state of a task would be replaced at the end of each task run, so it doesn't have any history and wouldn't be able to identify trends, while Event Log could do that.
Data belonging in the Historical Index
In this issue, the Reporting system index is referred to as the historical index, but it is also formerly the queue index that powered ESQueue.
To show the historical listing of reports, we'll continue the pattern of searching against the Reporting system index. That provides a historical listing, and is pretty much what we have today in the list of reports in the Management app.
New fields must be added to support scheduling. Certain parts of the schedule data can be "flattened" into the historical data data, such as the name of the schedule, and recurrence. We will need the task instance ID in this data to refer back to the schedule that created it since editing, pausing, retrying, etc will probably require access to the full params of the task instance.
Already exists for historical reports:
- Saved object
- File type
- File size
- Report date
- Status
- Retry button (using the job params in the document)
- Download
New field for the historical reports:
- Task instance ID
- Schedule name
- Recurrence (cron string | time units)
In the UI of historical reports, there should be a recurrence column showing a human-friendly value of the duration between reports, or the word "Custom". Therefore I propose the internal representation of recurrence in the system is time unit or cron string.