Skip to content

Commit

Permalink
Document task launching in detail
Browse files Browse the repository at this point in the history
  • Loading branch information
ezr-ondrej committed Nov 16, 2021
1 parent b32a0a1 commit b3f7d20
Show file tree
Hide file tree
Showing 6 changed files with 179 additions and 26 deletions.
39 changes: 13 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,29 +3,14 @@
A plugin into Foreman's Smart Proxy for running Dynflow actions on the Smart
Proxy.

## Split architecture
## Public API

This repository contains two gems, `smart_proxy_dynflow` and
`smart_proxy_dynflow_core`.

### smart_proxy_dynflow

A Smart Proxy plugin which allows to run Dynflow actions and provides a simple
API for triggering actions and querying information about execution plans.

### smart_proxy_dynflow_core

Historically this gem could be used to run as a standalone service or as a part
of the Smart Proxy process. This is not possible starting with
`smart_proxy_dynflow_core-0.4.0`. The gem is still kept around, but instead of
providing functionality on its own, it depends on `smart_proxy_dynflow` and
exposes certain parts of it under the old names and so it serves as a
compatibility layer.

#### GET /console
### GET /console
Serves the Dynflow console for human friendly task inspection.

#### POST /tasks
### POST /tasks
**Deprecated** it still works, but you should use `POST /tasks/launch`

Used for triggering a task, expects `action_class` and `action_input` in the
request's body. The action specified by `action_class` is then planned with
`action_input` provided to the action's `#plan` method.
Expand Down Expand Up @@ -60,7 +45,7 @@ END
Note: The example above requires `smart_proxy_remote_execution_ssh` Smart Proxy
plugin.

#### POST /tasks/$TASK_ID/cancel
### POST /tasks/$TASK_ID/cancel
Tries to cancel a task.

```
Expand All @@ -71,12 +56,12 @@ curl -X POST localhost:8008/tasks/dd5a8306-0e52-4f68-9e83-c7f51c9e95c3/cancel -d
}
```

#### GET /tasks/$TASK_ID/status
### GET /tasks/$TASK_ID/status
Allows querying the task by its id. Returns the full hash of the execution plan,
for details about output of this API call see `::Dynflow::ExecutionPlan#to_hash` and
`::Dynflow::Action#to_hash`.

#### GET /tasks/count
### GET /tasks/count

Returns the number of tasks. Optionally a state parameter can be provided to
obtain count of tasks in the specified state.
Expand All @@ -98,7 +83,7 @@ curl localhost:8008/tasks/count?state='stopped' 2>/dev/null
```

#### POST /tasks/$TASK_ID/done
### POST /tasks/$TASK_ID/done

Sends an `::ForemanTasksCore::Runner::ExternalEvent` event with full copy of the
parsed request's body to the task's step specified by `step_id`.
Expand All @@ -108,7 +93,7 @@ curl -X POST localhost:8008/tasks/dd5a8306-0e52-4f68-9e83-c7f51c9e95c3/done \
-d '{"step_id": 1, "my_custom_data": "something"}'
```

#### GET /tasks/operations
### GET /tasks/operations

`smart_proxy_dynflow` allows registering `TaskLauncher`s into a registry. A
`TaskLauncher` is an abstraction which defines how to start a suite of execution
Expand All @@ -117,12 +102,14 @@ implementation of the actions and their inputs.

This endpoint returns a list of registered `TaskLauncher`s from the registry.

#### POST /tasks/launch
### POST /tasks/launch

Launches a suite of execution plans to perform an operation. Parameter
`operation` specifies the operation and `input` is an input for task launcher
registered with the operation. `input` is specific to each operation.

More details can be found in [Task Launching docs](developer_docs/task_launching.asciidoc)

# Installation

**Clone smart-proxy**
Expand Down
1 change: 1 addition & 0 deletions developer_docs/architecture/batch_launching.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
15 changes: 15 additions & 0 deletions developer_docs/architecture/diagrams/batch_launching.mmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
sequenceDiagram
SmartProxyAPI->>SmartProxyAPI: /tasks/launch
SmartProxyAPI->>SmartProxyAPI: TaskLauncherRegistry.fetch(params['operation'])
SmartProxyAPI->>+BatchTaskLauncher: launch!(input = params['input'])
participant SingleLauncher
BatchTaskLauncher->>dynflow: trigger(action=BatchAction, input)
dynflow->>+BatchAction: plan(launcher, input)
BatchAction->>BatchTaskLauncher: launch_children(parent_action = self, input)
loop Every child in the input [{child_id => child_input}]
BatchTaskLauncher->>BatchTaskLauncher: SingleLauncher = child_launcher(parent_action)
BatchTaskLauncher->>+SingleLauncher: launch!(child_input = input[foreman_child_plan_id])
SingleLauncher->>dynflow: results = trigger(parent, child_input)
SingleLauncher->>-BatchTaskLauncher: results[foreman_child_plan_id] = results
end
BatchTaskLauncher->>-SmartProxyAPI: results
23 changes: 23 additions & 0 deletions developer_docs/architecture/diagrams/group_launching.mmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
sequenceDiagram
SmartProxyAPI->>SmartProxyAPI: /tasks/launch
SmartProxyAPI->>SmartProxyAPI: CustomGroupLauncher = TaskLauncherRegistry.fetch(params['operation'])
SmartProxyAPI->>+CustomGroupLauncher: launch!(input = params['input'])
participant SingleLauncher
CustomGroupLauncher->>dynflow: trigger(action=SingleRunnerBatch, input)
dynflow->>+SingleRunnerBatch: plan(launcher, input)
SingleRunnerBatch->>CustomGroupLauncher: launch_children(parent_action = self, input)
loop OutputCollector for every child
CustomGroupLauncher->>CustomGroupLauncher: SingleLauncher = child_launcher(parent_action)
CustomGroupLauncher->>+SingleLauncher: launch!(child_input = input[foreman_child_plan_id])
SingleLauncher->>dynflow: results = trigger(action=OutputCollector, parent, child_input)
SingleLauncher->>-CustomGroupLauncher: results[foreman_child_plan_id] = results
end
CustomGroupLauncher->>dynflow: trigger(action=BatchRunner, parent, input)
dynflow->>+BatchRunner: plan(launcher, input)
BatchRunner->>-BatchRunner: initiate_runner()
dynflow->>+BatchRunner: run()
BatchRunner->>BatchRunner: init_run()
BatchRunner->>CustomRunner: start
BatchRunner-->>-dynflow: triggered
dynflow-->CustomGroupLauncher: triggered
CustomGroupLauncher->>-SmartProxyAPI: results
1 change: 1 addition & 0 deletions developer_docs/architecture/group_launching.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
126 changes: 126 additions & 0 deletions developer_docs/task_launching.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
[[task-launching]]
= How the tasks are launched
:imagesdir: architecture/

Launching of the tasks done through `POST /tasks/launch` endpoint.
The previously used endpoint `POST /tasks` is being decomisioned so it is not documented.

For the launch to be succesful, you have to specify `operation` which has a defined launcher
in the `TaskLauncherRegistry`.

There are default launchers registered that can be utilized.
It is useful to understand concepts of these as those are just slighly changed in actuall launchers used.
For simle Tasks these can be used by themself.

[single-launcher]
== Single task triggering

Sample input for single task triggering:

[source, json]
----
{
"operation": "single",
"input": {
"action_class": "MyDynflowActionClass",
"action_input": { "some": "action data" }
}
}
----

[batch-launcher]
== Batch triggering

Batch triggering allows running multiple actions while making only one HTTP API call to the Proxy.

Sample input for batch triggering would be:

[source, json]
----
{
"operation": "batch",
"input": {
"<child_task_caller_ID>": {
"action_class": "MyDynflowActionClass",
"action_input": { "some": "childaction data" }
},
"<child2_task_caller_ID": {
"action_class": "MyDynflowActionClass",
"action_input": { "some": "childaction2 data" }
}
}
}
----

This allows running multiple independent actions, that share same parent action in the underlying dynflow,
but nothing much apart of that.

Following diagram tries to show how this gets processed internally.

[caption="Diagram: batch launching"]
image::batch_launching.svg[Batch launching]


[group-launcher]
== Group triggering

Group triggering is there to run single action on multiple nodes (Hosts).
This has the advantage of having only one runner and safe resources, if the runner actually support running for multiple nodes.

Best example is probably Ansible, but there would be more like that.

First we need to implement and register the actual group launcher as we need to define what runner to use for the group.

[source, ruby]
----
class Runner::CoolRunner < ::Proxy::Dynflow::Runner::Parent
def initialize(input, suspended_action:)
super input, suspended_action: suspended_action
# prepare the stuff from input you need to perform cool things
end
def start
# start doing the cool stuff for the nodes
end
def kill
# stop doing the cool stuff prematurely
end
end
class TaskLauncher::MyCoolLauncher < Proxy::Dynflow::TaskLauncher::AbstractGroup
def operation
'cool-operation'
end
def self.runner_class
Runner::CoolRunner
end
end
Proxy::Dynflow::TaskLauncherRegistry.register('cool-operation',
TaskLauncher::MyCoolLauncher)
----

Then the input for launching our cool-operation would be:

----
{
"operation": "cool-operation",
"input": {
"<child_task_caller_ID>": {
"action_input": { "some": "childaction data" }
},
"<child2_task_caller_ID": {
"action_input": { "some": "childaction2 data" }
}
}
}
----

You can notice that you do not need to define the action, as our Runner defines how to run stuff for every node.

The following diagram can be used to compare the launch with the Batch launching.

[caption="Diagram: group launching"]
image::group_launching.svg[Group launching]

0 comments on commit b3f7d20

Please sign in to comment.