Skip to content

adding dll scanning and loading #717

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Mar 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
193 changes: 193 additions & 0 deletions docs/api/rest/task-manager/argo-template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
<!--
~ Copyright 2022 MONAI Consortium
~
~ Licensed under the Apache License, Version 2.0 (the "License");
~ you may not use this file except in compliance with the License.
~ You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->

# Task Manager - Argo Template API

The template endpoint provides the following APIs to post a new template to Argo plugin of
the Task Manager.


## POST /argo/template

Posts a new template to Argo and returns the resultant template.

### Parameters

body, json string of the wrapped template

### Responses

Response Content Type: JSON

- the actual Argo resultant template.

| Code | Description |
| ---- | ------------------------------- |
| 200 | template posted sucsessfully. |
| 400 | bad json or template post error |
| 500 | Internal service error. |

### Example Request

file saved locally as workflowtemplate.json
```json
{
"Namespace": "argo",
"serverDryRun": false,
"submitOptions": {
"parameters": [
"name=value"
]
},
"template": {
"metadata": {
"name": "fantastic-tiger",
"namespace": "argo",
"labels": {
"example": "true"
}
},
"spec": {
"workflowMetadata": {
"labels": {
"example": "true"
}
},
"entrypoint": "argosay",
"arguments": {
"parameters": [
{
"name": "message",
"value": "hello argo"
}
]
},
"templates": [
{
"name": "argosay",
"inputs": {
"parameters": [
{
"name": "message",
"value": "{{workflow.parameters.message}}"
}
]
},
"container": {
"name": "main",
"image": "argoproj/argosay:v2",
"command": [
"/argosay"
],
"args": [
"echo",
"{{inputs.parameters.message}}"
]
}
}
],
"ttlStrategy": {
"secondsAfterCompletion": 300
},
"podGC": {
"strategy": "OnPodCompletion"
}
}
}
}
```

```bash
curl -d @workflowtemplate.json 'http://localhost:5000/argo/template'
```

### Example Response

```json
{
"metadata": {
"creationTimestamp": "2023-03-23T17:28:34+00:00",
"generation": 1,
"labels": {
"example": "true",
"workflows.argoproj.io/creator": "system-serviceaccount-argo-argo-argo-workflows-server"
},
"managedFields": [
{
"apiVersion": "argoproj.io/v1alpha1",
"fieldsType": "FieldsV1",
"fieldsV1": {},
"manager": "argo",
"operation": "Update",
"time": "2023-03-23T17:28:34+00:00"
}
],
"name": "fantastic-tiger",
"namespace": "argo",
"resourceVersion": "12030505",
"uid": "e7609791-2d30-4dae-a47e-f1796846068d"
},
"spec": {
"arguments": {
"parameters": [
{
"name": "message",
"value": "hello argo"
}
]
},
"entrypoint": "argosay",
"podGC": {
"strategy": "OnPodCompletion"
},
"templates": [
{
"container": {
"args": [
"echo",
"{{inputs.parameters.message}}"
],
"command": [
"/argosay"
],
"image": "argoproj/argosay:v2",
"name": "main",
"resources": {}
},
"inputs": {
"parameters": [
{
"name": "message",
"value": "{{workflow.parameters.message}}"
}
]
},
"metadata": {},
"name": "argosay",
"outputs": {}
}
],
"ttlStrategy": {
"secondsAfterCompletion": 300
},
"workflowMetadata": {
"labels": {
"example": "true"
}
}
}
}
```
2 changes: 2 additions & 0 deletions docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,5 @@ Enhanced the ArgoClient -> Argo_Get_WorkflowLogsAsync method to decode the json
Added Mongo Migraions, to allow changes without breaking current stored entries

Added resource constraints to the generated ARGO templates, so these can operate within a Kubernetes cluster with a Resource Quota in the ARGO namespace.

Added ability for plugins (currently Argo) to have controllers, any dll marked with the new Monai.Deploy.WorkflowManager.Shared.PlugInAttribute will have any controllers added to TaskManager
10 changes: 5 additions & 5 deletions src/Monai.Deploy.WorkflowManager.sln
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,10 @@ Global
{013395D4-2091-4AB8-96CB-867C0F982096}.Debug|Any CPU.Build.0 = Debug|Any CPU
{013395D4-2091-4AB8-96CB-867C0F982096}.Release|Any CPU.ActiveCfg = Release|Any CPU
{013395D4-2091-4AB8-96CB-867C0F982096}.Release|Any CPU.Build.0 = Release|Any CPU
{973A5B90-C143-46B6-899D-79E3D46370C4}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{973A5B90-C143-46B6-899D-79E3D46370C4}.Debug|Any CPU.Build.0 = Debug|Any CPU
{973A5B90-C143-46B6-899D-79E3D46370C4}.Release|Any CPU.ActiveCfg = Release|Any CPU
{973A5B90-C143-46B6-899D-79E3D46370C4}.Release|Any CPU.Build.0 = Release|Any CPU
{91A0D599-472F-4238-A1A3-07807F9C5F61}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{91A0D599-472F-4238-A1A3-07807F9C5F61}.Debug|Any CPU.Build.0 = Debug|Any CPU
{91A0D599-472F-4238-A1A3-07807F9C5F61}.Release|Any CPU.ActiveCfg = Release|Any CPU
Expand Down Expand Up @@ -224,10 +228,6 @@ Global
{75A4AEDA-0386-4B2D-9DBA-BC9AE733660E}.Debug|Any CPU.Build.0 = Debug|Any CPU
{75A4AEDA-0386-4B2D-9DBA-BC9AE733660E}.Release|Any CPU.ActiveCfg = Release|Any CPU
{75A4AEDA-0386-4B2D-9DBA-BC9AE733660E}.Release|Any CPU.Build.0 = Release|Any CPU
{973A5B90-C143-46B6-899D-79E3D46370C4}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{973A5B90-C143-46B6-899D-79E3D46370C4}.Debug|Any CPU.Build.0 = Debug|Any CPU
{973A5B90-C143-46B6-899D-79E3D46370C4}.Release|Any CPU.ActiveCfg = Release|Any CPU
{973A5B90-C143-46B6-899D-79E3D46370C4}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
Expand All @@ -246,6 +246,7 @@ Global
{918E4DE3-A7BF-4B7F-9B5A-5C36FEFA3C30} = {71DDEE7B-E213-4E39-A7F4-4646783A27F7}
{722C0D57-49F8-4178-88F0-06E369B797A3} = {71DDEE7B-E213-4E39-A7F4-4646783A27F7}
{013395D4-2091-4AB8-96CB-867C0F982096} = {541C5347-5D7D-44B7-95D3-B6FB3D9EB955}
{973A5B90-C143-46B6-899D-79E3D46370C4} = {71DDEE7B-E213-4E39-A7F4-4646783A27F7}
{91A0D599-472F-4238-A1A3-07807F9C5F61} = {71DDEE7B-E213-4E39-A7F4-4646783A27F7}
{7D85E95C-A263-429F-BF8B-8F4A922FD579} = {541C5347-5D7D-44B7-95D3-B6FB3D9EB955}
{A966A7B9-2D4E-4A93-8D20-BD140E7A7F85} = {71DDEE7B-E213-4E39-A7F4-4646783A27F7}
Expand All @@ -268,7 +269,6 @@ Global
{81E3F943-B992-4C81-AA09-A17C05081236} = {37A19144-CEA5-47A2-9FFD-22C522E8B895}
{C853A9E3-C53D-4B1A-BFDA-228689A8C94C} = {71DDEE7B-E213-4E39-A7F4-4646783A27F7}
{75A4AEDA-0386-4B2D-9DBA-BC9AE733660E} = {71DDEE7B-E213-4E39-A7F4-4646783A27F7}
{973A5B90-C143-46B6-899D-79E3D46370C4} = {71DDEE7B-E213-4E39-A7F4-4646783A27F7}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {DC0D56C8-D8CB-45CE-B528-F3DCF86D63ED}
Expand Down
25 changes: 25 additions & 0 deletions src/Shared/Shared/PlugInAttribute.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
/*
* Copyright 2022 MONAI Consortium
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

namespace Monai.Deploy.WorkflowManager.Shared
{
[AttributeUsage(AttributeTargets.Assembly)]
public class PlugInAttribute : Attribute
{
public PlugInAttribute() { }

}
}
40 changes: 34 additions & 6 deletions src/TaskManager/Plug-ins/Argo/ArgoClient.cs
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ public interface IArgoClient
Task<Version?> Argo_GetVersionAsync();

Task<string?> Argo_Get_WorkflowLogsAsync(string argoNamespace, string name, string podName, string logOptions_container);

Task<WorkflowTemplate> Argo_CreateWorkflowTemplateAsync(string argoNamespace, WorkflowTemplateCreateRequest body, CancellationToken cancellationToken);
}

public class ArgoClient : IArgoClient
Expand All @@ -62,7 +64,7 @@ public async Task<Workflow> Argo_CreateWorkflowAsync(string argoNamespace, Workf

var Method = "POST";
var content = new StringContent(Newtonsoft.Json.JsonConvert.SerializeObject(body));
return await SendRequest(content, urlBuilder, Method, cancellationToken).ConfigureAwait(false);
return await SendRequest<Workflow>(content, urlBuilder, Method, cancellationToken).ConfigureAwait(false);

}

Expand Down Expand Up @@ -99,7 +101,7 @@ public async Task<Workflow> Argo_StopWorkflowAsync(string argoNamespace, string

var Method = "PUT";
var content = new StringContent(Newtonsoft.Json.JsonConvert.SerializeObject(body));
return await SendRequest(content, urlBuilder, Method, new CancellationToken()).ConfigureAwait(false);
return await SendRequest<Workflow>(content, urlBuilder, Method, new CancellationToken()).ConfigureAwait(false);

}

Expand All @@ -114,7 +116,7 @@ public async Task<Workflow> Argo_TerminateWorkflowAsync(string argoNamespace, st

var Method = "PUT";
var content = new StringContent(Newtonsoft.Json.JsonConvert.SerializeObject(body));
return await SendRequest(content, urlBuilder, Method, new CancellationToken()).ConfigureAwait(false);
return await SendRequest<Workflow>(content, urlBuilder, Method, new CancellationToken()).ConfigureAwait(false);
}

public async Task<WorkflowTemplate> Argo_GetWorkflowTemplateAsync(string argoNamespace, string name, string getOptions_resourceVersion)
Expand Down Expand Up @@ -162,7 +164,7 @@ public async Task<WorkflowTemplate> Argo_GetWorkflowTemplateAsync(string argoNam
urlBuilder.Length--;
return await GetRequest<string>(urlBuilder, true).ConfigureAwait(false);
}
private async Task<Workflow> SendRequest(StringContent stringContent, StringBuilder urlBuilder, string Method, CancellationToken cancellationToken)
private async Task<T> SendRequest<T>(StringContent stringContent, StringBuilder urlBuilder, string Method, CancellationToken cancellationToken)
{
using (var request = new HttpRequestMessage())
{
Expand All @@ -172,7 +174,17 @@ private async Task<Workflow> SendRequest(StringContent stringContent, StringBuil
request.Headers.Accept.Add(System.Net.Http.Headers.MediaTypeWithQualityHeaderValue.Parse("application/json"));
request.RequestUri = new Uri(urlBuilder.ToString(), UriKind.RelativeOrAbsolute);

var response = await _httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false);
HttpResponseMessage? response = null;
try
{
response = await _httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false);
}
catch (Exception ex)
{
var mess = ex.Message;
throw;
}


try
{
Expand All @@ -186,7 +198,7 @@ private async Task<Workflow> SendRequest(StringContent stringContent, StringBuil
var status = (int)response.StatusCode;
if (status == 200)
{
var objectResponse_ = await ReadObjectResponseAsync<Workflow>(response, headers).ConfigureAwait(false);
var objectResponse_ = await ReadObjectResponseAsync<T>(response, headers).ConfigureAwait(false);
if (objectResponse_.Object == null)
{
throw new ApiException("Response was null which was not expected.", status, objectResponse_.Text, headers, null);
Expand Down Expand Up @@ -336,6 +348,22 @@ protected virtual async Task<ObjectResponseResult<string>> ReadLogResponseAsync(
}
}

/// <param name="cancellationToken">A cancellation token that can be used by other objects or threads to receive notice of cancellation.</param>
/// <returns>A successful response.</returns>
/// <exception cref="ApiException">A server side error occurred.</exception>
public virtual async Task<WorkflowTemplate> Argo_CreateWorkflowTemplateAsync(string argoNamespace, WorkflowTemplateCreateRequest body, CancellationToken cancellationToken)
{
Guard.Against.NullOrWhiteSpace(argoNamespace);
Guard.Against.Null(body);

var urlBuilder = new StringBuilder();
urlBuilder.Append(CultureInfo.InvariantCulture, $"{FormattedBaseUrl}/api/v1/workflow-templates/{argoNamespace}");

var Method = "POST";
var content = new StringContent(Newtonsoft.Json.JsonConvert.SerializeObject(body));

Check warning

Code scanning / CodeQL

Cross-site scripting

[User-provided value](1) flows to here and is written to HTML or JavaScript.
return await SendRequest<WorkflowTemplate>(content, urlBuilder, Method, cancellationToken).ConfigureAwait(false);
}

protected struct ObjectResponseResult<T>
{
public ObjectResponseResult(T responseObject, string responseText)
Expand Down
Loading