Function definition namespaces (Importing several function definition files)

[fjtirado]

As per spec "The functions property can be either an in-line function definition array, or an URI reference to a resource containing an array of functions definition. Referenced resource can be used by multiple workflow definitions."

This covers the following scenarios:

• all function definitions to be used by an specific workflow are defined in the external file
• all function definitions to be used are defined by the workflow itself.

But it does not cover two different scenarios that might arise:

• using function definitions defined in multiple external files. Think of a file defining all functions for a particular api, another file defining all functions for another api and a workflow that orchestate the integration between these two APIS. Ideally you do not want to create a third file which merge the two others.
• using an external file and adding specific functions for that particular workflow that wont be used in any other workflow. Ideally you do not want to update the external file with the private function

Therefore, I propose to update the Workflow specification to:

• Establish that functions property can be only an array of function definitions (where all functions privative to that workflow are defined), but not an external reference.
• Add a new property "namepaces" or "imports". That property will be an associative map, which value is the uri of a file containing array of function definitions and key is the namespace that identifies it within this workflow. The namespace is needed to avoid name clashing between different imported files. When calling the imported function within an state, the refName must have the namepace prefixed.

So, we will have something like in the header of the flow

"namespaces": {"myRpcAPI":"ARPCApi.json", "myOpenAPI": "OpenApi.json},
"functions": [{"name":"doSomething", "type": "expression", "operation":"jq  expression specific to this flow"}]

and

"refName": "myRpcAPI:doSomething"
"refName": "myOpenAPI:doSomething"
"refName":"doSomething"

[cdavernas]

@fjtirado Funny, I had the exact same proposal, as I started coding Synapse's resource server at the end of last week. Wanted to bring it up in today's meeting, but there you go! +1000

For namespaces, however, I'd try to find, if possible, something more elegant than file-based. And IMO the term is not ubiquitous enough, as the same concept could very well be applied to at least events and auth too. I have nothing in mind right now, but even something as functionNamespaces would be a better fit IMO.

[ricardozanini]

I liked the idea as well, and I agree that namespaces sounds too generic. Not sure even about functionNamespaces, but it seems better.

I'd rather use import somehow since the <namespace>: implies that is a namespace already.

Maybe functionImports?

[fjtirado]

yes, I think functionImports is fine, I just mention namespace becuase of my c++ heritage ;)

[ricardozanini]

Then:

{
    "functions": [{ 
         "operation": "myfile.yaml#myoperation"
         "name": "privateFunction1"
    }],
    "functionImports": {
         "sharedApi": "sharedFunctionDefinitions.json",
         "sharedApi2": "sharedFunctionDefinitions2.json"
    }
}

The format is "namespace": "reference".

Then in the states we refer by sharedApi:publicFunction1.

[fjtirado]

@ricardozanini functionImports should be an object (it is a dictionary) not an array.

[tsurdilo]

imo the aggregation should happen on runtime side. having an uri is enough for me. i rather not add this is possible, so sorry

[fjtirado]

@tsurdilo how do we agregate on runtime? can you give an example?

[tsurdilo]

  "functions": "htttp/mycomp.org/my/prod/function/definitions?namespace=abc"

this uri endpoint can aggregate all your function defs and stick in the functions array which then is also validatable against our single functions schema

[cdavernas]

this uri endpoint can aggregate all your function defs and stick in the functions array which then is also validatable against our single functions schema

@tsurdilo IMHO, your suggestion is impossible to define and/or implement. How can you, as a runtime, determine the format of the data returned by a random url? Sometimes it might just be a JSON of a functionDef collection, sometimes a collection of collections, etc.

[tsurdilo]

this is a compile time resolve so yes you can validate its response json, otherwise implement the validation on service side that returns the resolved json, thats fine too since in most cases you control it

[fjtirado]

how does that URI endpoint knows what needs to be aggregated?
Remember, the use case is, the writer of the flow has already written a function definition array (lets call it file 1) for API 1 and another function definition array (lets call it file 2) for API 2, and it is already using file 1 for flows that interacts just with api 1 and file 2 for flows that interacts with api 2
now he wants to write a flow that interacts both with API 1 and API 2, how he can make a reference to both arrays with just one URI?