Skip to content

Add a new catalogs property to workflow resources #1021

Closed
@cdavernas

Description

@cdavernas

What would you like to be added:

Add a new catalogs property to workflow resources.

Why is this needed:

This addition allows authors to define and configure catalogs within a workflow. The catalog's name can serve as shorthand for importing and calling custom functions, using the format {function}:{version}@{catalog}.

This format enables SDKs and runtimes to differentiate between imported functions and those defined at the top level, ensuring validation for tasks that call non-imported functions. It also guarantees that references to functions are validated at runtime. Additionally, this avoids the need for runtimes to search through all catalogs for a function definition, as the catalog is explicitly identified. It also supports defining functions with the same name and version across different catalogs.

We should define a default catalog, which may or may not be provided by the runtimes. This catalog can take any form, entirely at the runtime's discretion, and is automatically available to all workflows without the need for explicit declaration. This approach allows runtimes to offer mechanisms for users to create and share "private" functions specific to their runtime environment.

Proposal:

Calling a custom function, imported from a defined catalog:

document:
  dsl: 1.0.0-alpha4
  namespace: default
  name: lds-session-create
  version: '0.1.0'
use:
  catalogs:
    catalogA:
      endpoint:
        uri: https://custom-functions-catalog.com #this is the base uri, to which the catalog path structure should be appended (/functions/foo/1.0.0/...)
        authentication: #this allows for private and/or secured catalogs
          basic:
            username: admin
            password: 12345
do:
  - callCustomFunction:
      call: greet:1.5.2@catalogA
      with:
        greetings: Hello, World!

Calling a custom function, imported from the default catalog:

document:
  dsl: 1.0.0-alpha4
  namespace: default
  name: lds-session-create
  version: '0.1.0'
do:
  - callCustomFunction:
      call: greet:1.5.2@default
      with:
        greetings: Hello, World!

Additional notes:

Some concepts have been partially discussed in #962.

Metadata

Metadata

Assignees

Labels

area: specChanges in the Specificationchange: documentationImprovements or additions to documentation. It won't impact a version change.change: featureNew feature or request. Impacts in a minor version change

Type

No type

Projects

Status

Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions