Skip to content

composite template deprecation #664

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

composite template deprecation #664

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 25 additions & 9 deletions docs/users/fbc_onboarding.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,7 @@ a local cache is generated during a run.
The script will execute the following steps:
- Fetch a list of currently supported OCP catalogs
- Transform existing catalogs into a basic template
- Generate a composite template for an operator
- Generate an FBC catalog for a given operator
- Generate FBC catalog contributions for an operator for all supported catalog versions
- Update operator ci.yaml config

The following examples will be using `aqua` operator as an example. Change an operator name that matches the operator you want to convert.
Expand All @@ -61,19 +60,17 @@ $ tree operatos/aqua
operators/aqua
├── 0.0.1
...
├── catalogs.yaml
├── catalog-templates
│   ├── v4.12.yaml
│   ├── v4.13.yaml
│   ├── v4.14.yaml
│   ├── v4.15.yaml
│   └── v4.16.yaml
├── ci.yaml
└── composite-config.yaml
```
... and File-based catalog in `catalogs` directory
```bash
$ tree catalogs
$ tree (repository root)/catalogs
catalogs
├── v4.12
│   └── aqua
Expand All @@ -97,19 +94,38 @@ catalogs
Artifacts generated in the previous step need to be added to a git and submitted via pull request. The operator pipeline validates the content of the catalogs and releases changes into ocp catalogs.

```bash
$ git add operators/aqua/{catalog-templates,catalogs.yaml,composite-config.yaml,ci.yaml}
$ git add operators/aqua/{catalog-templates,ci.yaml}

$ git add
$ git add catalogs/{v4.12,v4.13,v4.14,v4.15,v4.16}/aqua

$ git commit --signoff -m "Add FBC resources for aqua operator"
```

## Generating catalogs from templates
Catalog templates are used to simplify a view of a catalog and allow easier manipulation of catalogs. The automated conversion pre-generates a basic + composite template that can be turned into full FBC using the following command:
Catalog templates are used to simplify a view of a catalog and allow easier manipulation of catalogs. The automated conversion pre-generates a basic template that can be turned into full FBC using the following command:

```bash
opm alpha render-template composite -f catalogs.yaml -c composite-config.yaml
opm alpha render-template basic ./catalog-templates/v${VERSION}.yaml > ${ROOT}/catalogs/v${VERSION}/${OPERATOR_NAME}/catalog.yaml
```
For the example `aqua` operator above, if could look like:
```bash
ROOT=../..
for VERSION in "4.12 4.13 4.14 4.15 4.16"
do
opm alpha render-template basic ./catalog-templates/v${VERSION}.yaml > ${ROOT}/catalogs/v${VERSION}/aqua/catalog.yaml
done
```

# -------------------------------------------------------------------
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the specific call-out here. I don't know if you're the right person @Allda. I got distracted from working on this a couple of times today, so I put the note in here so I wouldn't lose it. 😁

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have a tracker for it and we will consolidate the documentation and the tooling (makefile) to provide a single way of interacting with catalogs and onboardings.

Could you please remove the comment so we can potentially merge it without the notes?

## @Ales
not sure why the Makefile 'workflow' is separate from this doc, since the care-and-maintenance of catalog templates is pretty closely tied to an automation path, like the Makefile target:
```Makefile
# --- BASIC TEMPLATE ---
catalog: basic
```
I'd be tempted to merge these two docs and keep the "howto generate FBC contributions" is all about the Makefile,
deleting the contribution generation phases from the onboarding doc & script
# -------------------------------------------------------------------


Of course, you can choose any type of template that you prefer. More information about catalog templates can be found [here](https://olm.operatorframework.io/docs/reference/catalog-templates/)
8 changes: 2 additions & 6 deletions docs/users/fbc_workflow.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,10 @@ template the operator will be using.

* Basic template
* SemVer template
* Composite template

More information about each template can be found at [opm doc](https://olm.operatorframework.io/docs/reference/catalog-templates/).

The recommended template from the maintainability point of view is `SemVer` or `Composite` with
`SemVer`integration.
The recommended template from the maintainability point of view is `SemVer`.

## Generate catalogs using templates
To generate a final catalog for an operator a user needs to execute different `opm`
Expand All @@ -49,10 +47,8 @@ The right place for the Makefile is in the operator's root directory
├── 0.0.1
│   ├── manifests
│   └── metadata
├── catalogs.yaml
├── catalog-templates
├── ci.yaml
├── composite-config.yaml
└── Makefile

```
Expand All @@ -63,7 +59,7 @@ can be submitted as a PR in Github and once the PR is processed changes will be
OCP index.

```bash
$ tree catalogs
$ tree (repository-root)/catalogs
catalogs
├── v4.12
│   └── aqua
Expand Down
12 changes: 1 addition & 11 deletions fbc/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@
# OPM allows for the generation of catalogs using different templates.
# - basic: generates a basic catalog
# - semver: generates a catalog with semver versioning
# - composite: generates a catalog using a composite template

PDW=$(shell pwd)
OPERATOR_NAME=$(shell basename $(PDW))
Expand Down Expand Up @@ -47,9 +46,7 @@ fbc-onboarding: fbc-onboarding-deps clean
#
# --- SEMVER TEMPLATE ---
#catalog: semver
#
# --- COMPOSITE TEMPLATE ---
catalog: composite
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the removal of the composite as a default target for make catalog we should make the basic the default now.



# basic target provides an example FBC generation from a `basic` template type.
# this example takes a single file as input and generates a well-formed FBC operator contribution as an output
Expand All @@ -70,13 +67,6 @@ semver: ${BINDIR}/opm clean
${BINDIR}/opm alpha render-template semver -o yaml ${OPERATOR_CATALOG_TEMPLATE_DIR}/$${version}.yaml > ${CATALOG_DIR}/$${version}/${OPERATOR_NAME}/catalog.yaml; \
done

# composite target processes a composite template to generate the FBC contributions
# `render-template composite` has `--validate` option enabled by default,
# so no subsequent validation is required
.PHONY: composite
composite: ${BINDIR}/opm clean
${BINDIR}/opm alpha render-template composite -f ${PDW}/catalogs.yaml -c ${PDW}/composite-config.yaml

#
# validate target illustrates FBC validation
# all FBC must pass opm validation in order to be able to be used in a catalog
Expand Down
99 changes: 10 additions & 89 deletions operator-pipeline-images/operatorcert/entrypoints/fbc_onboarding.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,6 @@

LOGGER = logging.getLogger("operator-cert")

COMPOSITE_TEMPLATE_CATALOGS = "catalogs.yaml"
COMPOSITE_TEMPLATE_CONTRIBUTIONS = "composite-config.yaml"
CATALOG_TEMPLATES_DIR = "catalog-templates"


Expand Down Expand Up @@ -96,7 +94,7 @@ def opm_cache(image: str) -> bytes:
bytes: An output of the opm command
"""
LOGGER.debug("Building cache for %s", image)
output = run_command(["opm", "render", "-o", "yaml", "--migrate", image])
output = run_command(["opm", "render", "-o", "yaml", image])
return output.stdout


Expand Down Expand Up @@ -205,81 +203,6 @@ def generate_and_save_base_templates(
LOGGER.info("Template for %s saved to %s", version, template_path)


def generate_composite_templates(
operator: Operator, catalog_versions: List[str]
) -> Tuple[Dict[str, Any], Dict[str, Any]]:
"""
Generate a composite template for given operator and all supported versions

Args:
operator (Operator): A operator object
catalog_versions (List[str]): List of supported catalog versions

Returns:
Tuple[Dict, Dict]: catalog and contributions templates
"""
catalogs = [
{
"name": f"v{version}",
"destination": {
"workingDir": f"../../catalogs/v{version}",
},
"builders": ["olm.builder.basic", "olm.builder.semver"],
}
for version in catalog_versions
]
composite_catalogs = {
"schema": "olm.composite.catalogs",
"catalogs": catalogs,
}

components = [
{
"name": f"v{version}",
"destination": {
"path": operator.operator_name,
},
"strategy": {
"name": "basic",
"template": {
"schema": "olm.builder.basic",
"config": {
"input": f"{CATALOG_TEMPLATES_DIR}/v{version}.yaml",
"output": "catalog.yaml",
},
},
},
}
for version in catalog_versions
]

contributions = {"schema": "olm.composite", "components": components}
return composite_catalogs, contributions


def generate_and_save_composite_templates(
operator: Operator, catalog_versions: List[str]
) -> None:
"""
Generate and save composite templates for given operator and all supported versions

Args:
operator (Operator): A operator object
catalog_versions (List[str]): List of supported catalog versions
"""
catalog, contributions = generate_composite_templates(operator, catalog_versions)
with open(
os.path.join(operator.root, COMPOSITE_TEMPLATE_CATALOGS), "w", encoding="utf8"
) as f:
yaml.safe_dump(catalog, f, explicit_start=True)
with open(
os.path.join(operator.root, COMPOSITE_TEMPLATE_CONTRIBUTIONS),
"w",
encoding="utf8",
) as f:
yaml.safe_dump(contributions, f, explicit_start=True)


def update_operator_config(operator: Operator) -> None:
"""
Switch operator config to FBC
Expand All @@ -299,7 +222,7 @@ def update_operator_config(operator: Operator) -> None:
yaml.safe_dump(config, f, explicit_start=True)


def render_fbc_from_template(operator: Operator) -> None:
def render_fbc_from_template(operator: Operator, version: str) -> None:
"""
Render catalog from templates

Expand All @@ -311,11 +234,12 @@ def render_fbc_from_template(operator: Operator) -> None:
"opm",
"alpha",
"render-template",
"composite",
"-f",
COMPOSITE_TEMPLATE_CATALOGS,
"-c",
COMPOSITE_TEMPLATE_CONTRIBUTIONS,
"basic",
"-o",
"yaml",
os.path.join(operator.root, CATALOG_TEMPLATES_DIR, f"v{version}.yaml"),
">",
f"../../catalogs/v{version}/catalog.yaml"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're missing an operator name in the output path. Based on the design it should be ../../catalogs/v{version}/{operator-name}/catalog.yaml

],
cwd=operator.root,
)
Expand Down Expand Up @@ -356,11 +280,8 @@ def onboard_operator_to_fbc(
version, operator_name, cache_dir, template_dir
)

LOGGER.info("Generating composite templates")
generate_and_save_composite_templates(operator, supported_versions)

LOGGER.info("Rendering FBC from templates")
render_fbc_from_template(operator)
LOGGER.info("Rendering FBC from templates")
render_fbc_from_template(operator, version)

LOGGER.info("Updating operator config")
update_operator_config(operator)
Expand Down