Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: new features in 3.6 #13739

Merged
merged 9 commits into from
Oct 24, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .spelling
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ Ploomber
Postgres
PriorityClass
RCs
Risc-V
Roadmap
RoleBinding
SDKs
Expand Down Expand Up @@ -213,6 +214,7 @@ stateful
stderr
symlinks
temporality
tolerations
triaged
un-reconciled
v1
Expand Down
77 changes: 77 additions & 0 deletions docs/features-3.6.md

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

relative links to the docs for some of the features here would be really helpful to be able to see them in context when reading this

Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# New features

This is a concise list of new features.
See [the upgrade notes](upgrading.md#upgrading_to_v3.6) for information on breaking changes and deprecations.
Joibel marked this conversation as resolved.
Show resolved Hide resolved

## UI

* [#13519](https://github.com/argoproj/argo-workflows/pull/13519): The full name of the workflow is now visible in the list details of a workflow.
* [#13284](https://github.com/argoproj/argo-workflows/pull/13284): Various time displays can be switched between relative and absolute ISO time.
* [#10553](https://github.com/argoproj/argo-workflows/pull/10553): You can now use markdown in workflow titles and descriptions and it will be displayed in the UI.
* [#12350](https://github.com/argoproj/argo-workflows/pull/12350): The UI will now show the directory used for input artifacts.
* [#12873](https://github.com/argoproj/argo-workflows/pull/12873): You can also now see line numbers in the object view.
* [#13452](https://github.com/argoproj/argo-workflows/pull/13452): WorkflowTemplate and ClusterWorkflowTemplate will show you their execution history like you can see for CronWorkflows.
* [#12024](https://github.com/argoproj/argo-workflows/pull/12024): You will be able to see live logs from pods if retrieval of logs from archived workflows fails and the pod logs are available.
* [#12674](https://github.com/argoproj/argo-workflows/pull/12674): CronWorkflows and WorkflowTemplates now display their title and descriptions in the list view.
* [#12199](https://github.com/argoproj/argo-workflows/pull/12199): You can specify HTTP headers to use to detect IP addresses using the `IP_KEY_FUNC_HEADERS` environment variable. This is used in the rate limiter.
* [#13695](https://github.com/argoproj/argo-workflows/pull/13695): You can now retry a single node from a workflow, even if the workflow succeeded.
* [#13610](https://github.com/argoproj/argo-workflows/pull/13610): You can now filter with prefixes and patterns in the workflow list.

## Metrics

* [#13265](https://github.com/argoproj/argo-workflows/pull/13265): The workflow controller can now emit metrics over OpenTelemetry GRPC protocol
* [#13267](https://github.com/argoproj/argo-workflows/pull/13267): with selectable temporality
* [#13268](https://github.com/argoproj/argo-workflows/pull/13268): configuration of what is emitted
* Many of the metrics have been updated which will require you [to change how you use them](upgrading.md#metrics_changes) and there are some new ones:
* [#13269](https://github.com/argoproj/argo-workflows/pull/13269): Version information in the controller
* [#13270](https://github.com/argoproj/argo-workflows/pull/13270): Is this controller the leader
* [#13271](https://github.com/argoproj/argo-workflows/pull/13271): Kubernetes API calls duration
* [#13272](https://github.com/argoproj/argo-workflows/pull/13272): Pod phase monitoring
* [#13274](https://github.com/argoproj/argo-workflows/pull/13274): CronWorkflows counters
* [#13497](https://github.com/argoproj/argo-workflows/pull/13497): CronWorkflows policy counters
* [#13275](https://github.com/argoproj/argo-workflows/pull/13275): Workflow Template counters
* [#13735](https://github.com/argoproj/argo-workflows/pull/13735): Counters to check if you're using deprecated features
* [#11927](https://github.com/argoproj/argo-workflows/pull/11927): There is a new `retries` variable available in metrics describing the number of retries.
* [#11857](https://github.com/argoproj/argo-workflows/pull/11857): Pod missing metrics will be emitted before pods are created

## Controller

* [#13358](https://github.com/argoproj/argo-workflows/pull/13358): You can use multiple mutexes and semaphores in the same workflow or template, and use both type of lock at the same time
* [#13419](https://github.com/argoproj/argo-workflows/pull/13419): The controller uses a queue when archiving workflows to improve memory management when archiving a large number of workflows at once
* [#12441](https://github.com/argoproj/argo-workflows/pull/12441): Plugins can now be stopped, so that a stopped workflow will shutdown its plugin nodes
* The OSS artifact driver:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Plugins" and "Artifacts" sections might be good to split out so it's not just "Controller". Artifacts and Plugins also run in the Server + Executor and Agent, respectively, so not in the Controller

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've renamed it General. Most people don't care which component a feature happens in, it's an irrelevance to them.

Copy link

@agilgur5 agilgur5 Oct 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that's necessarily accurate -- if I use artifacts, having an "Artifacts" section would make it easy for me to see what's new with that "feature set".
Permissions are also relevant per component. But usage wise, it's less about the "architectural component" and more about the broader "feature set" (or "area" as we use in labels) like "Artifacts", "Plugins", "SSO", "SDKs", "CLI", etc, which is what I suggested above.

You also already did that with the other sections you wrote, such as "UI", "CLI", "Metrics", and "CronWorkflows", so this would be consistent with those existing ones as well.

And in general, I don't think a user would complain about more organization

* [#12188](https://github.com/argoproj/argo-workflows/pull/12188): Can now work with directories,
* [#12907](https://github.com/argoproj/argo-workflows/pull/12907): Supports deletion,
* [#12908](https://github.com/argoproj/argo-workflows/pull/12908): Supports streaming.
* [#12419](https://github.com/argoproj/argo-workflows/pull/12419): Pod deletion will now happen in parallel to speed it up.
* [#13360](https://github.com/argoproj/argo-workflows/pull/13360): You can use Shared Access Signatures to access artifacts stored in Azure.
* [#12413](https://github.com/argoproj/argo-workflows/pull/12413): Workflow pods now have a kubernetes finalizer to try to prevent them being deleted prematurely
* [#12325](https://github.com/argoproj/argo-workflows/pull/12325): Large environment variables will be offloaded to Config Maps
* [#12328](https://github.com/argoproj/argo-workflows/pull/12328): Large and flat workflows where there are many steps that need resolving at the same time could time out during template referencing. This is now much faster.
* [#12568](https://github.com/argoproj/argo-workflows/pull/12568): Kubernetes scheduling constraints such as node selectors and tolerations will now be honored where they are specified in a WorkflowTemplate. These will be applied to the task and step pods.
* [#12984](https://github.com/argoproj/argo-workflows/pull/12984): The pods created by workflows will have a `seccompProfile` of `RuntimeDefault` by default.
* [#12842](https://github.com/argoproj/argo-workflows/pull/12842): You can now template the `name` and `template` in a `templateRef`. This allows for fully data driven workflow DAGs.
* [#13194](https://github.com/argoproj/argo-workflows/pull/13194): The expr library has been upgraded providing some new functions in expressions.
* [#13746](https://github.com/argoproj/argo-workflows/pull/13746): Configuration option to avoid sending kubernetes Events for workflows.
* [#13742](https://github.com/argoproj/argo-workflows/pull/13742): `ARGO_TEMPLATE` environment variable can be configured not to contain input parameters to reduce storage usage.
Joibel marked this conversation as resolved.
Show resolved Hide resolved
* [#13745](https://github.com/argoproj/argo-workflows/pull/13745): Added an option to skip workflow duration estimation because it can be expensive.

## Cron Workflows

* [#12616](https://github.com/argoproj/argo-workflows/pull/12616): You can now specify multiple cron schedules on a single CronWorkflow.
* [#12305](https://github.com/argoproj/argo-workflows/pull/12305): You can also use a stop strategy on Cron Workflows to stop them running any more workflows after a set of conditions occur such as too many errors.
* [#13474](https://github.com/argoproj/argo-workflows/pull/13474): Cron Workflows also now have a when expression to further tune which occurrences of the workflow will run and which may be skipped
Joibel marked this conversation as resolved.
Show resolved Hide resolved

## CLI

* [#12803](https://github.com/argoproj/argo-workflows/pull/12803): You can now update Cron Workflows, Workflow Templates and Cluster Workflow Templates with the `update` command via the CLI
* [#13364](https://github.com/argoproj/argo-workflows/pull/13364): You can selectively list workflow templates using a `-l` label selector
* [#13128](https://github.com/argoproj/argo-workflows/pull/13128): The CLI will now generate shell completions for the [fish shell](https://fishshell.com/)
* [#12977](https://github.com/argoproj/argo-workflows/pull/12977): We also build and ship the CLI complied for [Risc-V](https://riscv.org/)
* [#12953](https://github.com/argoproj/argo-workflows/pull/12953): The lint command supports a `--no-color` flag
* [#13695](https://github.com/argoproj/argo-workflows/pull/12953): The `--output` flag is now validated

## Build and Development

* [#13000](https://github.com/argoproj/argo-workflows/pull/13000): There is now a `/retest` command for retesting PRs in Github that occasionally fail in a flakey test
Copy link

@agilgur5 agilgur5 Oct 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dev stuff is not needed and is counter to the stated purpose of having a concise list (and there's plenty we could bikeshed about if we had a dev list)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a new feature and this list is to enable people to discover new features. It isn't needed but I'd like to celebrate it by leaving it in.

Copy link

@agilgur5 agilgur5 Oct 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's in the blog post already, and I think that's sufficient. The blog post includes "non-concise" things.
There are also certainly other new "CI features" if we wanted to call them that, which, as I wrote above, could definitely make for more bikeshedding

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It also is not specific to 3.6 either since it's CI; and that can also make for more bikeshedding

* [#12867](https://github.com/argoproj/argo-workflows/pull/12867): You can supply your own HTTP client when using the go API client, allowing for adding a proxy

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The API client is an SDK change, so should be under an SDKs section

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SDK stands for software development kit, and this is under development. I don't think we need to split.

Copy link

@agilgur5 agilgur5 Oct 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Build and Development", especially with the first item being a CI thing, makes it sound like items that are only relevant to Contributors, but the SDK is user facing. It's also called an "SDK" in the docs and in our own labels. Similarly, all contributor facing docs are under the "Developer Guide" in the docs.

I definitely don't think we should be creating new names for existing things, that's confusing at best.

SDK stands for software development kit, and this is under development

If you want to debate the difference between denotation and connotation: the connotations are not equivalent in this context, and if you were to use denotation, the entire codebase could be considered "development", so that is not a sufficient boundary

2 changes: 2 additions & 0 deletions docs/upgrading.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summar

## Upgrading to v3.6

A list of [new features in 3.6](features-3.6.md).
Joibel marked this conversation as resolved.
Show resolved Hide resolved

### Deprecations

The following features are deprecated and will be removed in a future verison of Argo Workflows:
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,7 @@ nav:
- installation.md
- releases.md
- upgrading.md
- features-3.6.md
Joibel marked this conversation as resolved.
Show resolved Hide resolved
- security.md
- Configuration:
- managed-namespace.md
Expand Down
28 changes: 28 additions & 0 deletions ui/src/app/modals/new-version-modal.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,34 @@ export function NewVersionModal({version, dismiss}: {version: string; dismiss: (
<h4 className='new-version-modal-title'>
It looks like <b>{version}</b> has just been installed!
</h4>
<h5>v3.6</h5>
<ul className='new-version-modal-bullets'>
<li>
Metrics are available via the{' '}
<a href='https://opentelemetry.io/' target='_blank' rel='noreferrer'>
OpenTelemetry
</a>{' '}
protocol, and there are a number of new or changed metrics
</li>
<li>
<a href='https://argo-workflows.readthedocs.io/en/release-3.5/cron-workflows?utm_source=argo-ui' target='_blank' rel='noreferrer'>
Cron Workflows
Joibel marked this conversation as resolved.
Show resolved Hide resolved
</a>{' '}
have many more scheduling options including multiple schedules, a `when` expression and stopping strategies
Joibel marked this conversation as resolved.
Show resolved Hide resolved
</li>
<li>OSS artifacts are more versatile, supporting directories, deletion and streaming</li>
Joibel marked this conversation as resolved.
Show resolved Hide resolved
<li>Many UI updates including execution history of CronWorkflows</li>
<li>A number of improvements to performance and memory management</li>
</ul>
<p>
<a href='https://blog.argoproj.io/argo-workflows-3-6-aa037cd782be' target='_blank' rel='noreferrer'>
Learn more
</a>{' '}
or{' '}
<a href='https://argo-workflows.readthedocs.io/en/release-3.6/features-3.6/?utm_source=argo-ui' target='_blank' rel='noreferrer'>
see the list of changes
</a>
</p>
<h5>v3.5</h5>
<ul className='new-version-modal-bullets'>
<li>Removed Archived Workflows page; Workflows List page now shows both</li>
Expand Down
Loading