Skip to content

Define design for Renovate usage #15594

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Define design for Renovate usage #15594

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f11f90d
Define design for Renovate usage
mthalman Mar 5, 2025
37dbc01
Add example scenarios section
mthalman Mar 7, 2025
9feb5bc
Other updates from feedback
mthalman Mar 7, 2025
e134d08
added branch sync line
mthalman Mar 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 82 additions & 0 deletions Documentation/Renovate.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# Renovate Dependency Update Tool

## Introduction

This document outlines the integration of [Renovate](https://github.com/renovatebot/renovate) into Arcade to automate dependency updates.
Renovate is an automated dependency update tool that generates PRs for updating dependencies from a wide variety of sources.

Renovate is similar to Dependabot in its purpose.
Dependabot should be used when possible.
However, Renovate supports a much broader range of [dependency types](https://docs.renovatebot.com/modules/datasource/), most notably Docker and GitHub releases.
For example, Renovate can automatically generate a PR to update a referenced Docker image when a newer version is available.

### Example Scenarios

Here are two scenarios demonstrating the usefulness of Renovate automatically making dependency updates:

#### Container Tags

Repos that reference container tags from the [dotnet/dotnet-buildtools-prereqs-docker](https://github.com/dotnet/dotnet-buildtools-prereqs-docker) repo need to maintain those tags to ensure they are supported.

This can be as simple as automatically updating to a new major version of Linux distro:

```diff
-mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-amd64
+mcr.microsoft.com/dotnet-buildtools/prereqs:debian-12-helix-amd64
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

given our support matrix, do we actually want upgrades like this in our repos❓

separately, does Renovate handle updating FROM in Dockerfiles themselves❓ I mostly want to avoid someone accidentally merging a Renovate PR that bumps FROM library/debian:bookworm to FROM library/debian:bookworm in the Debian 12 Dockerfile

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

given our support matrix, do we actually want upgrades like this in our repos❓

Repo maintainers are free to choose what they want to have updated because they are in control of their configuration. So if they don't want something to be considered for updating by Renovate, they can choose to not have it included. But to answer your question, I would expect that in most cases, yes, you would want to update to target the latest supported version.

separately, does Renovate handle updating FROM in Dockerfiles themselves❓ I mostly want to avoid someone accidentally merging a Renovate PR that bumps FROM library/debian:bookworm to FROM library/debian:bookworm in the Debian 12 Dockerfile

I think you have a typo since you indicated bookworm twice. Did you mean you want to avoid it updating from bookworm to trixie for a Dockerfile specifically meant for Debian 12? Yeah, in that case Renovate allows you to control the versioning level of updates you want. For example, you could limit the updates to minor version only so that it would never update to a new major version. That would cover your scenario, if I'm understanding it right. For example, you could configure it so if you had a tag like debian:12.8, it would update it to debian:12.9 but not debian:13.x.

```

Or automatically pinning to the latest version of a tag by it's digest value:

```diff
-mcr.microsoft.com/dotnet-buildtools/prereqs:debian-12-helix-amd64@sha256:b99da50c4cb425e72ee69c2b8c1fdf99e0f71059aee19798e2f9310141ea48fb
+mcr.microsoft.com/dotnet-buildtools/prereqs:debian-12-helix-amd64@sha256:6bb6fef390e6f09a018f385e346b0fe5999d7662acd84ca2655e9a3c3e622b71
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe dotnet/runtime is considering moving toward this approach from our current floating tags. if that's correct, I can definitely see the value — means PR validation runs before a newer image breaks the world. please confirm

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, exactly. This is described in dotnet/runtime#113455.

```

Renovate can detect when these new container images are available and submit PRs to update sources accordingly.

Related issue: [Automatically update image references in consuming repos (#1321)](https://github.com/dotnet/dotnet-buildtools-prereqs-docker/issues/1321)

#### GitHub Release

There are many cases where a version of OSS published via GitHub releases is being referenced by a .NET repository.
Those versions can be kept updated automatically as new releases occur.

For example, there are Dockerfiles in the [dotnet/dotnet-buildtools-prereqs-docker](https://github.com/dotnet/dotnet-buildtools-prereqs-docker) repo which reference the LLVM version that can be maintained by having Renovate automatically check for new [LLVM releases](https://github.com/llvm/llvm-project/releases).

```diff
-LLVM_VERSION=19.1.7
+LLVM_VERSION=20.1.0
```

## Design
Copy link
Member

@richlander richlander Mar 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to cover a couple intended to scenarios and why they are important.

An important question to address will be the timing model we use. While that's not important to decide at this stage, it is important to discuss so that we validate that we've got a working model.

I'm thinking about dotnet/dotnet-buildtools-prereqs-docker -> dotnet/runtime:

Here are three options:

  • New PR per new image available
  • New PR per day
  • New PR on-demand

For people at all familiar with this repo, it doesn't take long to realize that the first option is going to generate too many PRs. See publish history to grasp why. The middle model seems like a good default. However, it is suprisingly common that we need a new image to unblock runtime. We had one today, with dotnet/dotnet-buildtools-prereqs-docker#1381. We could wait for the "overnight process" to deal with that. However, an on-demand model seems like it would be nice. Perhaps we can wait to prove that it is necessary. We could also write tools to enable manual sha-updating per dotnet/runtime#113185.

I see this thinking as part of the initial design.


### Fork Mode

Protecting GitHub repositories in the dotnet organization from direct access by the Renovate tool is crucial.
Renovate will be used in fork mode, limiting its permissions to forked repositories.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will these repos be long-lived? Where will they exist?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I read that the permissions/threat model is that Renovate is the same as all the OSS devs contributing on our repos. That would be good to call out explicitly (in a prior section).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will these repos be long-lived? Where will they exist?

Yes, once a dotnet repo is forked, the fork will be preserved indefinitely. The fork repo will be owned by the bot account (e.g. https://github.com/dotnet-renovate-bot/dotnet-runtime).

I read that the permissions/threat model is that Renovate is the same as all the OSS devs contributing on our repos. That would be good to call out explicitly (in a prior section).

Good point.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How will the repos get the latest commits? Will the bot update its main branch before creating its change in a topic branch?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the bot handles all of that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to see that covered in the design. It doesn't need to be deep.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you think this particular aspect belongs in the design? That's very much an implementation detail. And given that there's nothing we need to do to enable it, it seems unnecessary. There's all kinds of questions that could be asked of how Renovate works, but that doesn't mean they need to be called out in the design.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to know if renovate can have multiple PRs in-flights from the same repo at once. I assume so but wanted to validate.

This avoids giving write permissions to Renovate on any dotnet repository.
This means that Renovate acts as though it's any other outside contributor, not requiring any special permissions in the dotnet organization or repositories.

A GitHub bot account, `dotnet-renovate-bot`, is used to manage the Renovate operations.
This account has the ability to create forks from dotnet repositories, which will be the source of the head branch for PRs that are created.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This text implies but does not explicitly state that the PRs will be created by this bot. That would be good to fix.

The PRs generated by Renovate will be done using this bot account.
Renovate also handles synchronization from the upstream branch automatically.
GitHub scopes required by this account: `repo`, `workflow`.

### Repo Usage

Arcade provides an Azure DevOps pipeline YAML job template that repositories should utilize when making use of Renovate.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this exist yet? If so, then link it here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it doesn't. I purposely wrote it forward-looking so that it would be relevant once it's implemented.

This template handles the execution of Renovate, ensuring a standardized approach across all repositories.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

implementation detail but how complex would such a template need to be❓

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBD

Repositories wishing to make use of Renovate can reference this template from a pipeline YAML file, setting the schedule trigger as desired.
Consuming repositories are responsible for providing their own [Renovate configuration file](https://docs.renovatebot.com/configuration-options/) that describes which dependencies should be updated.

Repositories are in control of how often Renovate will run via the pipeline schedule trigger (once a day, once a week, etc).
Typically, a pipeline would be configured to update all dependencies at once in which case all dependencies are described in one Renovate config file.
But in some cases, it may be desired to have certain dependencies updated on a different schedule than others.
In that case, the repo maintainer would define multiple pipelines with those separate schedules, all configured to run Renovate.
Importantly, each of those pipelines would need to reference a separate Renovate config file which defines the scope of the updates that will be targeted.
Lastly, a maintainer is free to manually run the pipeline if an expected update is needed right away.

## Renovate Configuration Patterns

TBD