test folder ignoring in config:base preset

[fgreinacher]

The config:base preset currently includes the following rules:

renovate/lib/config/presets/internal/default.ts

Lines 42 to 55 in e5ccf2b

	ignoreModulesAndTests: {
	description:
	'Ignore `node_modules`, `bower_components`, `vendor` and various test/tests directories',
	ignorePaths: [
	'**/node_modules/**',
	'**/bower_components/**',
	'**/vendor/**',
	'**/examples/**',
	'**/__tests__/**',
	'**/test/**',
	'**/tests/**',
	'**/__fixtures__/**',
	],
	},

At least test and tests are pretty much standard directory names, e.g. in .NET projects. In my opinion these should not be ignored, because you usually want to keep also your test frameworks up to date.

What are your thoughts behind this?

[rarkins]

I don't see a problem with it as I think this rule benefits the majority who don't want to update tests/ and is easily changed by the minority who do.

[fgreinacher]

What are your thoughts @Lorilatschki @ap0llo @dschafhauser?

[ap0llo]

I don't see a reason why not to update test dependencies. I haven't run into this issue personally, but I would have been surprised by this default setting and would have expected all project to be updated

[Lorilatschki]

@fgreinacher I argee what @ap0llo mentioned above. When I started configuring renovate I was also wondering why test dependencies are not updated. In my opinion I see also no reason why not to update test dependencies the same like the common once.

[BlairCurrey]

I don’t see a problem with it as I think this rule benefits the majority who don’t want to update tests/ and is easily changed by the minority who do.

What is the easy change? Ignore the ignorePaths preset and redeclare the ignorePaths with the desired item removed? That seems to be the way, although I was hoping for some way to effectively remove/override the specific item so that I don't have to now maintain this list of ignorePaths (from config:base in my case) going forward. I also found it problematic to even confirm that this was the problem - it wasn't until this post that I found what config:base consisted of. I do not see it here https://docs.renovatebot.com/presets-config/ nor did a config:base return any results in the docs.

I can understand not changing these defaults for backwards compatibility but i also expected the test directory to be included, and this is from the perspective of a TypeScript monolith.

[HonkingGoose]

I do not see it here https://docs.renovatebot.com/presets-config/ nor did a config:base return any results in the docs.

You're not seeing many hits for config:base because that's a old term. We use config:recommended starting from Renovate version 36.

[fgreinacher]

Just to see what happens to a more less standard .NET repository after removing the test entries:

All of these are useful upgrades to test dependencies.

@rarkins What do you think about my proposal above?

Could we make the ignorePaths language-specific? 🤔

[fgreinacher]

How about we add a message to the new PRs? I'm thinking of something like this maybe:

The big idea here is that we explain what we changed, and try to give people an easy way out of the new behavior.

I love this idea! Could be technically challenging though! Maybe we can somehow simplify to just indicate a major version bump N days after it has been rolled out?

[rarkins]

Users of the hosted app unfortunately don't have an easy way or learning or controlling major version bumps

[fgreinacher]

Yes, and I think @HonkingGoose proposal could improve this situation a little bit!

[rarkins]

To display that message in every PR just in case would be a mistake. It would only be feasible if we could somehow identify which users have a test/tests folder despite the fact they're currently ignored.

[fgreinacher]

It would only be feasible if we could somehow identify which users have a test/tests folder despite the fact they're currently ignored.

For this specific case yes!

But generally we could also have a tiny header in each section that goes something like this:

Renovate has recently been updated from version X to version Y. Click here to see the changelog.

Even if the breaking changes are not relevant it might be a good trigger to look at the changelog and maybe discover other helpful new features.

[fgreinacher]

I know this is very old, but I still think this is a wrong default behavior.

Not updating dependencies in these directories is comparable to not updating devDependencies for NPM packages. People can miss important updates to test frameworks, coverage tools, or security linters.

[rarkins]

It's not the default behavior: https://docs.renovatebot.com/configuration-options/#ignorepaths

It's part of config:recommended: https://docs.renovatebot.com/presets-config/#configrecommended but config:recommended is not the default preset either: https://docs.renovatebot.com/self-hosted-configuration/#onboardingconfig

It's the default behavior in the Mend hosted app, but users are free to override defaults. Self-hosted administrators are in complete control too.

Setting "defaults" (which is arguable here if it's justified to say it's even a default) is about trade-offs. Even if 90% of people prefer one way as default, you don't do that if that's going to be a problem for the other 10% and cause them to churn immediately due to misunderstandings or misgivings. In this case I think that 90% of people don't want us upgrading test fixtures, html files in test folders, etc - and they complained loudly at the time.

It could be interesting if we log all the ignored files though, so that at least people can see that specific files are ignored in debug logs

[fgreinacher]

Sorry, that was rushed a bit. Of course it's not the default, but I'd still assume that most folks use the recommended config, but I might of course be in my bubble here.

Good background knowledge with the complaints, thanks for sharing! I'm still wondering if that rule should be language-specific. Do you remember where the complaints came from back then? In NPM that might very well be a good choice. But as mentioned above typical .NET projects have their test projects (where the test/dev dependencies are stated) within a test/tests directory.

[rarkins]

Yes, it was perhaps even back when Renovate was npm-only. I'm open to pushing ignoredPaths to be manager-specific

[SeanKilleen]

Came here to follow up on this because I just ran into confusion around this "default" behavior (in the recommended config). Our .NET project chose the recommended config preset and I was surprised to learn that the nuget packages in our test projects haven't been updated in some time (assumed they were keeping pace with the rest of our projects that were receiving updates). Wasn't expecting a whole sub-tree of our project to not receive updates.

Appreciate the explanation of the history here and it makes sense 👍 I like the idea of making it manager-specific because I do think this is a place where ecosystems may naturally have different defaults.

[dynamictulip]

We've also had problems with 'ignoreModulesAndTests' being included in the config:base. Like many/most .NET projects, our unit test csprojs (which each have their own dependencies) are grouped in a folder called "test". Sometimes a dependency mismatch between prod and test .NET code causes compilation issues.

In most .NET projects in all the companies I've recently worked with, there's been a dedicated package.json for cypress/playwright ui tests and this has also been in a folder nested under the "test" folder. It is important for UI test dependencies to be kept up to date but the recommended configs would prevent this too.

It would be good to have something in the docs that says "don't use preset configs (including config:recommended and config:best-practices) for .NET repos" or even a "beware that config:best-practices and config:recommended ignores test folders". There are quite a few places in the docs that tell you to use these templates and we've had a lot of confusion caused by people trying to "fix" our renovate config by including the recommended presets without realising that there's a problem with those presets for our projects.

[github-actions[bot]]

Hi there,

Get your discussion fixed faster by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this. Discussions without reproductions are less likely to be converted to Issues.

Please follow these steps:

1. Read our guide on creating a minimal reproduction.
2. Go to our minimal reproduction template repository.
3. Select the Use this template button to create a new repository based on the template.
4. Work on the minimal reproduction in your own repository.
5. Fill out the information in your repository's README.md.
6. Add the link to your reproduction to the first post of your Discussion. If you are not the original author, you can post a new comment with the link.

Good luck,

The Renovate team

[rarkins]

I created #31146 but would like to have a reproduction repo to verify it solves the problem

[fgreinacher]

Thanks @rarkins! Here you go: https://github.com/fgreinacher/renovate-nuget-test-deps/

[HonkingGoose]

Thanks for making a reproduction for us! I've forked it into our reproduction organization, and linked to your reproduction in the PR that fixed the problem. That way we can always test if we accidentally break thing related to the fix. 😄

• renovate-reproductions/12755-and-31146