Allow only valid preferences during setup:di:compile

[fredden]

Description

The setup:di:compile command has exclude lists to avoid loading / compiling dependency injection for 'test' classes. When preferences exist for these excluded classes, any plugins associated with the original classes are (potentially) rendered useless. This is because the child class (preference) can call the original class in a way which does not use Magento's plugin system (ie, parent::methodName()). And it's impossible to plugin a class that does not go through Magento's compilation process (to have interceptors created, etc). See magento/security-package#296 for a real-world example of this problem in action.

This pull request began as a change to the regular expressions used to exclude classes, and has grown into expanding the exclude list to cover preferences. Changes include:

• Removal of invalid preferences already within the code-base.
• Alerting (exception) for preferences that do not exist.
• Alerting (exception) for preferences that do not exist because of the existing exclude lists.
• Various bug-fixes for setup:di:compile-related classes.

Related Pull Requests

• Fix invalid preferences security-package#297
• Remove invalid preferences inventory#3315
• Move preference to area where class exists magento2-page-builder#743
  • Which Adobe duplicated to: https://github.com/magento-commerce/magento2-page-builder/pull/304

Intentionally not using the magic comment which links pull requests together within the Adobe testing system, because these pull requests are related but not part of this change. Do not reinstate that magic comment.

Fixed Issues (if relevant)

1. Unable to Plugin to \Magento\TwoFactorAuth\Observer\ControllerActionPredispatch security-package#296
2. [Issue] Allow only valid preferences during setup:di:compile #38517

Manual testing scenarios

1. Create a preference for a class which does not exist. The "for" class can exist or not; the "type" class should not exist.
2. Create a preference for a class which is specifically excluded (like a "test" class). The "for" class can exist or not; the "type" class should exist, but not be eligible for dependency compilation - for example if the path includes Test.

Before this pull request, these would silently fail; after this pull request, an error is shown.

Questions or comments

None

Contribution checklist

• Pull request has a meaningful description of its purpose
• All commits are accompanied by meaningful commit messages
• All new or changed code is covered with unit/integration tests (if applicable)
• README.md files for modified modules are updated and included in the pull request if any README.md predefined sections require an update
• All automated tests passed successfully (all builds are green)

Resolved issues:

1. resolves [Issue] Allow only valid preferences during setup:di:compile #38517: Allow only valid preferences during setup:di:compile

[m2-assistant]

Hi @fredden. Thank you for your contribution
Here are some useful tips how you can test your changes using Magento test environment.
Add the comment under your pull request to deploy test or vanilla Magento instance:

• @magento give me test instance - deploy test instance based on PR changes
• @magento give me 2.4-develop instance - deploy vanilla Magento instance

❗ Automated tests can be triggered manually with an appropriate comment:

• @magento run all tests - run or re-run all required tests against the PR changes
• @magento run <test-build(s)> - run or re-run specific test build(s)
  For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names. Allowed build names are:

1. Database Compare
2. Functional Tests CE
3. Functional Tests EE,
4. Functional Tests B2B
5. Integration Tests
6. Magento Health Index
7. Sample Data Tests CE
8. Sample Data Tests EE
9. Sample Data Tests B2B
10. Static Tests
11. Unit Tests
12. WebAPI Tests
13. Semantic Version Checker

You can find more information about the builds here

ℹ️ Please run only needed test builds instead of all when developing. Please run all test builds before sending your PR for review.

For more details, please, review the Magento Contributor Guide documentation.

⚠️ According to the Magento Contribution requirements, all Pull Requests must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of Pull Requests happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

🎥 You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[ihor-sviziev]

Hi @fredden,
What is a use case for writing plugin on a class that should be used only in tests?

[fredden]

I can't think of a good reason to create a plugin for a class that should only be used in tests. Perhaps the change here should be to exclude Preferences that match these regular expressions as well. I'll investigate further to see if that's something we can include here.

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[fredden]

They should be merged in separately on their own merits

I have edited the pull request description to remove the magic comment which links pull requests together in the Adobe testing system.

[fredden]

@magento run all tests

[engcom-Hotel]

Hello @fredden,

As per the comment below, it seems that PageBuilder is a prerequisite for this PR. Therefore, we are placing this PR on hold for now and will proceed with it once the PageBuilder PR is merged:

I think the main problem with this situation is that the pre-requisites are being combined with this work. They should be merged in separately on their own merits. When they're merged in, I expect the tests to pass here.

Thanks

[fredden]

@engcom-Hotel the PageBuilder changes have been merged in. Please can you organise for magento/inventory#3315 to be merged in too?

[engcom-Hotel]

Yes @fredden, I am on it. I will update you soon regarding the inventory PR.

[fredden]

@magento run all tests

[engcom-Hotel]

@magento run all tests

[fredden]

It looks like the prerequisite changes have been applied now. @engcom-Hotel please can you remove the 'on hold' label here and progress this through the process.

[engcom-Hotel]

We are moving it to Extended Testing for further processing.

Thanks

[engcom-Hotel]

@magento run Functional Tests EE

[engcom-Hotel]

Failed tests seem falky to me. The last 2 consecutive runs generate different errors. Hence moving this PR to Merge in Progress:

[engcom-Dash]

Moving this PR to extended testing since the copyright tag has not been updated in the mentioned files.

[engcom-Dash]

@magento run all test

[magento-automated-testing]

Failed to run the builds. Please try to re-run them later.

[engcom-Dash]

@magento run all tests

[engcom-Dash]

Got a green build so moving it to merge in progress.

[ihor-sviziev]

@engcom-Dash
Wow. Is it possible again to move PR from "merging in progress" to a real merge within four days? Great! Nice job!