DI compilation crashes when composer classmap includes generated code

[scottsb]

Preconditions (*)

Have the following in your project composer.json as recommended since #16435 was merged:

    "psr-0": {
        "": [
            "app/code/",
            "generated/code/"
        ]
    },

Reproduced on Magento 2.3.1, but if those lines are in your composer.json file the issue will apply to many older versions as well since the relevant code hasn't changed for a long time.

Steps to reproduce (*)

Run the following commands in sequence:

1. bin/magento setup:di:compile
2. composer dump-autoload -o
3. bin/magento setup:di:compile

Note that this is a minimal sequence to reproduce the issue. Normally this would be seen with steps #1-2 run during one deploy and then step #3 run the next deploy.

Expected result (*)

Compilation succeeds both times.

Actual result (*)

Second compilation command fails with an error like this:

Warning: include(/var/www/html/vendor/composer/../../generated/code/Magento/Framework/App/ResourceConnection/Proxy.php):
failed to open stream: No such file or directory in /var/www/html/vendor/composer/ClassLoader.php on line 444

Analysis

The issue is that the core Cli module includes magic early behavior to clear out the environment when a compilation command is being run, including deleting the generated code directories (see method Cli::assertCompilerPreparation() and note it's called before Cli::initObjectManager()). However, when the object manager is then initialized it tries to load classes from the Composer classmap that no longer exist, since they were in the deleted generated directories.

The method assertCompilerPreparation() has the following comment:

    /**
     * Temporary workaround until the compiler is able to clear the generation directory
     * @todo remove after MAGETWO-44493 resolved
     */

I don't know what MAGETWO-44493 is, but it appears that the DiCompileCommand class does indeed already clear the generation directory itself (futilely since it's already cleared by that point), so the comment is at a minimum misleading.

Possible solutions:

• Refactor the magic deletions out of the generic Cli class and into the relevant command classes after the object manager is initialized. This would be ideal, but I assume part of the goal is to prevent the object manager from loading old generated code, so I'm not sure how feasible this is.
• Leave the existing magic as it is and add additional magic to remove or replace the Composer autoloader when compiling to avoid trying to access deleted generated code, thereby skipping straight to dynamic generation.

Workaround

Until this is fixed the error can be avoided by manually deleting the generated directory contents and then triggering a temporary classmap that excludes generated (either by running composer install or a simple composer dump-autoload) before running DI compilation. (Credit for the workaround and helping along my investigation to jalogut/magento2-deployer-plus#29, which I came across when Googling this problem.)

[m2-assistant]

Hi @scottsb. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.3-develop instance - upcoming 2.3.x release

For more details, please, review the Magento Contributor Assistant documentation.

@scottsb do you confirm that you were able to reproduce the issue on vanilla Magento instance following steps to reproduce?

• yes
• no

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[m2-assistant]

Hi @engcom-qa-Charlie. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 5. Verify that the issue is reproducible on 2.2-develop branch.

  Details

  - Add the comment @magento give me 2.2-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.2-develop branch, please add the label Reproduced on 2.2.x

• 6. Add label Issue: Confirmed once verification is complete.

• 7. Make sure that automatic system confirms that report has been added to the backlog.

[magento-engcom-team]

✅ Confirmed by @engcom-qa-Charlie
Thank you for verifying the issue. Based on the provided information internal tickets MC-17500, MC-17501 were created

Issue Available: @engcom-qa-Charlie, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

[fascinosum]

Hi @scottsb,
we recommend the following deployment flow:

• composer install
• bin/magento setup:di:compile
• composer dump-autoload -o

We are not going to add additional magic to handle the case without composer install

[scottsb]

@fascinosum The presence or absence of composer install isn't really related to this bug. In actual fact, we do run composer install before bin/magento setup:di:compile and have done so all along, including when we first discovered this bug. It wasn't mentioned in the issue description because I was providing only the minimal necessary steps to trigger the bug.

[fascinosum]

HI @scottsb,
we are clearly understand what this issue is about. You are absolutely right, if we use the commands from the description, we will encounter the error. Please use the following order of commands for each you deployment

• composer install
• bin/magento setup:di:compile
• composer dump-autoload -o

As you know, composer dump-autoload -o is not a part of any Magento CLI command. If you use this instruction, you should clear composer cache using composer install before the bin/magento setup:di:compile command.
Please let us know if the issue is still reproducible with the recommended flow