Base URL placeholder defaults in multi-website configuration pollute config cache for one another

[danemacmillan]

Preconditions and environment

• Magento 2.4.3-p3
• PHP 7.4.33
• Config cache enabled: magento cache:enable config
• Magento Configured with two Web Sites, with codes base and base2
  • Each Web Site with one Store each, using codes store1 and store2.
    • Each Store with one Store View each, using codes en1 and en2.
      • Summary: base/store1/en1, base2/store2/en2.
• Nginx with fastcgi_param MAGE_RUN_TYPE website and each host configured with their appropriate website code: fastcgi_param MAGE_RUN_CODE base and fastcgi_param MAGE_RUN_CODE base2.

Steps to reproduce

With two Web Sites configured, now set their base URLs using the default placeholders, so that an explicit domain does not need to be configured, which will allow the deployment to exist at any domain, so long as it points to the correct deployment, as noted here:

{{base_url}} which represents a base URL defined by a virtual host setting or by a virtualization environment like Docker. For example, if you set up a virtual host with the hostname magento.example.com, you can install the software with --base-url={{base_url}} and access the Admin with a URL like http://magento.example.com/admin.

This is also noted in the Admin at Stores > Settings > Configuration > General > Web > Base URLs, should they want to set the URL values at each Web Site scope (not default).

To simplify the process, just use the commands to set the URLs.

Web Site code: base

magento config:set web/unsecure/base_url {{base_url}} --scope=website --scope-code=base
magento config:set web/unsecure/base_link_url {{unsecure_base_url}} --scope=website --scope-code=base
magento config:set web/unsecure/base_static_url "" --scope=website --scope-code=base
magento config:set web/unsecure/base_media_url "" --scope=website --scope-code=base

magento config:set web/secure/base_url {{base_url}} --scope=website --scope-code=base
magento config:set web/secure/base_link_url {{unsecure_base_url}} --scope=website --scope-code=base
magento config:set web/secure/base_static_url "" --scope=website --scope-code=base
magento config:set web/secure/base_media_url "" --scope=website --scope-code=base

Web Site code: base2

magento config:set web/unsecure/base_url {{base_url}} --scope=website --scope-code=base2
magento config:set web/unsecure/base_link_url {{unsecure_base_url}} --scope=website --scope-code=base2
magento config:set web/unsecure/base_static_url "" --scope=website --scope-code=base2
magento config:set web/unsecure/base_media_url "" --scope=website --scope-code=base2

magento config:set web/secure/base_url {{base_url}} --scope=website --scope-code=base2
magento config:set web/secure/base_link_url {{unsecure_base_url}} --scope=website --scope-code=base2
magento config:set web/secure/base_static_url "" --scope=website --scope-code=base2
magento config:set web/secure/base_media_url "" --scope=website --scope-code=base2

Ensure at the very least the config cache is enabled: magento cache:enable config. Everything else can be off.

With Nginx (or Apache) configured with the relevant MAGE_RUN_TYPE and MAGE_RUN_CODE per domain virtual host, navigate to the either of the domains first. For example purposes, base will respond to domain https://base.test/en1/ and base2 will respond to domain https://base2.test/en2/.

The first domain loads fine, with all its static and media assets also loading from the same base URL of https://base.test/, being https://base.test/media/ and https://base.test/static/.

Now navigate to the second domain. It loads fine, except that all of its static and media assets are not loading from https://base2.test/, but are loading from the first domain, which obviously causes numerous CSP and CORS errors, while additionally all of the links on the site are pointing to https://base.test/, which means the moment anyone interacts with the second site, they are brought to the first one.

Disabling the config cache eliminates this problem.

Worthwhile debugging notes

• Setting the base_static_url and base_media_url configs with placeholder values {{unsecure_base_url}}static/ and {{unsecure_base_url}}media/ does not remedy the issue. The problem is the same. For simplicity, they have been left empty, as they are optional anyway.

• Explicitly setting all of the URLs to real values instead of using the placeholders DOES NOT cause this problem. Both Web Sites load fine, respecting their specific URLs and not attempting to load from the other. The static and media URLs will load from their explicitly configured URLs, and not pollute each other's config cache values. In fact, the only reason I discovered this problem is because I was reviewing how to simplify deployments on different domains while still sharing a common DB without having to explicitly set domains in the DB. That was when I read that even the Base URL can be set to {{base_url}}, which would mean the domain that gets used is whatever is configured in the server's virtual host (Nginx or Apache). While it works for the Base URL, the static and media URLs pollute each other, based on whichever is accessed first.

Expected result

After navigating to https://base.test/en1/, navigating to https://base2.test/en2/ should load all of its static and media assets from its own base URL of https://base2.test/.

Actual result

Instead, what happens is that navigating to https://base2.test/en2/ will have all of its static and media assets loaded from the Base URL of the first Web Site that was accessed, being https://base.test/.

Additional information

Note that a similar problem was documented in #10693, though there was no mention of whether they also set the static and media URLs. The probably didn't. In all likelihood, the deployments that the user was making were single Web Site configurations, which would not have this problem, as there would be no other Web Site Base URLs polluting the Base URLs of the config cache.

Release note

No response

Triage and priority

• Severity: S0 - Affects critical data or functionality and leaves users without workaround.
• Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
• Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
• Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
• Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.

Edit 1: Research

This method gets called on all configs within all scopes, regardless of the Store View being loaded:

magento2/app/code/Magento/Store/Model/Config/Processor/Placeholder.php

Line 35 in 5f07eba

public function process(array $data)

It then eventually calls a method to process all placeholders, which then replaces all the {{base_url}} and similar URL placeholders with the value detected from the request via this method:

magento2/app/code/Magento/Store/Model/Config/Placeholder.php

Line 126 in 5f07eba

$distroBaseUrl = $this->request->getDistroBaseUrl();

What that means is that every Web Site-scoped config is replaced with the currently discovered HTTP_HOST, which is erroneous. The current Web Site scope would need to be known before attempting to replace placeholders, and the placeholder replacements themselves should only be done on the configs for the current Web Site scope, not all of them, as is being done now. What this means is that when the config cache is enabled, the moment the first Web Site within a multi-website configuration is hit, the domain for that site is used for every other Web Site scope config, and then saved to the cache. This happens because the code does not care about what scope the placeholder replacements should be made; instead it simply replaces all of them, regardless of scope. That means when the next site is loaded from a different domain, all of its Link, Static, and Media URLs have already been generated by the placeholder replacement operation from the first load of the other site.

In summary, what this means is that the URL placeholder replacement logic runs way too early on the configs, and instead needs to be run only once scopes are known, and only run under the currently active scope.

Also, given how early in the process URL placeholders are being replaced, it means, for example, that code like this will never be run, as replacements will have already occurred (keep in mind that this section would not address the other URL placeholders, though, even if they managed to get to this section:

magento2/app/code/Magento/Store/Model/Store.php

Line 669 in 5f07eba

if (false !== strpos($url, self::BASE_URL_PLACEHOLDER)) {

Edit 2: Research

Based on git logs, much of this functionality was rewritten in a very large commit (a978a4c) by Sergii Kovalenko, which may be the source of the problem.

[m2-assistant]

Hi @danemacmillan. Thank you for your report.
To speed up processing of this issue, make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, Add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release

For more details, review the Magento Contributor Assistant documentation.

Add a comment to assign the issue: @magento I am working on this

To learn more about issue processing workflow, refer to the Code Contributions.

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

⚠️ According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, join the Community Contributions Triage session to discuss the appropriate ticket.

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

[m2-assistant]

Hi @engcom-November. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.4-develop branch

  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 5. Add label Issue: Confirmed once verification is complete.

• 6. Make sure that automatic system confirms that report has been added to the backlog.

[engcom-November]

@magento give me 2.4-develop instance

[magento-deployment-service]

Hi @engcom-November. Thank you for your request. I'm working on Magento instance for you.

[magento-deployment-service]

Hi @engcom-November, here is your Magento Instance: https://882b1fcb1582f0688f079b4872326b69.instances.magento-community.engineering
Admin access: https://882b1fcb1582f0688f079b4872326b69.instances.magento-community.engineering/admin_d356
Login: 4a348d08
Password: acd23f1b1221

[engcom-November]

Hi @danemacmillan ,
Thank you for reporting and collaboration. Verified the issue on Magento 2.4-develop instance but unable to reproduce the issue with below steps performed:
Steps performed:

1. Enabled config cache: magento cache:enable config

2. Stores setup as shown below:
   

3. Apache server with virtual host configured as below:
   

4. Set the urls for both the website scopes as shown below:
   
   Note: static and media urls left blank

Getting 404 not found error for both the website scopes when trying to access : http://mg24.local/static, http://mg24.local/media
http://en.mg24.local/static, http://en. mg24.local/media

Kindly recheck the behavior on Magento 2.4-develop instance and elaborate missing steps / configuration details if the issue is still reproducible.
Thank you.

[danemacmillan]

Hi,

There are several problems with your attempt to reproduce. Mainly, they are just not following the steps mentioned in the original report.

1. Good.
2. Good.
3. Bad. You need to have 2 virtual hosts, one for each Web Site, each specifying their unique Web Site code, which would mean one virtual host for base and a second virtual host for base2. Additionally, they need to have unique server names (domain names); they cannot be the same. Give them a unique name each. In your case, I would use base.mg24.local for the virtual host that corresponds to the base Web Site scope code, and base2.mg24.local for the virtual host that corresponds with the base2 Web Site scope code.
4. Bad. Based on screenshots, you're only setting the Base URL for one scope (and probably for the "Default Config" scope, which is erroneous). Secure and unsecure URLs are not scopes. You need to switch to the "Main Website" Web Site scope (code being base) in the scope changer located at the top left of the admin (see image a), then set the Base URL, Base Link URL, Secure Base URL, and Secure Base Link URL; then you need to do the same thing for the "base2" Web Site scope (code being base2). Additionally, and more importantly, the whole point of this ticket is that the placeholders don't work: you're not using placeholders, you're explicitly setting a domain in the admin config. To repeat what was mentioned in the original ticket, you need to set the URLs with the placeholder values that Magento supports. That would look like this for each Web Site scope:

base:

• Base URL: {{base_url}}
• Base Link URL: {{unsecure_base_url}}
• Secure Base URL: {{base_url}}
• Secure Base Link URL: {{unsecure_base_url}}

base2:

• Base URL: {{base_url}}
• Base Link URL: {{unsecure_base_url}}
• Secure Base URL: {{base_url}}
• Secure Base Link URL: {{unsecure_base_url}}

You literally save the placeholder values (see image b). This was explicitly written in the original issue, with the commands to execute to do this.

Lastly, you don't go visit the static URLs directly, though the problem will likely exist there as well. You visit the site itself, which will in turn build all the link, static, and media URLs and then cache them. You having a 404 has nothing to do with the issue, and only to do with your own misconfigured machine.

References:

a) Note that I recreated what your scope changer menu would look like, based on the Web Sites you displayed in your screenshot.

b)

--

To be clear, I wrote all of this in my original ticket. Yes, it's a complex set of steps, but I've merely repeated myself here, but will happily continue to guide you until you can reproduce.

[engcom-November]

Thank you for the response @danemacmillan Verified the issue again on Magento 2.4-develop instance with updated steps and the issue is reproducible. Hence updating the description and confirming the issue.