Storing sessions in Redis causes longer loads

[vrielsa]

Preconditions (*)

1. Magento Community Edition 2.4.1
2. store sessions with redis
3. store cache with redis
4. varnish as fpc

Steps to reproduce (*)

This is tricky, I haven't found a way to consistently reproduce this issue. But the behaviour seems a lot like what PR #22829 fixes. After receiving feedback from a customer we noticed that some steps in the checkout takes quite a bit of time, when profiling the checkout with Blackfire we noticed that some end points (f/e estimate-shipping-methods) sometimes take over 4s to complete.
It looksl like to be coming from the class Magento\Framework\Session\Savehandler\Redis::read() method, which calls Cm\RedisSession\Handler::read. In the latter it uses usleep to wait for locks. Which on occasion takes 2s or more.

session config in env.php
'session' => [ 'save' => 'redis', 'redis' => [ 'host' => 'x.x.x.x', 'port' => 'xxxx', 'password' => '', 'timeout' => '2.5', 'persistent_identifier' => '', 'database' => '1', 'compression_threshold' => '0', 'compression_library' => 'gzip', 'log_level' => '2', 'max_concurrency' => '6', 'break_after_frontend' => '5', 'break_after_adminhtml' => '30', 'first_lifetime' => '600', 'bot_first_lifetime' => '60', 'bot_lifetime' => '7200', 'disable_locking' => '0', 'min_lifetime' => '60', 'max_lifetime' => '2592000' ] ],

Expected result (*)

Less lockout time

Actual result (*)

Lockouts +2s

---

Please provide Severity assessment for the Issue as Reporter. This information will help during Confirmation and Issue triage processes.

• Severity: S0 - Affects critical data or functionality and leaves users without workaround.
• Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
• Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
• Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
• [] Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.

[m2-assistant]

Hi @vrielsa. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release

For more details, please, review the Magento Contributor Assistant documentation.

Please, add a comment to assign the issue: @magento I am working on this

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

⚠️ According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

🎥 You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

[TimQSO]

Read this:

https://objectsource.co.uk/magento-2-performance-issue-with-sessions/

BTW I used Redis with locking disabled for almost a year now, including Black Friday and no issues whatsoever. It is much faster with locking disabled and there are no race conditions to be afraid of.

[vrielsa]

@GuiltyNL thank you, we've found that article as well.
We've disabled locking, which helps lots indeed.

[m2-assistant]

Hi @engcom-Bravo. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.4-develop branch

  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 5. Add label Issue: Confirmed once verification is complete.

• 6. Make sure that automatic system confirms that report has been added to the backlog.

[m2-assistant]

Hi @engcom-Echo. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.4-develop branch

  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 5. Add label Issue: Confirmed once verification is complete.

• 6. Make sure that automatic system confirms that report has been added to the backlog.

[ZIPITRO]

@GuiltyNL Indeed we use this also for long! No side effects :-)

[engcom-Echo]

Hi @vrielsa ,

It would be really helpful, if you can provide us with more information regarding the steps to re-produce the issue
Also as mentioned in comment it looks disable locking in redis has increased the performance.

Thanks

[engcom-Hotel]

Hello @vrielsa,

It seems that your issue has been resolved by disable locking in Redis. And also we have not received any update from your side from past 14 days, so we are closing this issue for now.

If you still facing any issue, either you can reopen this ticket or create new ticket.

Thanks

[m2-assistant]

Hi @engcom-Hotel. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.4-develop branch

  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 5. Add label Issue: Confirmed once verification is complete.

• 6. Make sure that automatic system confirms that report has been added to the backlog.

[ioweb-gr]

@engcom-Hotel it's my understanding that disabling the locking mechanism can result in race conditions and severe session data loss in high traffic websites. Why did you close this issue when in reality disabling the locking mechanism is a workaround to the real issue and not a solution? The underlying issue is that multiple simultaneous requests like ajax requests on the checkout will cause huge locks on the session, that can span over seconds causing huge delays in page loads.

We actually face the same issue and trying to find an alternative to disabling the locking mechanism.

Would you be so kind as to reopen the issue?

Moreover, I notice this can happen if using files for session too because php will by default lock the session files also.

An example here

We get similar hanging profiles with REDIS and FILES as session storage.

[TimQSO]

@ioweb-gr I can confirm that there are no issues at al when you disable the locking. I've tested it during Black Friday resulting in 180 to 300 orders per hour. With no issues at all.

[ioweb-gr]

@GuiltyNL I've actually had issues when disabling session locking \Magento\Checkout\Model\Session:getLastRealOrderId() would sometimes return null after redirection
It didn't happen with files storage or with locking enabled :(

It may be something else causing but just saying that the default behaviour is faulty, it doesn't mean we should resort to a workaround as it might not work in all cases so in my opinion this shouldn't be closed.

[TimQSO]

:( Shoot! Strange that it is running without issues on our server. :(

[ioweb-gr]

yeah I know it's a bummer :( I thought it would solve it quickly for us too

[bjornmeat]

@GuiltyNL Have you tried tuning the parameters for:

session-save-redis-max-concurrency

https://devdocs.magento.com/guides/v2.4/config-guide/redis/redis-session.html

[ioweb-gr]

The recommendation is to set to 10% of the php processes of the server. For me with about 90 fpm processes concurrently I raised from 6 to 9

Then just for kicks raised to

15 and 30. The issue remains, in certain page loads it would take 1.5 to 5 seconds longer waiting on the redis session to start. I profiled with blackfire and other profilers continuously

I could only mitigate this by offloading session storage to the DB. But this increased mysql CPU usage a lot.

[m2-assistant]

Hi @engcom-Echo. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
     Details

     If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.
• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.
• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.
• 4. Verify that the issue is reproducible on 2.4-develop branch
     Details

     - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
     - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
     - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[engcom-Echo]

Hi @vrielsa,

We tried again to reproduce it on Magento 2.4.1/2.4-dev but the issue is not reproducible on the local system as we are not able to generate the required traffic. But as given in the ticket we have analysed the Redis session code file (src/Cm/RedisSession/Handler.php, line no 554). After analysis, I found the following code which is the possible reason for the delay in request also it is logging the wait lock activity as well.

$this->_log( sprintf( "Waiting %.2f seconds for lock on ID %s (%d tries, lock pid is %s, %.5f seconds elapsed)", $sleepTime / 1000000, $sessionId, $tries, $lockPid, (microtime(true) - $timeStart) ) ); usleep($sleepTime);

By looking at the code and its implementation confirming this issue.

Thank You

[github-jira-sync-bot]

✅ Jira issue https://jira.corp.magento.com/browse/AC-2062 is successfully created for this GitHub issue.

[m2-assistant]

✅ Confirmed by @engcom-Echo. Thank you for verifying the issue.
Issue Available: @engcom-Echo, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

[ga3003]

Is there any update about this issue? I am using one custom payment method which redirects to the bank page for payment. The module used afterPlaceOrder method to redirect the customer to the bank page and created a controller where they are trying to get orderId using checkoutSession->getLastRealOrderId() function. After debugging, I found that this orderId is undefined.

FYI, we are using Redis for caching and the module has disabled FPC from the layout of the page by making block cacheable false.

[TimQSO]

@GuiltyNL Have you tried tuning the parameters for:

session-save-redis-max-concurrency

https://devdocs.magento.com/guides/v2.4/config-guide/redis/redis-session.html

Yes, no difference in performance.

[Adel-Magebinary]

Pretty sure this is happening to most of our clients under high load.

[Yonn-Trimoreau]

I've come up with an acceptable solution for this: colinmollenhour/php-redis-session-abstract#50

To temporarily fix this issue, here are the patches to apply:

On colinmollenhour/php-redis-session-abstract module => implement-write-lock-and-diff-patch.txt

And on magento/framework module => set-php-serialize-handler-patch.txt

If you want the full story and a potential definitive fix: #34758 (comment)