N°9940 - Setup cache reset races with read-only UI requests producing corrupt expression-cache PHP files

[hakonharnes]

Description

During setup, users may continue browsing iTop in read-only mode. Setup repeatedly calls MetaModel::ResetAllCaches(), which removes data/cache-production, while concurrent UI requests run ExpressionCache::Warmup() and include generated expression-cache PHP files.

We observe two symptoms:

1. Missing-directory warnings for multiple languages:

   PHP Warning: file_put_contents(.../data/cache-production/expressioncache/expressioncache-JAJP.php):
   Failed to open stream: No such file or directory
   

2. More seriously, an incomplete generated PHP cache file is sometimes included, causing a ParseError, HTTP 500, and occasionally halting setup.

LOCK_EX only coordinates cooperating writers. Expression-cache readers use include_once without a shared lock, and setup can delete the cache directory between its creation and file_put_contents().

Are you willing to create a PR?

Yes.

iTop version

3.2.3-1

PHP version

PHP 8.3.6

Reproduction procedure

1. Start setup/update on an iTop Community installation.
2. Accept read-only mode.
3. From another authenticated session, repeatedly browse or refresh back-office pages while setup is running.
4. Concurrent UI requests start the MetaModel and warm/read expression caches while setup resets cache-production.
5. Observe missing-directory warnings for several language cache files.
6. Intermittently, observe a PHP syntax error from an incomplete expressioncache-*.php file. This can return HTTP 500 and halt setup.

Expected: Users browsing in read-only mode do not interfere with setup.

Actual: Setup and UI requests concurrently delete, generate, and include the same executable cache files.

No custom data-model behavior appears necessary to trigger the underlying race.

Additional information

We appear to have the same issue described in this SourceForge report, including varying language files and line numbers:

PHP Parse error: Unclosed '(' on line 7 in .../expressioncache-ENGB.php on line 42

That reporter also observed a syntax error containing unexpected serialized expression content such as O:21:"Func", suggesting that the file was read before generation completed. We observe the same class of intermittent syntax failure during setup.

A previous change, "Prevent race conditions when rebuilding cache", added LOCK_EX, but this does not coordinate readers or protect against deletion of the parent directory.

Possible fixes

1. Atomic cache publication and isolation

• Publish generated caches atomically using a temporary file and rename().
• Treat missing or invalid cache entries as cache misses instead of fatal errors.
• Catch ParseError from damaged legacy cache files and regenerate them.
• Build setup caches separately and avoid deleting the cache generation currently serving users.

This could also cover router, icon, and dictionary JavaScript caches.

2. Disable the expression cache during read-only mode

Keep the current setup architecture, but disable expression-cache reads and writes while READONLY_MODE_FILE exists:

• Make GetCachedExpression() fall back to normal on-demand calculation.
• Skip ExpressionCache::Warmup().
• After the final cache reset, force-warm the cache before leaving read-only mode.

This is limited to the expression cache; read-only requests would be slightly slower during setup.

3. Extend the existing maintenance window

Keep read-only browsing during safe preparation, but do not exit maintenance mode immediately after compilation:

open
  -> read-only: copy / backup / validation
  -> maintenance: compile / schema / data / config / final cache reset / warmup
  -> open

• Enter maintenance before compiling the production model, as setup already does.
• Remain in maintenance through database-schema updates, data loading, configuration creation, final cache reset, and warmup.
• Exit maintenance only after setup is complete.

Would you prefer one of these approaches, or a different solution? Thanks.

[Rhuiedan]

Any update on this? Also having issues with the exact same.

[12:51:06] PHP Parse error: syntax error, unexpected string content "Pri", expecting ")"
in /var/www/html/itop/data/cache-production/expressioncache/expressioncache-PTBR.php on line 218
referer: http://***********/itop/setup/wizard.php

[jf-cbd]

Hello @hakonharnes, thank you for your issue and for proposing solutions. I just created the ticket internally (N°9940), and we also got this problem recently.
We should start working on it from September - our team is reduced during the summer break.
I'll keep you updated about our choice to solve it internally or to accept a PR on it.
@Rhuiedan we'll keep you updated as well; thank you for bringing it to our attention.