You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Publish recursive performance seems to have some gaps. We started replacing this method with something more performant and also by using asynchronous processing such as queued jobs. Most notable issues we found are:
long execution time
causing DB write issues such as timeouts and deadlocks (likely caused by the item above)
I've performed a benchmark on a content page that contains 21 content blocks, some with nested models and some with assets.
Benchmark for standard publishRecursive()
5.33 seconds - First time publish (no models have live version)
4.02 seconds - Single model modified on draft & published via page
Benchmark for customised publish action
3.03 seconds - First time publish (no models have live version)
1.10 seconds - Single model modified on draft & published via page
We found that the customised approach is much more resilient to DB deadlocks too. Processing thousands of queued jobs that execute this type of publish is fast and has almost no issues.
The performance gap is even wider if you go to pages with 100+ content blocks and if you introduce parallel processing via queued jobs.
How to reproduce
<?phpnamespaceApp\Tasks;
useApp\Pages\FeaturePage;
useException;
usePage;
useSilverStripe\Control\HTTPRequest;
useSilverStripe\Dev\BuildTask;
useSilverStripe\ORM\DataObject;
useSilverStripe\ORM\FieldType\DBDatetime;
useSilverStripe\Versioned\RecursivePublishable;
useSilverStripe\Versioned\Versioned;
useTractorCow\Fluent\Model\Locale;
useTractorCow\Fluent\State\FluentState;
classPerformantPublishTaskextendsBuildTask
{
privatestaticstring$segment = 'performant-publish-task';
/** * @var string */protected$title = 'Publish recursive benchmark task';
/** * @var string */protected$description = 'Benchmark the performance of publish recursive and potentially provide better options.';
/** * @param HTTPRequest|mixed $request */publicfunctionrun(mixed$request): void
{
$type = $request->getVar('type');
FluentState::singleton()->withState(function (FluentState$state) use ($type): void {// Comment out if your project doesn't have Fluent installed$defaultLocale = Locale::getDefault();
// Optionally set locale if configuredif ($defaultLocale) {
$state->setLocale($defaultLocale->Locale);
}
echo'Fetching page...' . PHP_EOL;
/** @var FeaturePage $page */$page = FeaturePage::get()->byID(928);// Add your page type and ID$performantPublish = ($type === 'experimental');
echosprintf('Running publish with type %s...', $performantPublish ? 'experimental' : 'standard') . PHP_EOL;
$start = microtime(true);
if ($performantPublish) {
$this->performantPublishRecursive($page);
} else {
$page->publishRecursive();
}
// Reformat to seconds$end = microtime(true) - $start;
echosprintf('%.2f', $end) . PHP_EOL;
});
}
/** * @param Page $page * @return void * @throws Exception */privatefunctionperformantPublishRecursive(Page$page): void
{
$now = DBDatetime::now()->Rfc2822();
DBDatetime::withFixedNow($now, function () use ($page) {
$modelsToPublish = [
$page,
];
/** @var DataObject|Versioned|RecursivePublishable $model */while ($model = array_shift($modelsToPublish)) {
// We don't have anything to publish on a non-versioned modelif (!$model->hasExtension(Versioned::class)) {
continue;
}
// Publish the current model if neededif (!$model->isPublished() || $model->stagesDiffer()) {
$model->publishSingle();
}
// Queue up owned models for publishing$ownedModels = $model
->findOwned(false)
->toArray();
$modelsToPublish = array_merge($modelsToPublish, $ownedModels);
}
});
}
}
Possible Solution
The provided code snippet is quite crude but it shows the opportunity for some options that might help with the performance of publishRecursive(). Here is a list of recommendations:
avoid using true recursion, the base performance of this is poor, use stack based recursion instead
add configuration to opt out of with translation wrapper, this might not be needed in some cases, if publishing of assets is involved the operation might be really slow due to slow shared drive on some infrastructure setups, translation wrap might cause DB deadlock and timeouts in such cases
add configuration to skip on using ChangeSet, this feature seems to have only very marginal use (mostly related to campaign admin) and it also produces DB records that have to be cleaned up regularly, versioned history seems to be working fine without it due to with fixed now
Validations
Check that there isn't already an issue that reports the same bug
Double check that your reproduction steps work in a fresh installation of silverstripe/installer (with any code examples you've provided)
The text was updated successfully, but these errors were encountered:
Module version(s) affected
1.13.7
Description
Publish recursive performance seems to have some gaps. We started replacing this method with something more performant and also by using asynchronous processing such as queued jobs. Most notable issues we found are:
I've performed a benchmark on a content page that contains 21 content blocks, some with nested models and some with assets.
Benchmark for standard publishRecursive()
5.33
seconds - First time publish (no models have live version)4.02
seconds - Single model modified on draft & published via pageBenchmark for customised publish action
3.03
seconds - First time publish (no models have live version)1.10
seconds - Single model modified on draft & published via pageWe found that the customised approach is much more resilient to DB deadlocks too. Processing thousands of queued jobs that execute this type of publish is fast and has almost no issues.
The performance gap is even wider if you go to pages with 100+ content blocks and if you introduce parallel processing via queued jobs.
How to reproduce
Possible Solution
The provided code snippet is quite crude but it shows the opportunity for some options that might help with the performance of
publishRecursive()
. Here is a list of recommendations:with translation
wrapper, this might not be needed in some cases, if publishing of assets is involved the operation might be really slow due to slow shared drive on some infrastructure setups, translation wrap might cause DB deadlock and timeouts in such caseswith fixed now
Validations
silverstripe/installer
(with any code examples you've provided)The text was updated successfully, but these errors were encountered: