Stache parallel warming: does it actually help?

[o1y]

I’ve been taking a closer look at the new Stache warming config introduced in #12894. Really appreciate the work on this 🙌

After digging into the implementation / benchmarking it on a few real sites I might be missing something, but I’m currently not seeing meaningful improvements in warming times for larger collections..

Some details about my Statamic site:

• A multi-site with around 8000 entries across multiple collections
• 6300 assets
• Plus a few global sets, navs etc.

Benchmark for stache:warm:

Mode	Time	CPU
Sequential	56.8s	102%
Parallel with process driver	57.9s	109%

Enabling parallel warming (via STATAMIC_STACHE_PARALLEL_WARMING=true) made it always a bit slower 🥲 I monitored the processes during parallel warming. 13 spawn, 12 finish in 0.4s and 1 process runs for the remaining seconds.

15:04:47.271    0  <- warming starts
15:04:47.558    0
15:04:47.856   13  <- all child processes spawn
15:04:48.211    1  <- 12 finish within 0.4s
15:04:48.503    1  <- one process grinds for ~55 seconds
15:04:48.803    1
...

The real work is probably for most real-world sites inside a single CollectionEntriesStore, which processes in my case all 8000 files sequentially.

I’m struggling to think of a realistic scenario where the new config options would make a difference. The only potentially expensive stores seem to be entries and maybe terms.. Would it make more sense to explore parallelism within a store instead (e.g. chunking 8000 entries across cores for YAML parsing)?

Curious to hear the team’s thinking here, maybe parallel warming was meant to address a different use case? :)

[o1y]

I went a bit further down the rabbit hole and profiled where the warming time is actually spent. The majority of the Stache warming time appears to be really in YAML parsing. When I switch from Symfony’s YAML to the PECL yaml extension, warming time drops from 56.8s to 22.18s in my setup 🤯

Unfortunately, we cannot just swap the parser since the PECL extension only supports YAML 1.1. Concurrent YAML parsing would likely have a much larger impact on Stache warming than the current store-level parallelism approach

---

If you want to try this with pecl yaml change in https://github.com/statamic/cms/blob/6.x/src/Yaml/Yaml.php#L63

- $yaml = $this->yaml->parse($str);
+ $yaml = @yaml_parse($str);