Reduce CI duration

[mattwang44]

Since we currently run a complete HTML build process in CI, the CI has an unacceptable duration (~16minutes).

Possible solutions:

• Cache CPython directory with actions/cache.
• Avoid the full building process in CI.

[cschan1828]

Now even a single CI workflow takes around 27 minutes.

In my opinion, we can consider checking .rst files using sphinx-build -b gettext when running CI. On the other hand, CD should be kept as unchanged as it requires full html build.

[cschan1828]

After investigating the Makefile, I found it is more practical to change the build target to "text". It is not elegant because it occupies extra space for text files. However, it effectively reduces the validation time. In the future we can further optimizations.

[ezio-melotti]

Is this issue specific to the zh-tw translations? Have you checked if other Python translation projects have the same issue, and if/how they fixed it?

[mattwang44]

CPython repo has a Makefile for doc build, and all translation repos that run doc build with GitHub Action depend on it. It provides different build modes and we currently setup our CI action to run a full doc build with the autobuild-dev-html mode, which is the sphinx-native html mode with -E (always rebuild completely) and -a (write all output files) options. We've tried out text & html mode locally and they do much faster than autobuild-dev-html, and that's why we believe the -Ea option increase the duration and changing the build mode would probably be a decent solution for us.

Here are some metrics from those translation repos that have adopted GitHub Action. French repo also builds with autobuild-dev-html but I have no idea why their actions are still blazing fast like the regular html mode...

language	build mode in GitHub Action	translation progress (calculated with potodo)	GitHub Action appr. duration (min)
Spanish	html	95.73% @3.11	4 - 7
Simplified Chinese	html	81.16% @3.10	7 - 9
Turkish	html	1.27% @3.11	4 - 5
Brazilian Portuguese	html	43.46% @3.12	5 - 6
French	autobuild-dev-html	48.14% @3.11	2 - 4
Japanese	html	77.09% @3.11	7 - 9
Polish	html & latex	4.2% @3.12	2 - 3 for html part
Ukrainian	html & latex	84.48% @3.12	3 - 5 for html part
Traditional Chinese	autobuild-dev-html	28.34% @3.11	~25

[cschan1828]

I think we can have a simple but effective solution first, then keep working on optimizations.

[mattwang44]

@ken71301
As we mentioned at the sprint, please refer to the discussion above.

My untested thoughts:

• New workflow (e.g. daily) to do the full rebuild (make all), and utilize the cache action to cache the build artifacts.
• PR CI workflow restores the cached artifacts, and try building for each updated .po files (e.g. for filename in <find-updated-files>; do make build $$filename; done) -> expected to be much faster than full build

[ken71301]

@mattwang44
It seems like FR also use html mode now.
python/python-docs-fr#426

[ken71301]

In latest sprint Josix and me just tried load of setting and it all seems not effective at all.
So I start believing that cache might be our next option, though I still need to check other's test flow to make sure how we can do it.

[mattwang44]

With this patch (issue) released along with Sphinx 8.1, the build duration in CI has significantly dropped from ~25 mins to ~4 mins 🚀. I believe doc build time would then not be an issue for us. I'll close this issue if no objections in the following days.