Multiple pdf files for single project

[dschult]

Despite the ability to create multiple pdf files (via latex) with the sphinx builder (see #245), the code for the sphinx backend only moves one pdf file to the target directory. The rest are created but never used.

Would it make more sense to move all the pdf files from the appropriate _build directory to the target directory? If we did so, would they show up in the "downloads" section of the webpage? If this sounds reasonable I could try to make a PR for it.

[agjohnson]

This would be a great addition! I'm afraid the majority of the logic is likely hard coding the pdf output filename without regard for what the user is configuring. This goes for the download page as well.

Thinking through the changes required for this, I realized there are a number of design decisions to be made here. Some quick thoughts:

• We can't completely remove the hardcoded name, as we definitely want to provide a per-version pdf
• The download page will grow in complexity with additional pdfs, it should show the file(s) by version. More likely, the version page could be repurposed to display these files, this might be the more correct solution, but is a farther out target.
• We could add this as a configuration option to our readthedocs.yaml, but again, display of downloads is the primary issue.

Feel free to discuss any changes you'd like to see here. We'll want to hash out the design of this a bit more before you go down the wrong path, though don't let that stop you from trying of course :)

[dschult]

Perhaps the downloads page could list the files in groups by version.
That is, the hardcoded name could refer to a directory inside the media server. In the folder for each version would be files that don't necessarily (and most likely wouldn't) depend on version. The downloads page could ask users to choose a version which would then provide the list of downloads for that version.

Would it be reasonable to hardcode directory names instead of pdf file names -- moving/copying them as needed instead of moving/copying individual files?

[cwalther]

The rest are created but never used.

Note that this is no longer true – the build now fails before having created any of the PDFs. See #8612.

[stretch4x4]

I would certainly like to see this added also. We are looking to add user documentation alongside our existing documentation. Being able to break this up into smaller pdfs would better match our current setup, as opposed to one gigantic pdf the combines both user centric documentation and technical dev based documentation

[ericholscher]

Next step here is a quick proposal for how to implement this, and then once we agree, doing the actual implementation work.

We can split this up into 3 changes:

• Build backend changes to upload multiple PDFs
• Web backend changes to track multiple PDF files
• Dashboard changes to show multiple PDF files

[agjohnson]

We discussed the dashboard UI implications here: readthedocs/ext-theme#77

I'd like to see this implemented in the new dashboard primarily/solely. This is because the project downloads page does not exist in the new dashboard. Instead of solving this multiple ways in our dashboards, I'd like to reduce our focus to just the one implementation of this.

The other area that we need to reflect this change is in the theme flyout menu, where we currently link to only a single file. This was the largest blocker to this change when we started the change last. There was an issue somewhere with more discussion that I'm not yet finding. @humitos would have more background here though.

We could still ship this without the flyout change, and doc authors/readers could always get the files from our UI, but the flyout is a fairly significant part of this work that I don't think we'll handle until we get to the addons JS customization work.

[humitos]

I'm pretty excited we are prioritizing this work! 💯

The other area that we need to reflect this change is in the theme flyout menu, where we currently link to only a single file.

This is the most important thing here. I think we will need to re-thing and re-design some part of the flyout to support this. However, I think this shouldn't be a blocker for the initial backend + dashboard UI implementation.

There was an issue somewhere with more discussion that I'm not yet finding. @humitos would have more background here though.

I remember we discussed about multiple PDF file in different places, but I think we never touched on the flyout.

Note the build system backend is already prepared to support multiple PDF/ePub/etc files. We will need to remove this restriction and add a new one to check that all the files in that folder are PDF/ePub/etc only:

readthedocs.org/readthedocs/projects/tasks/builds.py

Lines 576 to 597 in 8458022

	# Check if there are multiple files on artifact directories.
	# These output format does not support multiple files yet.
	# In case multiple files are found, the upload for this format is not performed.
	if artifact_type in ARTIFACT_TYPES_WITHOUT_MULTIPLE_FILES_SUPPORT:
	artifact_format_files = len(os.listdir(artifact_directory))
	if artifact_format_files > 1:
	log.error(
	"Multiple files are not supported for this format. "
	"Skipping this output format.",
	output_format=artifact_type,
	)
	raise BuildUserError(
	BuildUserError.BUILD_OUTPUT_HAS_MULTIPLE_FILES.format(
	artifact_type=artifact_type
	)
	)
	if artifact_format_files == 0:
	raise BuildUserError(
	BuildUserError.BUILD_OUTPUT_HAS_0_FILES.format(
	artifact_type=artifact_type
	)
	)

Then we will need a way to know "how many PDF files a version has and its names" (either by doing a listdir on S3 for that version, or by saving this data into the db) so we return this via the API and also to the dashboard. Finally, we will need El Proxito to handle each of this URLs properly.

The current URL is, for example for test-builds, https://test-builds.readthedocs.io/_/downloads/en/all-formats-build-jobs/pdf/. Note this URL does not include the filename and it's unique. So, we will need to decide what to do here. Should we keep the old URL working? Which file it will point to if the project has multiple PDFs? Should we re-use this URL to show a list of all the PDFs available for this project & version?

Some extra kind-of-related context for the future:

• Build: standardize output directory for artifacts #9888 (comment)
• Build: notes from output directory standardization #9918
• https://github.com/readthedocs/meta/issues/77
• Build: implement build output contract #9088

[benjaoming]

Thanks for the amazing overview @humitos 💯

I'm going to split out this issue into sub-tasks to protect the comment history here :)