Patch in inventory version in deploydocs

[goerz]

Alternative to #2442, #2433 #2448, and #2449.

Still work in progress (a little difficult to test).

I also think this is an extremely cumbersome approach, and hope we don't go with this one. My preferred alternative at this point would probably be the very simple #2442 or #2449.

[mortenpi]

For what it's worth, for testing, I would factor the bit in deploydocs into a function that takes the path to the inventory and the new version number as an argument, and then updates the file. Then we could have like 1 or 2 reference cases in the repo that we unit test against.

[goerz]

I don't think I can continue with this approach in good conscience.

With the Projects.toml fallback, this PR is functionally equivalent to #2442, as the tag for a tagged version should always match the version in Project.toml (since that's where TagBot ultimately gets it from).

Without the Project.toml fallback (Morten's current preference), there's no way I can think of to meet the requirements in #2424 (comment) / #2442 (comment) since there's no way to obtain a version for any dev deployment.

I also really don't like that the inventory isn't complete after makedocs: I routinely build the docs locally, where deploydocs never runs. Still, I should be able to use the inventory that comes out of that for other purposes. For example, it would mess with the instructions in What if I want to link to a project that does not provide an inventory file?, which in the future would be "If the project you want to link to uses an old version of Documenter that doesn't generate inventories, clone it, instantiate the docs environment, update Documenter, run include("docs/make.jl"), convert the resulting inventory to .toml, and copy it into your own docs/src/inventories." If the objects.inv isn't complete until after deploydocs, I'd have to augment that with "manually fill in the version" (which nobody is going to remember to do). Well, I guess I'd end up writing a convenience function for the entire workflow and add that to DocInventories. But you get my point.

I don't think it's so terrible if the inventory has its own logic on where to get its version metadata from, even if it's not the "authoritative source" used for the selection menu. We don't even need to expand on this in any detail in the documentation: "a plain text header that includes the project name […] and a project version obtained from the Project.toml" is all we need to say, and #2442 really just does the right thing without much fuss.

I should have started with just reading the Project.toml, instead of a version/inventory_version keyword from makedocs/HMTL.

Anyway, any thoughts on this from @fredrikekre or @odow? I'm still open to fresh ideas. This is certainly not to just push through my preferred solution. Maybe I'm overlooking something fundamental, but having had a few nights to sleep on this, I've only gotten more and more convinced that #2442 is the right thing to do. It also happens to be the simplest solution, with no change to Documenter's existing API.

[goerz]

There is one potential advantage of setting the version in deploydocs, though: it might work better out of the box for the Julia project itself, see #2442 (comment)

[goerz]

And just to be clear: if someone can tell me how we can get a version for dev-deployments, I'm still willing to give this PR another go.

[mortenpi]

I think this was said somewhere, but to double check: does the version have to be a version number (format-wise)? It can't be a random string (e.g. dev, to match the version selector)?

And if not, could we just leave it empty for dev docs? You'd normally link against stable/ anyway, which would have a good version number in the inventory?

Strictly speaking, dev/ docs don't have a well-defined version (you don't know if it's going to be a patch, minor, or major bump¹). For packages that are strict about maintaining -DEV versions in the Project.toml, how about adding a dev_version option to deploydocs that allows the user to explicitly set the version, and we provide a function to also automatically determine it from the Project.toml (e.g. dev_version = Documenter.project_version())?

Yes, this would mean that getting a correct version in the inventory for dev versions is something the package authors would have to configure. But, for dev versions, I don't feel like we have strong enough standard practice to safely assume that the Project.toml contains a meaningful version number.

Footnotes

1. Well, +DEV on top of the last release would perhaps be systematic, but is also confusing, and not really practiced? ↩

[goerz]

I think this was said somewhere, but to double check: does the version have to be a version number (format-wise)? It can't be a random string (e.g. dev, to match the version selector)?

It's mostly just metadata for human consumption. I don't think anything really breaks depending on the version. But in principle it's supposed to be a proper version number without a v prefix. The only place I'm aware of where it enters programmatically is how the tooltip is determined in Sphinx:

reftitle = _('(in %s v%s)') % (proj, version)

So with dev, the tooltip would read "(in Documenter vdev)".

And if not, could we just leave it empty for dev docs?

Technically, sure, but definitely not what I had in mind.

You'd normally link against stable/ anyway

I'm not sure that's true. For intra-org @extrefs, I dynamically link dev-to-dev and stable-to-stable

Strictly speaking, dev/ docs don't have a well-defined version

Every project will have its own policy on that – or lack of policy, in which case the dev version string is just going to be the previous release. But that's still ok; just as ok as pkgversion returning that version after a dev-install, at least. The version from Project.toml" is still the most "correct" information.

For packages that are strict about maintaining -DEV versions in the Project.toml, how about adding a dev_version option to deploydocs that allows the user to explicitly set the version, and we provide a function to also automatically determine it from the Project.toml (e.g. dev_version = Documenter.project_version())?

I don't think that's necessary. We'd already have inventory_version in makedocs. No need for multiple options (and #2442 wouldn't need any options).

Yes, this would mean that getting a correct version in the inventory for dev versions is something the package authors would have to configure.

Everyone deploys dev-builds and would have to set that up. Or, more likely, not set it up, so that we'd end up with a lot of inventories in the wild that lack full metadata. Which is why I was hoping to avoid everyone having to tweak their docs/make.jl file.

We should definitely make sure that Julia-nightly automatically gets the correct version number into their inventory. Julia has some kind of custom setup: they don't have "dev" in the versions-selection menu, but, e.g., v1.11-dev. With #2442 it definitely works (now that I also read the VERSION file). I'm 90% sure that setting it in deploydocs like we're trying to do here also works, but it's hard to know for sure since I can't easily run deploydocs locally. (One of the issues I have with setting the version in deploydocs instead of makedocs).

The most correct behavior for the inventory version really is "whatever is the project version", i.e., "whatever is in Project.toml", not "whatever the versions-selection-menu shows as a label". If you really want the version to be "whatever the versions-selection-menu shows", that's better than nothing, but then I would still keep inventory-writing features in DocumenterInterLinks as well that override the default Documenter behavior.

[goerz]

I did the refactoring and completed the tests. This now behaves as @mortenpi requested:

• If an inventory_version is given as an option to HTML, use that. Otherwise, the inventory version is left blank in the makedocs step
• In deploydocs, if the inventory version is blank, patch in a version derived from deployment folder name. This will generally set the correct version for tagged releases, but result in a blank version for dev-deployments.

Since this does not fulfill the minimum requirements of setting a version for dev-deployments, I do not consider this an acceptable solution, but at least we have a complete PR now to compare against #2442.

[goerz]

@mortenpi I understand where you're coming from and wanting to avoid "multiple sources of truth", and I totally respect you having the final word on these architectural decisions.

There are technical reasons I don't like the approach in this PR very much: inventory isn't complete after makedocs, no dev versions, and just the code being a little convoluted. But I can probably get over those, or work around them somehow.

To boil down why I have a different perspective on the "multiple sources of truth" issue (reiterating some of my previous comments):

• The labels in the versions-selection-menu are just labels for different subfolders in the gh-pages branch. They happen to coincide with "version" because the tag names that determine the folder names are of the form "vX.Y.Z", but semantically they're not the "project version" that should go in the inventory (according to Sphinx' original semantics). We're just trying to recover the project version from the folder name by dropping the v prefix here.
• The original "source of truth" is the project's Project.toml file anyway (or a VERSION file in the case of the Julia project itself). For any tagged release, the label in the versions-menu will match the version in the Project.toml. I can't think of any normal situation where getting the inventory-version from the deploy folder name would give a conflicting result to getting it from Project.toml. Just Project.toml is strictly more informative, since for dev, the folder name no longer contains version information, so we can't recover version information from it.

I understand it's a judgement call, and no hard feelings, but I really hope I can bring you around to this perspective (and maybe @fredrikekre and @odow can still chime in to sway opinions).

[mortenpi]

The labels in the versions-selection-menu are just labels for different subfolders in the gh-pages branch.

So, I disagree with this pretty strongly. Semantically, they are versions. They just happen to be implemented via subdirectories and symlinks. The assumption that they are versions is baked in pretty strongly.

The original "source of truth" is the project's Project.toml file anyway (or a VERSION file in the case of the Julia project itself).

So, I don't actually disagree with this. But we are stuck with design decisions that predate Project.tomls. Arguably, we should maybe create an alternative DeployConfig that uses Project.toml to do the versioning.. but I am not sure we really want to go down the rabbit hole of deploydocs refactoring in the context of this PR.

The second related design decision that is causing us pain is the distinction between makedocs and deploydocs. What we want is makedocs to be able to know the version, but that is only determined once deploydocs runs, which happens after makedocs. Trying to duplicate or override the logic used by deploydocs in makedocs does not seem like good idea, and just adds too much complexity to the system as a whole.

So what I am trying to get to here is some solution that doesn't break the existing assumptions, but gives us a working documentation inventory that we could start using in the ecosystem. And then, at some point, we can fix it up when we overhaul the whole make.jl approach (my preference goes to declarative configuration; #1350, #2353).

In deploydocs, if the inventory version is blank, patch in a version derived from deployment folder name.

My gut reaction is that if deploydocs is deploying a tag (not dev version), it should unconditionally overwrite the version. Theoretically, unless the user has made a mistake in they makedocs call, the version should match the tag anyway, and it would ensure that any mistakes would get fixed by deploydocs. But also, maybe it's also okay for deploydocs to defer to the user here, and be like "sure, if that's what you want".

[mortenpi]

One more note: I think I would nearly be okay with deploydocs using the version in the packages's Project.toml as the fallback to get the version number for dev versions.

However, I don't like the brittleness when it comes to edge cases, where Documenter is not used in the very common, but not universal docs/Project.toml way for a single package -- we don't really know which Project.toml to read:

• The docs environment (i.e. docs/Project.toml) doesn't have a version, so we can't use the active project.
• We could pick the version of a dependency of the environment, but we don't know which one is the "package".
• We could go up a directory and look for a Project.toml, and that would work like 95% of the time, but there's no particular reason to assume that the env one level up is actually the correct environment. And so having this type of automagical behavior also feels pretty icky to me -- generally I've been trying to get rid of that kind of stuff.¹

Hence the strong preference for an explicit way of setting that version. And writing vDEV or something into the inventory seems like a good enough default.

Footnotes

1. Side note: if subprojects would link back to main projects, then it would be fine, I think. But we don't have that metadata. (https://github.com/JuliaLang/Pkg.jl/issues/1233) ↩

[goerz]

The labels in the versions-selection-menu are just labels for different subfolders in the gh-pages branch.

So, I disagree with this pretty strongly. Semantically, they are versions. They just happen to be implemented via subdirectories and symlinks. The assumption that they are versions is baked in pretty strongly.

There's a part of this I agree with, and maybe that's the common ground we've been trying to get to: If the tag/branch/deployment folder is a valid version, then the "project version" (whether it's extracted from a Project.toml file or given by the user as an explicit inventory_version argument) must match it. There should never be a situation where the project version is different from the version shown in the versions-selection menu. I'd be happy to make that a hard rule and throw an error if there is a mismatch. I can do (and have done) that even in #2442: Set the inventory version from Project.toml/JuliaProject.toml/VERSION in makedocs, and then check in deploydocs that that version matches the tagged version.

The original "source of truth" is the project's Project.toml file anyway (or a VERSION file in the case of the Julia project itself).

So, I don't actually disagree with this. But we are stuck with design decisions that predate Project.tomls. Arguably, we should maybe create an alternative DeployConfig that uses Project.toml to do the versioning. but I am not sure we really want to go down the rabbit hole of deploydocs refactoring in the context of this PR.

I agree on not going down that rabbit hole right now, but I actually don't even think it would be a good idea: The versions in the version-selection menu should be derived from git tags, not from the project version in the Project.toml. Otherwise, if people forget to bump their project version after they'll make a release, they'll likely overwrite their "stable" deployment the next time they push to master.

The second related design decision that is causing us pain is the distinction between makedocs and deploydocs. What we want is makedocs to be able to know the version, but that is only determined once deploydocs runs, which happens after makedocs. Trying to duplicate or override the logic used by deploydocs in makedocs does not seem like good idea, and just adds too much complexity to the system as a whole.

I agree with that, but I don't think the situation is that dire, if we just enforce that the version in makedocs must agree with the version in deploydocs. There isn't much duplication of logic in that.

So what I am trying to get to here is some solution that doesn't break the existing assumptions, but gives us a working documentation inventory that we could start using in the ecosystem. And then, at some point, we can fix it up when we overhaul the whole make.jl approach (my preference goes to declarative configuration; #1350, #2353).

In deploydocs, if the inventory version is blank, patch in a version derived from deployment folder name.

My gut reaction is that if deploydocs is deploying a tag (not dev version), it should unconditionally overwrite the version.

As a general rule, I'm allergic to automatic values silently overwriting values that have been explicitly set by a human. But I am okay with throwing an error if deploydocs detects that the user gave an unambiguously wrong version.

Theoretically, unless the user has made a mistake in they makedocs call, the version should match the tag anyway, and it would ensure that any mistakes would get fixed by deploydocs.

I agree with that 100%, but let's just throw an error and let the user fix their mistake.

But also, maybe it's also okay for deploydocs to defer to the user here, and be like "sure, if that's what you want".

Although my general rule is "a human should always have the last word", I don't think we need to allow a situation where the versions-menu shows "v1.0", but the user manually sets the inventory version to "v0.1". That's clearly a mistake.

One more note: I think I would nearly be okay with deploydocs using the version in the packages's Project.toml as the fallback to get the version number for dev versions.

I've added another #2448 that is just #2443 with the additional fallback.

However, I don't like the brittleness when it comes to edge cases, where Documenter is not used in the very common, but not universal docs/Project.toml way for a single package -- we don't really know which Project.toml to read:

I don't think that part is very tricky. The way I've implemented this is pretty straightforward:

• Start from the current working directory (seems to always be docs/, but if I overlooked something and there is some possibility that pwd during the build is not a subfolder of docs, we can change that to the docs/src folder, which Documenter knows the absolute path of)
• Check for files Project.toml, JuliaProject.toml, VERSION (what Julia itself uses), in that order. Ignore TOML files that do not define a version (Only the main Project.toml should define version, and docs/Project.toml specifically generally does not contain a version)
• If no version could be extracted from any of the files, go up one directory, and repeat until we reach the root of the file system
• If we reach the root of the file system without finding a file that specifies a project version, show a warning and return an empty string

So, basically, we identify the main Project.toml file as the first one that defines a version. It doesn't have to be in the current directory, or one level up, specifically.

This should work for at least 99% of projects. It should work for monorepos, too (although I haven't found any monorepo setups that use Documenter and deploy more than just the documentation of their "main" package). The most common situation where it might fail is for independent documentation repos (where the documentation is in a separate repo from the package). These repos currently might have a Project.toml in their root directory that does not define a version. Such setups would now see a warning when they do a dev-deployment, and an error if they deploy a tagged version (not necessarily an error if that would be considered semver-breaking). They'll have several options to fix their setup: define a version in the main Project.toml matching the version of the project that's being documented, use a VERSION file (like Julia does), or manually set the inventory version in their make.jl file before deploydocs is called.

I could also imagine that some repos might have a version in docs/Project.toml by accident (which would then take precedence over the main Project.toml), but they'll see an error message so that they can remove the stray version.

So in the worst case, people will get yelled at that they need to fix their setup and have a version somewhere that matches the tagged version they're deploying. In no case would there be a "wrong" version either in the inventory or in the menu.

So now, we have three competing PRs on top of #2424 to deal with the version setting

• Obtain inventory version from Project.toml #2442 – makedocs gets the version for the inventory from the closest Project.toml/JuliaProject.toml/VERSION that defines a version. Then deploydocs checks whether that version matches the tag for the release that's being deployed (for non-dev deployments). There is no need for a manual inventory_version option. I'd be extremely happy with this option! (I would probably even be able to come around to letting deploydocs overwrite the version unconditionally) (edit: closed in favor of Overwrite inventory version in deploydocs #2449)
• Patch inventory version in deploydocs with fallback to Project.toml #2448 – The inventory version is set by deploydocs. For tagged releases, it uses the version from the tag, and for dev-releases, it falls back to the same logic as Obtain inventory version from Project.toml #2442 to get the version from the closest Project.toml. The ultimate result is identical to Obtain inventory version from Project.toml #2442 (at least that's the intent), but the code is a bit more convoluted than necessary, and I'm not enthusiastic that the objects.inv file isn't generally complete after the makedocs step. (note: closed in favor of Overwrite inventory version in deploydocs #2449)
• Patch in inventory version in deploydocs #2443 – This PR, which sets the version in deploydocs without a fallback. As a result, dev-deployments would typically have an empty version in the inventory, and we'd have to push people to modify their setup and specify a manual inventory_version. I'd find this one hard to swallow.

@mortenpi Let me know if you have further questions to help you decide between these options, or if there is anything that requires tweaking or refactoring in any one of the three PRs. But besides that, tell me which one of these you decide on, and I'll merge that into #2424 and wrap everything up with a CHANGELOG entry and such.

[goerz]

I turned #2442 (comment) into a fourth candidate PR:

• Overwrite inventory version in deploydocs #2449 – Initially set the inventory version in makedocs based on the closest Project.toml/JuliaProject.toml/VERSION file that defines a version. Then unconditionally overwrite the version in deploydocs for a tagged release (where deploydocs can know the correct version). This overwrite shouldn't actually change the version for all normal setups, but it'll still let deploydocs have the last word.

Having slept on this variation another night, it seems like a pretty good compromise, as it takes @mortenpi comment to heart that

My gut reaction is that if deploydocs is deploying a tag (not dev version), it should unconditionally overwrite the version. Theoretically, unless the user has made a mistake in they makedocs call, the version should match the tag anyway, and it would ensure that any mistakes would get fixed by deploydocs.

This is under the assumption that

I think I would nearly be okay with deploydocs using the version in the packages's Project.toml as the fallback to get the version number for dev versions

is ultimately something you can bring yourself around to. Without any fallback, we'll just have to leave the version blank for dev-deployments, and I'll just have to be unhappy ;-)

[goerz]

Closing in favor of #2449 (comment)