Deploy pre-release Cloud doc builds internally

[nrichers]

We're between a rock and a hard place on the Cloud team when it comes to finding UI help link issues:

• We now test for the validity of UI links, which is great, but ...
• We don't build prerelease docs until about a week before a release, so as to avoid preanouncing new features, which means that the UI link tests for new releases typically fail time and again because the docs aren't being built, yet.
• Because of this timing, we also tend to run out of runway before a GA to fix legitimate doc link issues that do exist.

A relatively simple solution might see us deploy the pre-release doc builds for Cloud to an internal server somewhere. The link tests could check if a link exists externally and then check the internal server as a fallback, and fail only if the target URL exists in neither location. (Alternatively, if we can prevent pre-releases from showing up in the version drop-down, that might get us closer to where we need to be.)

Nico did suggest that we move the doc link testing to the E2E tests, but I'd like to try this approach first. We have a poor track record, full of broken doc links from the UI, and if we can make sure that a link exists when a PR gets submitted, I think that helps us.

@nik9000 @bevacqua @brittenhouse @elastic/cloud-writers

[nik9000]

I wonder how our desire not to leak features early squares with our desire to make the cloud docs public? Would we only make the currently released docs public? I ask because what we implement might not make sense depending on the answer.

For example, it is fairly easy to do link checking locally now. Maybe it is as simple as having @nrichers do the link checking locally before we merge so as not to break the main build. That is pretty much what I did when we merged the new JS client last week. It is fairly labor intensive but might make sense.

Ultimately the best way to split the work up that I've found for this sort of thing is to do it on PRs. That way the person who tries to add the bad link gets notified before they merge it and the bad links don't pile up.

We can do link checking on PR builds now using the docs build infrastructure, but I'm not comfortable deploying that process to cloud because it is fairly experimental. I deployed it to a few docs repos and the APM team and have found a few false positive failures.

You can totally do link checking in your build, but the docs build is still going to do it. If you can do something on the PR builds on your side that prevents most of the link failures I'm totally fine with it.

An in the middle option would be to set up the mechanisms that I use for PR based link checking in APM and use them in a periodic build for cloud. Because it'd be experimental for a few months @nrichers and I could watch on it and filter out false positives. That'd be a lot less labor intensive than building locally but still get you false-positive filtering.

[nrichers]

@nik9000 note this is for links from our ECE Cloud UI and ESS console UI to our docs, not for links within the docs themselves. There is file we added a while back that contains the links from the UI to the external docs and it's those links that we want to check.

(Everything you wrote I agree with, but I think this issue is for a different context. My apologies if I was being obtuse. Cloud PRs that add new help links for UI changes now check that those links actually exist as part of the PR build process. Because we don't build pre-release docs, the UI link tests always fail.)

EDIT: Here's an example of the file we are testing: https://github.com/elastic/cloud/blob/master/cloud-ui/public/constants/docLinks.ts.

[nik9000]

@nik9000 note this is for links from our ECE Cloud UI and ESS console UI to our docs, not for links within the docs themselves. There is file we added a while back that contains the links from the UI to the external docs and it's those links that we want to check.

AH! That is a different thing. We do a thing for Kibana now that I talked about generalizing here that I think might help with this. Basically if there exists a way for us to extract the links from your repo we can check them at link check time. We'd still have to deal with the "when do we run build on the new branch problem", but that is a problem we can solve I think.

The simplest solution would be if cloud had some kind of config file I could load or spat out a list that you committed with PRs or something. I could run the typescript as part of the build but dependencies for that get a little snaky. I don't want to pull in all of cloud's deps in the docker image.

[nrichers]

We do already check the UI to doc links as part of the PR process, that puzzle piece we have and I don't think we want to change it. What you're describing is more like the end-to-end tests that our UI link checking might need to move into, if we can't fix the PR process itself where links are checked for their validity when the PR goes through its tests.

I think what we need is to have a fully deployed build of the Cloud docs that the PR links can check against, that's really the only piece this issue should address. Unlike all the other products at Elastic, we don't build docs for pre-releases today, which has all sorts of repercussions:

• We can't check the UI to doc links until very late in the release cycle, well past code freeze <-- the piece covered by this issue
• We can't provide doc pieces for other PRs (larger features are broken up across many PRs and a PR with docs and help links might also be required by another PR)
• We can't have internal users at Elastic try out our docs and provide feedback, except on a one-off basis via Firebase

Should we maybe chat over Zoom for a few mins rather than trade comments in GH? 😀 I was talking to @bevacqua today about how hard his code would be to modify to check an internal docs server for a URL and it sounds very doable. Similarly, I think deploying the docs to an internal location should also be quite doable, the question is just where and how we can automate the process. (Even just S3 and a manually deployed doc build might work in the interim, but I think we want to make sure that whatever changes Nico makes dovetail with where the docs might end up getting deployed to internally.)