Broken links detected for the deployment to devdocs

[tlylt]

Please confirm that you have searched existing issues in the repo

Yes, I have searched the existing issues

Any related issues?

No response

Tell us about your environment

Windows 10

MarkBind version

Master branch

Describe the bug and the steps to reproduce it

See logs in every CI run for recently merged PRs, in Deploy DG on any commit to master, to markbind.org/devdocs

(This is likely due to how the site.json and host differs for the build of our docs into different domains)

v4.1.0
info: Website generation started at 6:37:12 AM
info: Building assets...
info: Assets built
info: Generating pages...
[------------------------] 0 / 24 pages built[=-----------------------] 1 / 24 pages built[===---------------------] 3 / 24 pages built[====--------------------] 4 / 24 pages built[=====-------------------] 5 / 24 pages built[=======-----------------] 7 / 24 pages built[========----------------] 8 / 24 pages built[==========--------------] 10 / 24 pages built[===========-------------] 11 / 24 pages built[============------------] 12 / 24 pages built[==============----------] 14 / 24 pages built[================--------] 16 / 24 pages built[=================-------] 17 / 24 pages built[==================------] 18 / 24 pages built[===================-----] 19 / 24 pages built[====================----] 20 / 24 pages built[=====================---] 21 / 24 pages built[======================--] 22 / 24 pages built[========================] 24 / 24 pages built
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/components/popups.html' found in file '/home/runner/work/markbind/markbind/docs/devGuide/bootcamp/contributeToDocs.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/gettingStarted.html' found in file '/home/runner/work/markbind/markbind/docs/devGuide/bootcamp/exploreMarkBind.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/gitignoreFile.html' found in file '/home/runner/work/markbind/markbind/docs/devGuide/bootcamp/exploreMarkBind.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/formattingContents.html' found in file '/home/runner/work/markbind/markbind/docs/devGuide/bootcamp/exploreMarkBind.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/usingComponents.html' found in file '/home/runner/work/markbind/markbind/docs/devGuide/bootcamp/exploreMarkBind.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/formattingContents.html#text-styles' found in file '/home/runner/work/markbind/markbind/docs/devGuide/bootcamp/exploreMarkBind.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/formattingContents.html#code' found in file '/home/runner/work/markbind/markbind/docs/devGuide/bootcamp/exploreMarkBind.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/formattingContents.html#emoji' found in file '/home/runner/work/markbind/markbind/docs/devGuide/bootcamp/exploreMarkBind.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/components/presentation.html#boxes' found in file '/home/runner/work/markbind/markbind/docs/devGuide/bootcamp/exploreMarkBind.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/components/presentation.html#panels' found in file '/home/runner/work/markbind/markbind/docs/devGuide/bootcamp/exploreMarkBind.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/components/popups.html#tooltips' found in file '/home/runner/work/markbind/markbind/docs/devGuide/bootcamp/exploreMarkBind.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/tweakingThePageStructure.html#frontmatter' found in file '/home/runner/work/markbind/markbind/docs/devGuide/bootcamp/exploreMarkBind.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/tweakingThePageStructure.html#layouts' found in file '/home/runner/work/markbind/markbind/docs/devGuide/bootcamp/exploreMarkBind.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/siteJsonFile.html' found in file '/home/runner/work/markbind/markbind/docs/devGuide/bootcamp/exploreMarkBind.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/deployingTheSite.html#deploying-via-github-actions' found in file '/home/runner/work/markbind/markbind/docs/devGuide/bootcamp/exploreMarkBind.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/themes.html' found in file '/home/runner/work/markbind/markbind/docs/devGuide/bootcamp/exploreMarkBind.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/reusingContents.html#includes' found in file '/home/runner/work/markbind/markbind/docs/devGuide/bootcamp/exploreMarkBind.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/index.html' found in file '/home/runner/work/markbind/markbind/docs/_markbind/layouts/default.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/index.html' found in file '/home/runner/work/markbind/markbind/docs/_markbind/layouts/devGuide.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/addingPages.html' found in file '/home/runner/work/markbind/markbind/docs/userGuide/addingPages.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/markBindSyntaxOverview.html' found in file '/home/runner/work/markbind/markbind/docs/userGuide/markBindSyntaxOverview.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/formattingContents.html' found in file '/home/runner/work/markbind/markbind/docs/userGuide/formattingContents.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/usingComponents.html' found in file '/home/runner/work/markbind/markbind/docs/userGuide/usingComponents.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/usingHtmlJavaScriptCss.html' found in file '/home/runner/work/markbind/markbind/docs/userGuide/usingHtmlJavaScriptCss.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/tweakingThePageStructure.html' found in file '/home/runner/work/markbind/markbind/docs/userGuide/tweakingThePageStructure.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/reusingContents.html' found in file '/home/runner/work/markbind/markbind/docs/userGuide/reusingContents.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/gettingStarted.html' found in file '/home/runner/work/markbind/markbind/docs/userGuide/authoringContents.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/addingPages.html' found in file '/home/runner/work/markbind/markbind/docs/userGuide/authoringContents.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/deployingTheSite.html' found in file '/home/runner/work/markbind/markbind/docs/userGuide/deployingTheSite.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/siteJsonFile.html#baseurl' found in file '/home/runner/work/markbind/markbind/docs/userGuide/deployingTheSite.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/cliCommands.html#serve-command' found in file '/home/runner/work/markbind/markbind/docs/userGuide/deployingTheSite.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/cliCommands.html#build-command' found in file '/home/runner/work/markbind/markbind/docs/userGuide/deployingTheSite.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/themes.html' found in file '/home/runner/work/markbind/markbind/docs/userGuide/deployingTheSite.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/markBindInTheProjectWorkflow.html' found in file '/home/runner/work/markbind/markbind/docs/userGuide/deployingTheSite.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/deployingTheSite.html' found in file '/home/runner/work/markbind/markbind/docs/userGuide/siteJsonFile.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/deployingTheSite.html#using-ci-platforms' found in file '/home/runner/work/markbind/markbind/docs/userGuide/tipsAndTricks.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/deployingTheSite.html#deploying-to-netlify' found in file '/home/runner/work/markbind/markbind/docs/userGuide/tipsAndTricks.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide' found in file '/home/runner/work/markbind/markbind/docs/index.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/markBindSyntaxOverview.html' found in file '/home/runner/work/markbind/markbind/docs/index.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/formattingContents.html#icons' found in file '/home/runner/work/markbind/markbind/docs/index.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/formattingContents.html#emoji' found in file '/home/runner/work/markbind/markbind/docs/index.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/makingTheSiteSearchable.html' found in file '/home/runner/work/markbind/markbind/docs/index.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/tweakingThePageStructure.html' found in file '/home/runner/work/markbind/markbind/docs/index.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/reusingContents.html' found in file '/home/runner/work/markbind/markbind/docs/index.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/deployingTheSite.html' found in file '/home/runner/work/markbind/markbind/docs/index.md'
warn: You might have an invalid intra-link! Ignore this warning if it was intended.
'/devdocs/userGuide/markBindInTheProjectWorkflow.html' found in file '/home/runner/work/markbind/markbind/docs/index.md'
info: Pages built
info: Site data built
info: Website generation complete! Total build time: 1.917s
info: Build success!

And the intra-site links are indeed broken, e.g.

• on this page https://markbind.org/devdocs/devGuide/bootcamp/exploreMarkBind.html
• click on "user guide"

Expected behavior

No broken link and no warning logged.

Anything else?

No response

[itsyme]

Hi @tlylt! I tried to fix the links by removing /devdocs which should work as the resultant url will be the correct one. However, when I tried to build the dg with npm run build:dg, I still get the same warnings. I think this is because when building the developer guide this way, it does not have access to the files in the user guide. I came to this conclusion by prepending ../ to the url multiple times and it returned devdocs/... every time. The other way I can think of for solving this and removing the warnings is hardcoding the urls, which I think may not be desirable. What are your thoughts?

[tlylt]

@itsyme thanks for investigating. Could you help me test by trying to build just the dg (like in the github workflow) locally and see if the navigation to UG still works? I suspect that's not the case and if this is so then we might need to relook at our docs deployment strategy.

[itsyme]

Hi @tlylt! I managed to build the DG however when I tried to deploy it using npm run deploy:dg I get an error: The environment variable GITHUB_TOKEN does not exist.. I tried to search for other ways to deploy the dg alone in the developer guide but could not find any solutions. Do you have any suggestions?

[tlylt]

Hi @tlylt! I managed to build the DG however when I tried to deploy it using npm run deploy:dg I get an error: The environment variable GITHUB_TOKEN does not exist.. I tried to search for other ways to deploy the dg alone in the developer guide but could not find any solutions. Do you have any suggestions?

It's ok, I was talking about building and not deploying. Deploying can only be done with the access token.

[itsyme]

Hi @tlylt! Sorry for the confusion. When opening the developer guide locally after the changes made, the links to the user guide don't work.

[tlylt]

Thank you @itsyme for the testing. I think we will need further discussion on how to fix this issue.

To test locally:

• Run cd docs && ../packages/cli/index.js serve -s dg-site.json
• try to navigate from DG to UG, e.g. go to onboarding BootCamp, explore markbind as a user, click on user guide hyperlink
• will have broken link warnings as well as when landed on ug pages: Cannot GET /devdocs/userGuide/gettingStarted.html
• In fact, just building and serving the UG alone will also log broken link warnings (due to the navbar reference to DG):
  

It seems like building and deploying the same site (or parts of the same site, in our case, UG, and DG) separately via different site.json is going to be problematic for intra-site link management. @damithc have you encountered something similar for other MarkBind sites?

Some related work done trying to fix this problem:

• Link to DevGuide in markbind.org #1161
• Add DG to markbind.org #1211
• Fix broken links in user guide and dev guide #1388 (comment)

Pinging @MarkBind/active-devs for context/comments/suggestions on what's the best way forward.

[jovyntls]

Would it help if we implemented a feature that allows users to specify alternate baseURLs in their site.json? For instance, we allow users to specify altUrls in their site.json:

{
"baseUrl": "...",
"altUrls": {
    "devGuide": "...",
    "userGuide": "..."
  },
}

They can then use these URLs in their pages, e.g. src="{{devGuide}}/development/settingUp.html.

This may solve the issue as we can specify different URLs in our site.json, dg-site.json, etc.; then building from different directories could use the respective site.jsons, which ensures that the right URL is used. For instance, we can specify a different URL for {{devGuide}} in our dg-site.json and root site.json.

As a bonus, it could also allow authors to specify shorthands for various links.

Would such a feature solve this issue?

[tlylt]

They can then use these URLs in their pages, e.g. src="{{devGuide}}/development/settingUp.html.

This may solve the issue as we can specify different URLs in our site.json, dg-site.json, etc.; then building from different directories could use the respective site.jsons, which ensures that the right URL is used. For instance, we can specify a different URL for {{devGuide}} in our dg-site.json and root site.json.

As a bonus, it could also allow authors to specify shorthands for various links.

Would such a feature solve this issue?

Thank you @jovyntls, I think that gave me some inspiration on how to proceed.

I think fundamentally building the UG and the DG separately will result in broken link issues if a cross-reference is done. This is because the individual site has no way to access the content outside of it. So locally, the relative links to another site will not work at all. However, when deployed to the cloud, the fact that the UG and DG share the same domain but different base URL means they can create an illusion that the two exist together.

I think what you are suggesting might also be possible with the use of "tags", perhaps similar to how we are using the "environment--dg" to handle decision to display the link to DG in our navigation bar. @itsyme would you like to try and see if that leads to a viable solution?

[itsyme]

They can then use these URLs in their pages, e.g. src="{{devGuide}}/development/settingUp.html.
This may solve the issue as we can specify different URLs in our site.json, dg-site.json, etc.; then building from different directories could use the respective site.jsons, which ensures that the right URL is used. For instance, we can specify a different URL for {{devGuide}} in our dg-site.json and root site.json.
As a bonus, it could also allow authors to specify shorthands for various links.
Would such a feature solve this issue?

Thank you @jovyntls, I think that gave me some inspiration on how to proceed.

I think fundamentally building the UG and the DG separately will result in broken link issues if a cross-reference is done. This is because the individual site has no way to access the content outside of it. So locally, the relative links to another site will not work at all. However, when deployed to the cloud, the fact that the UG and DG share the same domain but different base URL means they can create an illusion that the two exist together.

I think what you are suggesting might also be possible with the use of "tags", perhaps similar to how we are using the "environment--dg" to handle decision to display the link to DG in our navigation bar. @itsyme would you like to try and see if that leads to a viable solution?

Hi @tlylt! Is this approach in line with what you have in mind?

1. Create a new environment--combined
2. Display the exact link https://... when it is environment--dg or environment--ug, and display the intra-site link when it is environment--combined

[tlylt]

1. Create a new environment--combined
2. Display the exact link https://... when it is environment--dg or environment--ug, and display the intra-site link when it is environment--combined

Yes, something like that. Can it be done just by manipulating the existing two tags? If not creating a new tag is fine as well.