Fix all broken links, add github action to check for broken links on each PR

Tim Zakian · bors-diem · commit 2cdfa0e69d35 · 2022-03-09T01:33:35.000Z
Closes: #102
diff --git a/.github/workflows/ci-test.yml b/.github/workflows/ci-test.yml
@@ -273,3 +273,20 @@ jobs:
         uses: ./.github/actions/early-terminator
         with:
           github-token: ${{secrets.GITHUB_TOKEN}}
+  check-for-broken-links:
+    name: Check for broken links in files
+    runs-on: ubuntu-20.04
+    needs:
+      - prepare
+    steps:
+      - uses: actions/checkout@v2
+      - name: Set up Ruby 2.6
+        uses: actions/setup-ruby@v1
+        with:
+          ruby-version: 2.6.x
+      - name: Run Checks
+        run: |
+          gem install awesome_bot
+          # Don't look in git or target dirs. Don't check png, bib, tex, or shell files
+          # We allow links to be redirects, allow duplicates, and we also allow Too Many Requests (429) errors
+          find . -not \( -path "./.git*" -prune \) -not \( -path "./target" -prune \) -type f -not -name "*.png" -not -name "*.sh" -not -name "*.bib" -not -name "*.tex" | while read arg; do awesome_bot --allow-redirect --allow-dupe --allow 429 --skip-save-results $arg; done
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -38,12 +38,9 @@ detailed guidance about how to contribute to the project.
 #### Documentation
 
 Diem's developer website is also open source (the code can be found in this
-[repository](https://github.com/diem/diem/developers.diem.com/website/)).  It is built using
+[repository](https://github.com/diem/diem/tree/main/developers.diem.com)).  It is built using
 [Docusaurus](https://docusaurus.io/):
 
-If you know Markdown, you can already contribute! How to contribute can be found in the [website
-repo](https://github.com/diem/developers.diem.com/website/CONTRIBUTING.md).
-
 ## Developer Workflow
 
 Changes to the project are proposed through pull requests. The general pull
diff --git a/README.md b/README.md
@@ -14,7 +14,7 @@ Move is a new programmable platform for blockchains and other applications where
 
 ## Contributing
 
-To begin contributing, [sign the CLA](https://diem.com/en-US/cla-sign/). You can learn more about contributing to the Diem project by reading our [Contribution Guide](https://developers.diem.com/docs/community/contributing) and by viewing our [Code of Conduct](https://developers.diem.com/docs/policies/code-of-conduct).
+To begin contributing, [sign the CLA](https://diem.com/en-US/cla-sign/). You can learn more about contributing to the Diem project by reading our [Contribution Guide](https://github.com/diem/diem/blob/main/CONTRIBUTING.md) and by viewing our [Code of Conduct](https://developers.diem.com/docs/policies/code-of-conduct).
 
 ## Community
 
diff --git a/documentation/coding_guidelines.md b/documentation/coding_guidelines.md
@@ -21,7 +21,7 @@ diem$ cargo xfmt
 diem$ cargo xclippy --all-targets
 ```
 
-In general, we follow the recommendations from [rust-lang-nursery](https://rust-lang-nursery.github.io/api-guidelines/about.html) and [The Rust Programming Language](https://doc.rust-lang.org/book/).  The remainder of this guide provides detailed guidelines on specific topics in order to achieve uniformity of the codebase.
+In general, we follow the recommendations from [rust-lang-nursery](https://rust-lang.github.io/api-guidelines/) and [The Rust Programming Language](https://doc.rust-lang.org/book/).  The remainder of this guide provides detailed guidelines on specific topics in order to achieve uniformity of the codebase.
 
 ## Code documentation
 
@@ -243,11 +243,11 @@ This is a brief and simplified mini guide of the different functions that exist
 
 We currently use [log](https://docs.rs/log/) for logging.
 
-* [error!](https://docs.rs/log/0.4.10/log/macro.error.html) - Error-level messages have the highest urgency in [log](https://docs.rs/log/).  An unexpected error has occurred (e.g. exceeded the maximum number of retries to complete an RPC or inability to store data to local storage).
-* [warn!](https://docs.rs/log/0.4.4.10/log/macro.warn.html) - Warn-level messages help notify admins about automatically handled issues (e.g. retrying a failed network connection or receiving the same message multiple times, etc.).
-* [info!](https://docs.rs/log/0.4.4.10/log/macro.info.html) - Info-level messages are well suited for "one-time" events (such as logging state on one-time startup and shutdown) or periodic events that are not frequently occurring - e.g. changing the validator set every day.
-* [debug!](https://docs.rs/log/0.4.4.10/log/macro.debug.html) - Debug-level messages can occur frequently (i.e. potentially > 1 message per second) and are not typically expected to be enabled in production.
-* [trace!](https://docs.rs/log/0.4.4.10/log/macro.trace.html) - Trace-level logging is typically only used for function entry/exit.
+* [error!](https://docs.rs/log/0.4.14/log/macro.error.html) - Error-level messages have the highest urgency in [log](https://docs.rs/log/).  An unexpected error has occurred (e.g. exceeded the maximum number of retries to complete an RPC or inability to store data to local storage).
+* [warn!](https://docs.rs/log/0.4.14/log/macro.warn.html) - Warn-level messages help notify admins about automatically handled issues (e.g. retrying a failed network connection or receiving the same message multiple times, etc.).
+* [info!](https://docs.rs/log/0.4.14/log/macro.info.html) - Info-level messages are well suited for "one-time" events (such as logging state on one-time startup and shutdown) or periodic events that are not frequently occurring - e.g. changing the validator set every day.
+* [debug!](https://docs.rs/log/0.4.14/log/macro.debug.html) - Debug-level messages can occur frequently (i.e. potentially > 1 message per second) and are not typically expected to be enabled in production.
+* [trace!](https://docs.rs/log/0.4.14/log/macro.trace.html) - Trace-level logging is typically only used for function entry/exit.
 
 ### Testing
 
diff --git a/language/README.md b/language/README.md
@@ -1,7 +1,7 @@
 ---
 id: move-language
 title: Move Language
-custom_edit_url: https://github.com/diem/diem/edit/main/language/README.md
+custom_edit_url: https://github.com/diem/move/edit/main/language/README.md
 ---
 
 
diff --git a/language/changes/4-unit-testing.md b/language/changes/4-unit-testing.md
@@ -336,7 +336,7 @@ In the development of unit tests, there were a couple of alternatives considered
 
 ### Expected Value Tests
 
-Move already supports [expected value tests](https://github.com/diem/diem/tree/main/language/tools/move-cli#testing-with-the-move-cli) in the Move CLI, however these cover a different aspect of testing for Move. In particular, there is no concept of test-only code and test-only dependencies, which makes testing non-public functions much more difficult. Additionally, each expected value test entry must be a transaction or script function. Because of this, the expected value tests rely on a specific Move adapter implementation, whereas the unit tests do not rely on an adapter implementation and rely solely on the Move compiler and VM.
+Move already supports [expected value tests](https://github.com/diem/move/tree/main/language/tools/move-cli#testing-with-the-move-cli) in the Move CLI, however these cover a different aspect of testing for Move. In particular, there is no concept of test-only code and test-only dependencies, which makes testing non-public functions much more difficult. Additionally, each expected value test entry must be a transaction or script function. Because of this, the expected value tests rely on a specific Move adapter implementation, whereas the unit tests do not rely on an adapter implementation and rely solely on the Move compiler and VM.
 
 Due to the design of unit tests, unit tests make it easier to test the individual units of code that comprise a Move module. However, because the unit tests do not require or use an adapter, they do not support certain features that you may  wish to test. E.g., there is no way in Move unit tests to query if a specific event has been emitted.
 
diff --git a/language/move-analyzer/editors/code/README.md b/language/move-analyzer/editors/code/README.md
@@ -22,7 +22,7 @@ The `move-analyzer` language server is a Rust program that is part of
 1. You may clone [the Diem repository](https://github.com/diem/diem) yourself and build
    `move-analyzer` from its source code. This is recommended for Diem hackathon participants, and
    Diem & Move core developers. To do so, follow the instructions in the Move tutorial's
-   [Step 0: Installation](https://github.com/diem/diem/tree/main/language/documentation/tutorial#step-0-installation).
+   [Step 0: Installation](https://github.com/diem/move/tree/main/language/documentation/tutorial#step-0-installation).
 2. You may use Rust's package manager `cargo` to install `move-analyzer` in your user's PATH. This
    is recommended for people who do not work on Diem & Move core.
    1. If you don't already have a Rust toolchain installed, you should install
diff --git a/language/move-analyzer/editors/code/package.json b/language/move-analyzer/editors/code/package.json
@@ -6,7 +6,7 @@
     "license": "Apache-2.0",
     "version": "0.0.5",
     "preview": true,
-    "homepage": "https://developers.diem.com/docs/move/move-overview",
+    "homepage": "https://developers.diem.com",
     "repository": {
         "url": "https://github.com/diem/diem.git",
         "type": "git"
diff --git a/language/move-binary-format/README.md b/language/move-binary-format/README.md
@@ -1,7 +1,7 @@
 ---
 id: vm
 title: Virtual Machine
-custom_edit_url: https://github.com/diem/diem/edit/main/language/move-binary-format/README.md
+custom_edit_url: https://github.com/diem/move/edit/main/language/move-binary-format/README.md
 ---
 
 
@@ -14,7 +14,7 @@ metering logical abstraction is also defined there.
 
 The MoveVM is a stack machine with a static type system. The MoveVM honors
 the specification of the Move language through a mix of file format,
-verification (for reference [bytcode verifier README](https://github.com/diem/diem/blob/main/language/bytecode-verifier/README.md))
+verification (for reference [bytcode verifier README](https://github.com/diem/move/blob/main/language/move-bytecode-verifier/README.md))
 and runtime constraints. The structure of the file format allows the
 definition of modules, types (resources and unrestricted types), and
 functions. Code is expressed via bytecode instructions, which may have
@@ -40,7 +40,7 @@ The `CompiledModule` and `CompiledScript` definitions in
 `diem/language/move-binary-format/src/file_format.rs` are the top-level structs for a Move
 *Module* or *Transaction Script*, respectively. These structs provide a
 simple abstraction over the file format. Additionally, a set of
-[*Views*](https://github.com/diem/diem/blob/main/language/move-binary-format/src/views.rs) are defined to easily navigate and inspect
+[*Views*](https://github.com/diem/move/blob/main/language/move-binary-format/src/views.rs) are defined to easily navigate and inspect
 `CompiledModule`s and `CompiledScript`s.
 
 ## Folder Structure
diff --git a/language/move-bytecode-verifier/README.md b/language/move-bytecode-verifier/README.md
@@ -1,7 +1,7 @@
 ---
 id: bytecode-verifier
 title: Bytecode Verifier
-custom_edit_url: https://github.com/diem/diem/edit/main/language/bytecode-verifier/README.md
+custom_edit_url: https://github.com/diem/move/edit/main/language/move-bytecode-verifier/README.md
 ---
 
 
diff --git a/language/move-bytecode-verifier/invalid-mutations/src/helpers.rs b/language/move-bytecode-verifier/invalid-mutations/src/helpers.rs
@@ -9,15 +9,13 @@ use std::{collections::BTreeSet, ops::Index as OpsIndex};
 ///
 /// If `indexes_len` is greater than `max`, all indexes will be returned.
 ///
-/// This function implements [Robert Floyd's F2
-/// algorithm](https://blog.acolyer.org/2018/01/30/a-sample-of-brilliance/) for sampling without
-/// replacement.
+/// This function implements Robert Floyd's F2 algorithm for sampling without replacement.
 pub(crate) fn pick_idxs<T, P>(max: usize, indexes: &T, indexes_len: usize) -> Vec<usize>
 where
     T: OpsIndex<usize, Output = P> + ?Sized,
     P: AsRef<PropIndex>,
 {
-    // See https://blog.acolyer.org/2018/01/30/a-sample-of-brilliance/ (the F2 algorithm)
+    // See https://dl.acm.org/doi/10.1145/30401.315746 (the F2 algorithm)
     // for a longer explanation. This is a variant that works with zero-indexing.
     let mut selected = BTreeSet::new();
     let to_select = indexes_len.min(max);
@@ -40,9 +38,7 @@ where
 ///
 /// If the number of `Index` instances is greater than `max`, all indexes will be returned.
 ///
-/// This function implements [Robert Floyd's F2
-/// algorithm](https://blog.acolyer.org/2018/01/30/a-sample-of-brilliance/) for sampling without
-/// replacement.
+/// This function implements Robert Floyd's F2 algorithm for sampling without replacement.
 #[inline]
 pub(crate) fn pick_slice_idxs(max: usize, indexes: &[impl AsRef<PropIndex>]) -> Vec<usize> {
     pick_idxs(max, indexes, indexes.len())
diff --git a/language/move-compiler/README.md b/language/move-compiler/README.md
@@ -1,7 +1,7 @@
 ---
 id: move-compiler
 title: Move Source Language
-custom_edit_url: https://github.com/diem/diem/edit/main/language/move-compiler/README.md
+custom_edit_url: https://github.com/diem/move/edit/main/language/move-compiler/README.md
 ---
 
 ## Code under this subtree is experimental. It is out of scope for the Diem Bug Bounty until it is no longer marked experimental.
diff --git a/language/move-ir-compiler/README.md b/language/move-ir-compiler/README.md
@@ -1,7 +1,7 @@
 ---
 id: ir-to-bytecode
 title: Move IR Compiler
-custom_edit_url: https://github.com/diem/diem/edit/main/language/compiler/README.md
+custom_edit_url: https://github.com/diem/move/edit/main/language/move-ir-compiler/README.md
 ---
 
 
@@ -14,7 +14,7 @@ The Move IR compiler compiles the Move IR down to its bytecode representation.
 The Move IR compiler compiles modules and scripts written in Move down to
 their respective bytecode representations. The two data types used to
 represent these outputs are `CompiledModule` and `CompiledScript`. These
-data types are defined in [file_format.rs](https://github.com/diem/diem/blob/main/language/move-binary-format/src/file_format.rs).
+data types are defined in [file_format.rs](https://github.com/diem/move/blob/main/language/move-binary-format/src/file_format.rs).
 
 Beyond translating Move IR to Move bytecode, the compiler's purpose is as a
 testing tool for the bytecode verifier. Because of this, its job is to
@@ -24,7 +24,7 @@ performed during the compilation process. In fact, the compiler goes out of
 its way to push these semantic checks into the bytecode, and compile
 semantically invalid code in the Move IR to equivalent---semantically
 invalid---bytecode programs. The semantics of the compiled bytecode is
-then verified by the [bytecode verifier](https://github.com/diem/diem/blob/main/language/bytecode-verifier/README.md). The compiler command line
+then verified by the [bytecode verifier](https://github.com/diem/move/blob/main/language/move-bytecode-verifier/README.md). The compiler command line
 automatically calls the bytecode verifier at the end of compilation.
 
 ## Command-line options
diff --git a/language/move-prover/README.md b/language/move-prover/README.md
@@ -1,7 +1,7 @@
 ---
 id: move-prover
 title: Formal verification code
-custom_edit_url: https://github.com/diem/diem/edit/main/language/move-prover/README.md
+custom_edit_url: https://github.com/diem/move/edit/main/language/move-prover/README.md
 ---
 
 
diff --git a/language/move-prover/lab/README.md b/language/move-prover/lab/README.md
@@ -11,8 +11,9 @@ pip install jupyterlab
 pip install notebook
 ```
 
-For the Rust kernel, follow the installation instructions at [google/evcxr](https://github.
-com/google/evcxr/blob/master/evcxr_jupyter/README.md). You need version 0.9.0 or later.
+For the Rust kernel, follow the installation instructions at
+[google/evcxr](https://github.com/google/evcxr/blob/master/evcxr_jupyter/README.md).
+You need version 0.9.0 or later.
 
 
 # Caveats
diff --git a/language/move-prover/tests/testsuite.rs b/language/move-prover/tests/testsuite.rs
@@ -156,7 +156,7 @@ fn test_runner_for_feature(path: &Path, feature: &Feature) -> datatest_stable::R
         {
             warn!(
                 "Prover tools are not configured, verification tests will be skipped. \
-        See https://github.com/diem/diem/tree/main/language/move-prover/doc/user/install.md \
+        See https://github.com/diem/move/tree/main/language/move-prover/doc/user/install.md \
         for instructions."
             );
         }
diff --git a/language/testing-infra/test-generation/README.md b/language/testing-infra/test-generation/README.md
@@ -1,7 +1,7 @@
 ---
 id: bytecode-test-generation
 title: Bytecode Test Generation Tool
-custom_edit_url: https://github.com/diem/diem/edit/main/language/testing-infra/test-generation/README.md
+custom_edit_url: https://github.com/diem/move/edit/main/language/testing-infra/test-generation/README.md
 ---
 
 # Bytecode Test Generation Tool
