Merge pull request #3482 from olehermanse/master

Integrated prettier and black with CFEngine CLI

Author: olehermanse

.github/workflows/formatting.yml

@@ -25,16 +25,9 @@ jobs:
           pipx install cfengine black
       - name: Run formatting command to (hopefully not) make changes
         run: |
-          # TODO: We can integrate these commands into cfengine dev docs-format
-          #       https://northerntech.atlassian.net/browse/CFE-4572
           # TODO: There are (small) diffs between cfbs pretty and prettier
           #       https://northerntech.atlassian.net/browse/CFE-4571
-          find . -name '*.markdown' -type f | parallel -j16 prettier -w {}
-          black .
           cfengine dev docs-format
-      - name: Check output.log file for warnings
-        run: |
-          ! grep WARNING output.log
       - name: Check if there are changes
         run: |
           git diff --exit-code

.gitignore

@@ -1,9 +1,6 @@
 # Build artifacts
 cfdoc_log.markdown
 
-# Log file from running markdowner.py:
-output.log
-
 # emacs
 
 *~

DOCTOOLS.README.md

@@ -35,7 +35,7 @@ Git workflow.
 1. Using Windows Explorer, open a directory where you wish to store CFEngine related Git repositories (e.g. D:\Projects\CFEngine).
 2. Right click to bring up the contextual menu.
 3. Select `Git Bash`
-![Contextual Menu With Git Bash](Doctools-ReadMe-Git-Bash.png)
+   ![Contextual Menu With Git Bash](Doctools-ReadMe-Git-Bash.png)
 4. A `Command Prompt` style window will open. This is where you will enter Git commands.
 
 ### Using Git
@@ -44,12 +44,12 @@ Git is a very powerful and robust command line tool at its heart. While it has a
 
 #### Good Git commands to learn
 
-* clone
-* add
-* commit
-* push
-* pull
-* reset
+- clone
+- add
+- commit
+- push
+- pull
+- reset
 
 #### Basic examples
 
@@ -77,6 +77,6 @@ Note: If working with a clone that you do not have rights to push to then you wi
 
 ### Git (and related) resources
 
-* The Wikipedia article on Git provides a good overview of the software: http://en.wikipedia.org/wiki/Git_(software).
-* The official Git documentation has excellent content for learning how to use Git: http://git-scm.com/documentation.
-* GitHub's Help section has many useful tips and tricks, as well as guidance on how to setup Git, create Git repos, fork of repos and contribute to Git supported software projects: https://help.github.com/
+- The Wikipedia article on Git provides a good overview of the software: http://en.wikipedia.org/wiki/Git_(software).
+- The official Git documentation has excellent content for learning how to use Git: http://git-scm.com/documentation.
+- GitHub's Help section has many useful tips and tricks, as well as guidance on how to setup Git, create Git repos, fork of repos and contribute to Git supported software projects: https://help.github.com/

README.md

@@ -169,33 +169,19 @@ To the markdown parser, it doesn't matter how you wrap your sentences, a complet
 
 ### Tools and automatic formatting
 
-**prettier**
+We use a combination of tools to automatically format the documentation.
+If you want to run them locally, install them:
 
-It is recommended, but not required, to use [`prettier`](https://prettier.io/) to automatically format markdown files.
-This replaces some cases where two characters would produce the same output, and prefers the less ambiguous one, i.e. `-` instead of `*` for bulleted lists, `_` instead of `*` for italics.
-It also eliminates some common inconsistencies, such as trailing whitespace, too many / too few consecutive newline characters, etc.
-It doesn't help you with everything, for example, the one sentence per line style mentioned above is not something `prettier`
-does automatically, so you'd still have to do that manually.
-Formatting with `prettier` is currently not enforced, but recommended, especially if you are creating new files.
-
-**markdowner.py**
-
-We have a Python script and GitHub Action to automatically fix some common markdown mistakes.
-This is intentionally not very strict, it only fixes very specific things, such as:
-
-- Trims trailing whitespace at the end of lines
-- Replace some utf-8 symbols which have an ascii lookalike
-- Ensures exactly 1 newline before the end of the file
-- De-indents code blocks where everything inside the code block is indented
+```
+pipx install cfengine black
+npm install --global prettier
+```
 
-When someone makes one of these "mistakes" it is highlighted in the Pull Request by the GitHub Action.
-If you want to run this script locally and have it fix these things for you, you can:
+And run the command:
 
-```bash
-find . -name '*.markdown' -type f -exec python3 .github/workflows/markdowner.py {} all \;
 ```
-
-In many cases, you can also configure your editor to help you with these things.
+cfengine dev docs-format
+```
 
 ## Documentation structure
 

generator/build/README.md

@@ -1,63 +1,58 @@
-Building docs in a container
-============================
+# Building docs in a container
 
 This set of scripts lets you build docs in a container.
 Currently, it's aimed towards using buildah on oslo-dc.
 
-Prereq
-------
+## Prereq
 
 You will need to have the following repos checked out:
 
-* core (used for changelog, examples)
-* nova (used for changelog)
-* enterprise (used for changelog)
-* masterfiles (used to document masterfies)
-* documentation
+- core (used for changelog, examples)
+- nova (used for changelog)
+- enterprise (used for changelog)
+- masterfiles (used to document masterfies)
+- documentation
 
-Usage
------
+## Usage
 
 If you have buildah installed:
 
 1. clone the above repos (run `clone.sh`)
 
 2. export the following env variables:
+   - `$BRANCH` - Branch that we're actually building artifacts for,
+     for example, 3.18, master, or pr
 
-	* `$BRANCH` - Branch that we're actually building artifacts for,
-	  for example, 3.18, master, or pr
+   - `$PACKAGE_JOB` - where to take CFEngine HUB package from,
+     a dir at http://buildcache.cloud.cfengine.com/packages/,
+     usually `testing-pr`
 
-	* `$PACKAGE_JOB` - where to take CFEngine HUB package from,
-	  a dir at http://buildcache.cloud.cfengine.com/packages/,
-	  usually `testing-pr`
+   - `$PACKAGE_UPLOAD_DIRECTORY` - where to take CFEngine HUB package from,
+     a dir at http://buildcache.cloud.cfengine.com/packages/testing-pr/,
+     for example, `jenkins-master-nightly-pipeline-943`
 
-	* `$PACKAGE_UPLOAD_DIRECTORY` - where to take CFEngine HUB package from,
-	  a dir at http://buildcache.cloud.cfengine.com/packages/testing-pr/,
-	  for example, `jenkins-master-nightly-pipeline-943`
-
-	* `$PACKAGE_BUILD` - RELEASE of the build to be downloaded, usually 1
+   - `$PACKAGE_BUILD` - RELEASE of the build to be downloaded, usually 1
 
 3. `cd` to directory with all repos and run `run.sh` from this dir
 
-Details
--------
+## Details
 
 To make a container, cd to this directory and run:
 
-	buildah build-using-dockerfile -t docs .
+    buildah build-using-dockerfile -t docs .
 
 or
 
-	docker build --tag docs .
+    docker build --tag docs .
 
 To run docs job in a container, run:
 
-	c=$(buildah from -v $PWD:/nt docs)
-	buildah run $c bash -x documentation/generator/build/main.sh $BRANCH $PACKAGE_JOB $PACKAGE_UPLOAD_DIRECTORY $PACKAGE_BUILD
-	buildah run $c bash -x documentation/generator/_scripts/_publish.sh $BRANCH
-	buildah rm $c
+    c=$(buildah from -v $PWD:/nt docs)
+    buildah run $c bash -x documentation/generator/build/main.sh $BRANCH $PACKAGE_JOB $PACKAGE_UPLOAD_DIRECTORY $PACKAGE_BUILD
+    buildah run $c bash -x documentation/generator/_scripts/_publish.sh $BRANCH
+    buildah rm $c
 
 or
 
-	docker run --rm -it -v $PWD:/nt docs bash -x documentation/generator/build/main.sh $BRANCH $PACKAGE_JOB $PACKAGE_UPLOAD_DIRECTORY $PACKAGE_BUILD
-	docker run --rm -it -v $PWD:/nt docs bash -x documentation/generator/_scripts/_publish.sh $BRANCH
+    docker run --rm -it -v $PWD:/nt docs bash -x documentation/generator/build/main.sh $BRANCH $PACKAGE_JOB $PACKAGE_UPLOAD_DIRECTORY $PACKAGE_BUILD
+    docker run --rm -it -v $PWD:/nt docs bash -x documentation/generator/_scripts/_publish.sh $BRANCH