Find a replacement for marker (markdown linter)

[ivanvc]

What would you like to be added?

Right now we're using marker as the markdown linter. However, I see two issues:

1. Its functionality is limited, from its README

   This is a tool for finding issues in CommonMark documentation. Right now, it only identifies broken links (malformed URLs, non-existent paths, etc.).

2. It's written in Rust, and while dropping our GitHub action for static checking (Complete migration of verify workflow to prow #18058). We wont run this linter in the Prow infrastructure as we don't have Rust/Cargo available (refer to https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/etcd-io_etcd/18058/pull-etcd-verify/1793352130300481536#1:build-log.txt%3A659-662).

While trying to fix 2, I opened a PR crawford/marker#32 to be able to use a published binary, rather than building the project itself.

However, it would be better to use a Go-written Markdown linter as alternative.

Why is this needed?

To keep linting markdown files.

[ivanvc]

cc. @jmhbnz

[lavishpal]

@ivanvc Can we use markdownlint which is officially on Go.pkg.dev : https://pkg.go.dev/github.com/svendowideit/markdownlint
https://github.com/SvenDowideit/markdownlint/blob/v0.9.4/main.go

[ivanvc]

@ivanvc Can we use markdownlint which is officially on Go.pkg.dev : https://pkg.go.dev/github.com/svendowideit/markdownlint
https://github.com/SvenDowideit/markdownlint/blob/v0.9.4/main.go

It looks like an abandoned project. It was originally done by Docker, but it's now a public archive, and its last commit is from 2017. I think, ideally we should replace it with other linter that has more wide adoption.

Their README mentions that it was in use for https://docs.docker.com. However, Docker docs is now using NodeJS' markdownlint:

• https://github.com/docker/docs/blob/b952723881c6e6cd7dcc556ab4c1d5a1818ec08d/package.json#L33
• https://github.com/docker/docs/blob/b952723881c6e6cd7dcc556ab4c1d5a1818ec08d/Dockerfile#L35

[ivanvc]

Surveying GitHub doesn't seem to be promising. There are some projects, but they seem unmaintained, or abandoned: https://github.com/search?q=markdown+linter+language%3AGo&type=repositories

[fuweid]

It's written in Rust

IMO, it's not related to RUST. It needs a release to publish executable binary. For Go ecosystem, go install can help us. However, it doesn't mean we should replace it. Just my two cents.

As far as I know, the marker is good one for the linter. I was thinking about how to add makefile to install it without cargo, mentioned by #16708 (comment).

Is it possible to be considered as an option? upload binary to storage which support anonymous download, like

etcd/scripts/install-marker.sh

Lines 12 to 15 in bb701b9

	MARKER_URL="https://storage.googleapis.com/etcd/test-binaries/marker-v0.4.0-x86_64-unknown-linux-gnu"
	if [ "${ARCH}" == "darwin" ]; then
	MARKER_URL="https://storage.googleapis.com/etcd/test-binaries/marker-v0.4.0-x86_64-apple-darwin"
	fi

Thanks,
Wei

[ivanvc]

Hey @fuweid, I agree. After researching a native-Go alternative and yielding no results, I'm leaning towards having a linter that we can use as a binary. I opened PR crawford/marker#32 to have marker distributed as a binary. Therefore, we won't need to have cargo installed locally, nor do we need to host the compiled binary (#16708 (comment)).

However, my original point is that the use-case for marker is limited; as it states on its README, it only checks broken links. It doesn't do any other Markdown linting:

This is a tool for finding issues in CommonMark documentation. Right now, it only identifies broken links (malformed URLs, non-existent paths, etc.).

[fuweid]

@ivanvc Nice!

However, my original point is that the use-case for marker is limited; as it states on its README, it only checks broken links. It doesn't do any other Markdown linting:

If we do have new linter requirement, maybe we can file pull request to marker. The detector for broken linters is common use case because it's easy to break it by refactor of docs.

[ivanvc]

With #18318 merged, I was looking into cleaning up scripts/install-marker.sh. But it looks like it's only used by tests/Dockerfile. Is this a Dockerfile we want to maintain? The Ubuntu version used is no longer maintained (21.10). Building the image fails because the APT repositories don't work anymore.

@jmhbnz, thoughts? (or @ahrtr, as I see you were the last to commit to that Dockerfile).

[ahrtr]

But it looks like it's only used by tests/Dockerfile. Is this a Dockerfile we want to maintain?

I am not aware of any usage on the tests/Dockerfile. Probably we can just remove it.

[ivanvc]

Are we okay with keeping marker as the markdown linter, considering it only checks for broken links? Do we want or need to do more linting, considering that running our Markdown files against NodeJS' markdownlint-cli with the default configuration raises too many warnings?

[jmhbnz]

Discussed during sig-etcd triage meeting. We recently put work into updating etcd-io/website to put a proper markdown linting workflow in place.

I am supportive of adding a new prow presubmit for etcd-io/etcd to do the same.

Once the new workflow is in place I would suggest we use a good first issue to get help from the community to address the backlog of markdown files that need updates.