Skip to content

Commit

Permalink
doc: prepare for commonmark
Browse files Browse the repository at this point in the history
We are still using Pandoc’s Markdown parser, which differs from CommonMark spec slightly.

Notably:
- Line breaks in lists behave differently.
- Admonitions do not support the simpler syntax jgm/commonmark-hs#75
- The auto_identifiers uses a different algorithm – I made the previous ones explicit.
- Languages (classes) of code blocks cannot contain whitespace so we have to use “pycon” alias instead of Python “console” as GitHub’s linguist

While at it, I also fixed the following issues:
- ShellSesssion was used
- Removed some pointless docbook tags.
  • Loading branch information
jtojnar committed Jun 7, 2021
1 parent ce6b1a4 commit 6ecc641
Show file tree
Hide file tree
Showing 44 changed files with 454 additions and 388 deletions.
25 changes: 12 additions & 13 deletions doc/builders/fetchers.chapter.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,59 +20,58 @@ The main difference between `fetchurl` and `fetchzip` is in how they store the c

`fetchpatch` works very similarly to `fetchurl` with the same arguments expected. It expects patch files as a source and performs normalization on them before computing the checksum. For example it will remove comments or other unstable parts that are sometimes added by version control systems and can change over time.


Other fetcher functions allow you to add source code directly from a VCS such as subversion or git. These are mostly straightforward nambes based on the name of the command used with the VCS system. Because they give you a working repository, they act most like `fetchzip`.

## `fetchsvn`
## `fetchsvn` {#fetchsvn}

Used with Subversion. Expects `url` to a Subversion directory, `rev`, and `sha256`.

## `fetchgit`
## `fetchgit` {#fetchgit}

Used with Git. Expects `url` to a Git repo, `rev`, and `sha256`. `rev` in this case can be full the git commit id (SHA1 hash) or a tag name like `refs/tags/v1.0`.

Additionally the following optional arguments can be given: `fetchSubmodules = true` makes `fetchgit` also fetch the submodules of a repository. If `deepClone` is set to true, the entire repository is cloned as opposing to just creating a shallow clone. `deepClone = true` also implies `leaveDotGit = true` which means that the `.git` directory of the clone won't be removed after checkout.

## `fetchfossil`
## `fetchfossil` {#fetchfossil}

Used with Fossil. Expects `url` to a Fossil archive, `rev`, and `sha256`.

## `fetchcvs`
## `fetchcvs` {#fetchcvs}

Used with CVS. Expects `cvsRoot`, `tag`, and `sha256`.

## `fetchhg`
## `fetchhg` {#fetchhg}

Used with Mercurial. Expects `url`, `rev`, and `sha256`.

A number of fetcher functions wrap part of `fetchurl` and `fetchzip`. They are mainly convenience functions intended for commonly used destinations of source code in Nixpkgs. These wrapper fetchers are listed below.

## `fetchFromGitHub`
## `fetchFromGitHub` {#fetchfromgithub}

`fetchFromGitHub` expects four arguments. `owner` is a string corresponding to the GitHub user or organization that controls this repository. `repo` corresponds to the name of the software repository. These are located at the top of every GitHub HTML page as `owner`/`repo`. `rev` corresponds to the Git commit hash or tag (e.g `v1.0`) that will be downloaded from Git. Finally, `sha256` corresponds to the hash of the extracted directory. Again, other hash algorithms are also available but `sha256` is currently preferred.

`fetchFromGitHub` uses `fetchzip` to download the source archive generated by GitHub for the specified revision. If `leaveDotGit`, `deepClone` or `fetchSubmodules` are set to `true`, `fetchFromGitHub` will use `fetchgit` instead. Refer to its section for documentation of these options.

## `fetchFromGitLab`
## `fetchFromGitLab` {#fetchfromgitlab}

This is used with GitLab repositories. The arguments expected are very similar to fetchFromGitHub above.

## `fetchFromGitiles`
## `fetchFromGitiles` {#fetchfromgitiles}

This is used with Gitiles repositories. The arguments expected are similar to fetchgit.

## `fetchFromBitbucket`
## `fetchFromBitbucket` {#fetchfrombitbucket}

This is used with BitBucket repositories. The arguments expected are very similar to fetchFromGitHub above.

## `fetchFromSavannah`
## `fetchFromSavannah` {#fetchfromsavannah}

This is used with Savannah repositories. The arguments expected are very similar to fetchFromGitHub above.

## `fetchFromRepoOrCz`
## `fetchFromRepoOrCz` {#fetchfromrepoorcz}

This is used with repo.or.cz repositories. The arguments expected are very similar to fetchFromGitHub above.

## `fetchFromSourcehut`
## `fetchFromSourcehut` {#fetchfromsourcehut}

This is used with sourcehut repositories. The arguments expected are very similar to fetchFromGitHub above. Don't forget the tilde (~) in front of the user name!
2 changes: 1 addition & 1 deletion doc/builders/images/appimagetools.section.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

`pkgs.appimageTools` is a set of functions for extracting and wrapping [AppImage](https://appimage.org/) files. They are meant to be used if traditional packaging from source is infeasible, or it would take too long. To quickly run an AppImage file, `pkgs.appimage-run` can be used as well.

::: warning
::: {.warning}
The `appimageTools` API is unstable and may be subject to backwards-incompatible changes in the future.
:::

Expand Down
4 changes: 2 additions & 2 deletions doc/builders/images/dockertools.section.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# pkgs.dockerTools {#sec-pkgs-dockerTools}

`pkgs.dockerTools` is a set of functions for creating and manipulating Docker images according to the [ Docker Image Specification v1.2.0 ](https://github.com/moby/moby/blob/master/image/spec/v1.2.md#docker-image-specification-v120). Docker itself is not used to perform any of the operations done by these functions.
`pkgs.dockerTools` is a set of functions for creating and manipulating Docker images according to the [Docker Image Specification v1.2.0](https://github.com/moby/moby/blob/master/image/spec/v1.2.md#docker-image-specification-v120). Docker itself is not used to perform any of the operations done by these functions.

## buildImage {#ssec-pkgs-dockerTools-buildImage}

Expand Down Expand Up @@ -52,7 +52,7 @@ The above example will build a Docker image `redis/latest` from the given base i

> **_NOTE:_** Using this parameter requires the `kvm` device to be available.
- `config` is used to specify the configuration of the containers that will be started off the built image in Docker. The available options are listed in the [ Docker Image Specification v1.2.0 ](https://github.com/moby/moby/blob/master/image/spec/v1.2.md#image-json-field-descriptions).
- `config` is used to specify the configuration of the containers that will be started off the built image in Docker. The available options are listed in the [Docker Image Specification v1.2.0](https://github.com/moby/moby/blob/master/image/spec/v1.2.md#image-json-field-descriptions).

After the new layer has been created, its closure (to which `contents`, `config` and `runAsRoot` contribute) will be copied in the layer itself. Only new dependencies that are not already in the existing layers will be copied.

Expand Down
4 changes: 2 additions & 2 deletions doc/builders/images/snaptools.section.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Currently, `makeSnap` does not support creating GUI stubs.

The following expression packages GNU Hello as a Snapcraft snap.

```{#ex-snapTools-buildSnap-hello .nix}
``` {#ex-snapTools-buildSnap-hello .nix}
let
inherit (import <nixpkgs> { }) snapTools hello;
in snapTools.makeSnap {
Expand All @@ -35,7 +35,7 @@ in snapTools.makeSnap {

Graphical programs require many more integrations with the host. This example uses Firefox as an example, because it is one of the most complicated programs we could package.

```{#ex-snapTools-buildSnap-firefox .nix}
``` {#ex-snapTools-buildSnap-firefox .nix}
let
inherit (import <nixpkgs> { }) snapTools firefox;
in snapTools.makeSnap {
Expand Down
6 changes: 3 additions & 3 deletions doc/builders/packages/cataclysm-dda.section.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Cataclysm: Dark Days Ahead {#cataclysm-dark-days-ahead}

## How to install Cataclysm DDA
## How to install Cataclysm DDA {#how-to-install-cataclysm-dda}

To install the latest stable release of Cataclysm DDA to your profile, execute
`nix-env -f "<nixpkgs>" -iA cataclysm-dda`. For the curses build (build
Expand Down Expand Up @@ -34,7 +34,7 @@ cataclysm-dda.override {
}
```

## Important note for overriding packages
## Important note for overriding packages {#important-note-for-overriding-packages}

After applying `overrideAttrs`, you need to fix `passthru.pkgs` and
`passthru.withMods` attributes either manually or by using `attachPkgs`:
Expand Down Expand Up @@ -69,7 +69,7 @@ in
goodExample2.withMods (_: []) # parallel building enabled
```

## Customizing with mods
## Customizing with mods {#customizing-with-mods}

To install Cataclysm DDA with mods of your choice, you can use `withMods`
attribute:
Expand Down
2 changes: 1 addition & 1 deletion doc/builders/packages/elm.section.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,6 @@ To start a development environment do
nix-shell -p elmPackages.elm elmPackages.elm-format
```

To update the Elm compiler, see <filename>nixpkgs/pkgs/development/compilers/elm/README.md</filename>.
To update the Elm compiler, see `nixpkgs/pkgs/development/compilers/elm/README.md`.

To package Elm applications, [read about elm2nix](https://github.com/hercules-ci/elm2nix#elm2nix).
2 changes: 1 addition & 1 deletion doc/builders/packages/firefox.section.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Firefox {#sec-firefox}

## Build wrapped Firefox with extensions and policies
## Build wrapped Firefox with extensions and policies {#build-wrapped-firefox-with-extensions-and-policies}

The `wrapFirefox` function allows to pass policies, preferences and extension that are available to firefox. With the help of `fetchFirefoxAddon` this allows build a firefox version that already comes with addons pre-installed:

Expand Down
4 changes: 2 additions & 2 deletions doc/builders/packages/opengl.section.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ OpenGL support varies depending on which hardware is used and which drivers are

Broadly, we support both GL vendors: Mesa and NVIDIA.

## NixOS Desktop
## NixOS Desktop {#nixos-desktop}

The NixOS desktop or other non-headless configurations are the primary target for OpenGL libraries and applications. The current solution for discovering which drivers are available is based on [libglvnd](https://gitlab.freedesktop.org/glvnd/libglvnd). `libglvnd` performs "vendor-neutral dispatch", trying a variety of techniques to find the system's GL implementation. In practice, this will be either via standard GLX for X11 users or EGL for Wayland users, and supporting either NVIDIA or Mesa extensions.

## Nix on GNU/Linux
## Nix on GNU/Linux {#nix-on-gnulinux}

If you are using a non-NixOS GNU/Linux/X11 desktop with free software video drivers, consider launching OpenGL-dependent programs from Nixpkgs with Nixpkgs versions of `libglvnd` and `mesa.drivers` in `LD_LIBRARY_PATH`. For Mesa drivers, the Linux kernel version doesn't have to match nixpkgs.

Expand Down
23 changes: 13 additions & 10 deletions doc/builders/packages/steam.section.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ Use `programs.steam.enable = true;` if you want to add steam to systemPackages a
## Troubleshooting {#sec-steam-troub}

- **Steam fails to start. What do I do?**

Try to run

```ShellSession
Expand All @@ -32,34 +33,36 @@ Use `programs.steam.enable = true;` if you want to add steam to systemPackages a

- The `newStdcpp` parameter was removed since NixOS 17.09 and should not be needed anymore.
- Steam ships statically linked with a version of libcrypto that conflics with the one dynamically loaded by radeonsi_dri.so. If you get the error

```
steam.sh: line 713: 7842 Segmentation fault (core dumped)
```

have a look at [this pull request](https://github.com/NixOS/nixpkgs/pull/20269).

- **Java**

1. There is no java in steam chrootenv by default. If you get a message like

```
/home/foo/.local/share/Steam/SteamApps/common/towns/towns.sh: line 1: java: command not found
```
```
/home/foo/.local/share/Steam/SteamApps/common/towns/towns.sh: line 1: java: command not found
```

You need to add
you need to add

```nix
steam.override { withJava = true; };
```
```nix
steam.override { withJava = true; };
```

## steam-run {#sec-steam-run}

The FHS-compatible chroot used for steam can also be used to run other linux games that expect a FHS environment. To do it, add

```nix
pkgs.steam.override ({
nativeOnly = true;
newStdcpp = true;
}).run
nativeOnly = true;
newStdcpp = true;
}).run
```

to your configuration, rebuild, and run the game with
Expand Down
8 changes: 4 additions & 4 deletions doc/builders/packages/xorg.section.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

The Nix expressions for the X.org packages reside in `pkgs/servers/x11/xorg/default.nix`. This file is automatically generated from lists of tarballs in an X.org release. As such it should not be modified directly; rather, you should modify the lists, the generator script or the file `pkgs/servers/x11/xorg/overrides.nix`, in which you can override or add to the derivations produced by the generator.

## Katamari Tarballs
## Katamari Tarballs {#katamari-tarballs}

X.org upstream releases used to include [katamari](https://en.wiktionary.org/wiki/%E3%81%8B%E3%81%9F%E3%81%BE%E3%82%8A) releases, which included a holistic recommended version for each tarball, up until 7.7. To create a list of tarballs in a katamari release:

Expand All @@ -14,11 +14,11 @@ cat $(PRINT_PATH=1 nix-prefetch-url $url | tail -n 1) \
| sort > "tarballs-$release.list"
```

## Individual Tarballs
## Individual Tarballs {#individual-tarballs}

The upstream release process for [X11R7.8](https://x.org/wiki/Releases/7.8/) does not include a planned katamari. Instead, each component of X.org is released as its own tarball. We maintain `pkgs/servers/x11/xorg/tarballs.list` as a list of tarballs for each individual package. This list includes X.org core libraries and protocol descriptions, extra newer X11 interface libraries, like `xorg.libxcb`, and classic utilities which are largely unused but still available if needed, like `xorg.imake`.

## Generating Nix Expressions
## Generating Nix Expressions {#generating-nix-expressions}

The generator is invoked as follows:

Expand All @@ -29,6 +29,6 @@ cd pkgs/servers/x11/xorg

For each of the tarballs in the `.list` files, the script downloads it, unpacks it, and searches its `configure.ac` and `*.pc.in` files for dependencies. This information is used to generate `default.nix`. The generator caches downloaded tarballs between runs. Pay close attention to the `NOT FOUND: $NAME` messages at the end of the run, since they may indicate missing dependencies. (Some might be optional dependencies, however.)

## Overriding the Generator
## Overriding the Generator {#overriding-the-generator}

If the expression for a package requires derivation attributes that the generator cannot figure out automatically (say, `patches` or a `postInstall` hook), you should modify `pkgs/servers/x11/xorg/overrides.nix`.
2 changes: 1 addition & 1 deletion doc/builders/trivial-builders.chapter.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ This works just like `runCommand`. The only difference is that it also provides

Variant of `runCommand` that forces the derivation to be built locally, it is not substituted. This is intended for very cheap commands (<1s execution time). It saves on the network roundrip and can speed up a build.

::: note
::: {.note}
This sets [`allowSubstitutes` to `false`](https://nixos.org/nix/manual/#adv-attr-allowSubstitutes), so only use `runCommandLocal` if you are certain the user will always have a builder for the `system` of the derivation. This should be true for most trivial use cases (e.g. just copying some files to a different location or adding symlinks), because there the `system` is usually the same as `builtins.currentSystem`.
:::

Expand Down
4 changes: 2 additions & 2 deletions doc/contributing/coding-conventions.chapter.md
Original file line number Diff line number Diff line change
Expand Up @@ -462,9 +462,9 @@ Preferred source hash type is sha256. There are several ways to get it.

For package updates it is enough to change one symbol to make hash fake. For new packages, you can use `lib.fakeSha256`, `lib.fakeSha512` or any other fake hash.

This is last resort method when reconstructing source URL is non-trivial and `nix-prefetch-url -A` isn't applicable (for example, [one of `kodi` dependencies](https://github.com/NixOS/nixpkgs/blob/d2ab091dd308b99e4912b805a5eb088dd536adb9/pkgs/applications/video/kodi/default.nix#L73")). The easiest way then would be replace hash with a fake one and rebuild. Nix build will fail and error message will contain desired hash.
This is last resort method when reconstructing source URL is non-trivial and `nix-prefetch-url -A` isnt applicable (for example, [one of `kodi` dependencies](https://github.com/NixOS/nixpkgs/blob/d2ab091dd308b99e4912b805a5eb088dd536adb9/pkgs/applications/video/kodi/default.nix#L73)). The easiest way then would be replace hash with a fake one and rebuild. Nix build will fail and error message will contain desired hash.

::: warning
::: {.warning}
This method has security problems. Check below for details.
:::

Expand Down
2 changes: 1 addition & 1 deletion doc/contributing/contributing-to-documentation.chapter.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ If the build succeeds, the manual will be in `./result/share/doc/nixpkgs/manual.

## Syntax {#sec-contributing-markup}

As per [RFC 0062](https://github.com/NixOS/rfcs/pull/72), all new documentation content should be written in [CommonMark](https://commonmark.org/) Markdown dialect.
As per [RFC 0072](https://github.com/NixOS/rfcs/pull/72), all new documentation content should be written in [CommonMark](https://commonmark.org/) Markdown dialect.

Additionally, the following syntax extensions are currently used:

Expand Down
5 changes: 4 additions & 1 deletion doc/contributing/reviewing-contributions.chapter.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Reviewing contributions {#chap-reviewing-contributions}

::: warning
::: {.warning}
The following section is a draft, and the policy for reviewing is still being discussed in issues such as [#11166](https://github.com/NixOS/nixpkgs/issues/11166) and [#20836](https://github.com/NixOS/nixpkgs/issues/20836).
:::

Expand Down Expand Up @@ -35,15 +35,18 @@ Reviewing process:
- Building the package locally.
- pull requests are often targeted to the master or staging branch, and building the pull request locally when it is submitted can trigger many source builds.
- It is possible to rebase the changes on nixos-unstable or nixpkgs-unstable for easier review by running the following commands from a nixpkgs clone.

```ShellSession
$ git fetch origin nixos-unstable
$ git fetch origin pull/PRNUMBER/head
$ git rebase --onto nixos-unstable BASEBRANCH FETCH_HEAD
```

- The first command fetches the nixos-unstable branch.
- The second command fetches the pull request changes, `PRNUMBER` is the number at the end of the pull request title and `BASEBRANCH` the base branch of the pull request.
- The third command rebases the pull request changes to the nixos-unstable branch.
- The [nixpkgs-review](https://github.com/Mic92/nixpkgs-review) tool can be used to review a pull request content in a single command. `PRNUMBER` should be replaced by the number at the end of the pull request title. You can also provide the full github pull request url.

```ShellSession
$ nix-shell -p nixpkgs-review --run "nixpkgs-review pr PRNUMBER"
```
Expand Down
4 changes: 3 additions & 1 deletion doc/contributing/submitting-changes.chapter.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ Security fixes are submitted in the same way as other changes and thus the same
- If a new version fixing the vulnerability has been released, update the package;
- If the security fix comes in the form of a patch and a CVE is available, then add the patch to the Nixpkgs tree, and apply it to the package.
The name of the patch should be the CVE identifier, so e.g. `CVE-2019-13636.patch`; If a patch is fetched the name needs to be set as well, e.g.:

```nix
(fetchpatch {
name = "CVE-2019-11068.patch";
Expand All @@ -89,7 +90,7 @@ There is currently no policy when to remove a package.

Before removing a package, one should try to find a new maintainer or fix smaller issues first.

### Steps to remove a package from Nixpkgs
### Steps to remove a package from Nixpkgs {#steps-to-remove-a-package-from-nixpkgs}

We use jbidwatcher as an example for a discontinued project here.

Expand All @@ -100,6 +101,7 @@ We use jbidwatcher as an example for a discontinued project here.
1. Add an alias for the package name in `pkgs/top-level/aliases.nix` (There is also `pkgs/misc/vim-plugins/aliases.nix`. Package sets typically do not have aliases, so we can't add them there.)

For example in this case:

```
jbidwatcher = throw "jbidwatcher was discontinued in march 2021"; # added 2021-03-15
```
Expand Down
Loading

0 comments on commit 6ecc641

Please sign in to comment.