Skip to content

Commit

Permalink
docs: more structured toc (#1142)
Browse files Browse the repository at this point in the history
Co-authored-by: Ruben Arts <ruben@prefix.dev>
Co-authored-by: Ruben Arts <ruben.arts@hotmail.com>
  • Loading branch information
3 people authored Apr 9, 2024
1 parent bf02059 commit cb4bcbc
Show file tree
Hide file tree
Showing 14 changed files with 310 additions and 138 deletions.
5 changes: 2 additions & 3 deletions docs/advanced/explain_info_command.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@ default
Tasks: docs, test-all, test, build, lint, install, build-docs
```


## Global info

The first part of the info output is information that is always available and tells you what pixi can read on your machine.
Expand Down Expand Up @@ -84,11 +83,11 @@ The size of the previously mentioned "Cache dir" in Mebibytes.
## Project info

Everything below `Project` is info about the project you're currently in.
This info is only available if your path has a [manifest file](../configuration.md).
This info is only available if your path has a [manifest file](../reference/configuration.md).

### Manifest file

The path to the [manifest file](../configuration.md) that describes the project.
The path to the [manifest file](../reference/configuration.md) that describes the project.

### Last updated

Expand Down
37 changes: 29 additions & 8 deletions docs/advanced/pyproject_toml.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,35 @@
# `pyproject.toml` in pixi

We support the use of the `pyproject.toml` as our manifest file in pixi.
This allows the user to keep one file with all configuration.
The `pyproject.toml` file is a standard for Python projects.
We don't advise to use the `pyproject.toml` file for anything else than python projects, the `pixi.toml` is better suited for other types of projects.

## Initial setup of the `pyproject.toml` file

When you already have a `pyproject.toml` file in your project, you can add the following section to it:

```toml
[tool.pixi.project]
channels = ["conda-forge"]
platforms = ["linux-64", "osx-arm64", "osx-64", "win-64"]
```

This is the minimum requirement for pixi to understand and parse the project.

However, it is recommended you use `pixi init` in a folder that has a `pyproject.toml` file. Pixi will automatically

- Add the above `[tool.pixi.project]` section to the file, auto-detecting your current platform;
- Add the current project as an editable pypi dependency;
- Add some defaults to the `.gitignore` and `.gitattributes` file.
- Add the above `[tool.pixi.project]` section to the file, auto-detecting your current platform;
- Add the current project as an editable pypi dependency;
- Add some defaults to the `.gitignore` and `.gitattributes` file.

## Python dependency

The `pyproject.toml` file supports the `requires_python` field.
Pixi understands that field and automatically adds the version to the dependencies.

This is an example of a `pyproject.toml` file with the `requires_python` field, which will be used as the python dependency:

```toml title="pyproject.toml"
[project]
name = "my_project"
Expand All @@ -33,7 +39,9 @@ requires-python = ">=3.9"
channels = ["conda-forge"]
platforms = ["linux-64", "osx-arm64", "osx-64", "win-64"]
```

Which is equivalent to:

```toml title="equivalent pixi.toml"
[project]
name = "my_project"
Expand All @@ -45,10 +53,12 @@ python = ">=3.9"
```

## Dependency section

The `pyproject.toml` file supports the `dependencies` field.
Pixi understands that field and automatically adds the dependencies to the project as `[pypi-dependencies]`.

This is an example of a `pyproject.toml` file with the `dependencies` field:

```toml title="pyproject.toml"
[project]
name = "my_project"
Expand All @@ -65,6 +75,7 @@ platforms = ["linux-64", "osx-arm64", "osx-64", "win-64"]
```

Which is equivalent to:

```toml title="equivalent pixi.toml"
[project]
name = "my_project"
Expand All @@ -81,6 +92,7 @@ python = ">=3.9"
```

You can overwrite these with conda dependencies by adding them to the `dependencies` field:

```toml title="pyproject.toml"
[project]
name = "my_project"
Expand All @@ -100,15 +112,18 @@ numpy = "*"
pandas = "*"
matplotlib = "*"
```

This would result in the conda dependencies being installed and the pypi dependencies being ignored.
As pixi takes the conda dependencies over the pypi dependencies.

## Optional dependencies
If your python project includes groups of optional dependencies, pixi will automatically interpret them as [pixi features](../configuration.md#the-feature-table) of the same name with the associated `pypi-dependencies`.

If your python project includes groups of optional dependencies, pixi will automatically interpret them as [pixi features](../reference/configuration.md#the-feature-table) of the same name with the associated `pypi-dependencies`.

You can add them to pixi environments manually, or use `pixi init` to setup the project, which will create one environment per feature. Self-references to other groups of optional dependencies are also handled.

For instance, imagine you have a project folder with a `pyproject.toml` file similar to:

```toml
[project]
name = "my_project"
Expand All @@ -120,6 +135,7 @@ all = ["package2","my_project[test]"]
```

Running `pixi init` in that project folder will transform the `pyproject.toml` file into:

```toml
[project]
name = "my_project"
Expand All @@ -138,16 +154,19 @@ default = {features = [], solve-group = "default"}
test = {features = ["test"], solve-group = "default"}
all = {features = ["all", "test"], solve-group = "default"}
```

In this example, three environments will be created by pixi:

- **default** with 'package1' as pypi dependency
- **test** with 'package1' and 'pytest' as pypi dependencies
- **all** with 'package1', 'package2' and 'pytest' as pypi dependencies
- **default** with 'package1' as pypi dependency
- **test** with 'package1' and 'pytest' as pypi dependencies
- **all** with 'package1', 'package2' and 'pytest' as pypi dependencies

All environments will be solved together, as indicated by the common `solve-group`, and added to the lock file. You can edit the `[tool.pixi.environments]` section manually to adapt it to your use case (e.g. if you do not need a particular environment).

## Example

As the `pyproject.toml` file supports the full pixi spec with `[tool.pixi]` prepended an example would look like this:

```toml title="pyproject.toml"
[project]
name = "my_project"
Expand Down Expand Up @@ -185,7 +204,8 @@ test = ["test"]
```

## Build-system section
The `pyproject.toml` file normally contains a `[build-system]` section. Pixi will use this section to build and install the project if it is added as a pypi path dependency.

The `pyproject.toml` file normally contains a `[build-system]` section. Pixi will use this section to build and install the project if it is added as a pypi path dependency.

If the `pyproject.toml` file does not contain any `[build-system]` section, pixi will fall back to [uv](https://github.com/astral-sh/uv)'s default, which is equivalent to the below:

Expand All @@ -194,6 +214,7 @@ If the `pyproject.toml` file does not contain any `[build-system]` section, pixi
requires = ["setuptools >= 40.8.0"]
build-backend = "setuptools.build_meta:__legacy__"
```

Including a `[build-system]` section is **highly recommended**. If you are not sure of the [build-backend](https://packaging.python.org/en/latest/tutorials/packaging-projects/#choosing-build-backend) you want to use, including the `[build-system]` section below in your `pyproject.toml` is a good starting point

```toml title="pyproject.toml"
Expand Down
2 changes: 1 addition & 1 deletion docs/examples/cpp-sdl.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Run the `start` command
pixi run start
```

Using the [`depends_on`](../advanced/advanced_tasks.md#depends-on) feature you only needed to run the `start` task but under water it is running the following tasks.
Using the [`depends_on`](../features/advanced_tasks.md#depends-on) feature you only needed to run the `start` task but under water it is running the following tasks.

```shell
# Configure the CMake project
Expand Down
41 changes: 23 additions & 18 deletions docs/advanced/advanced_tasks.md → docs/features/advanced_tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,24 +105,28 @@ pixi run style
```

## Working directory

Pixi tasks support the definition of a working directory.

`cwd`" stands for Current Working Directory.
The directory is relative to the pixi package root, where the `pixi.toml` file is located.

Consider a pixi project structured as follows:

```shell
├── pixi.toml
└── scripts
└── bar.py
```

To add a task to run the `bar.py` file, use:

```shell
pixi task add bar "python bar.py" --cwd scripts
```

This will add the following line to [manifest file](../configuration.md):
This will add the following line to [manifest file](../reference/configuration.md):

```toml title="pixi.toml"
[tasks]
bar = { cmd = "python bar.py", cwd = "scripts" }
Expand Down Expand Up @@ -170,6 +174,7 @@ This is [`deno_task_shell`](https://deno.land/manual@v1.35.0/tools/task_runner#b
The task shell is a limited implementation of a bourne-shell interface.

### Built-in commands

Next to running actual executable like `./myprogram`, `cmake` or `python` the shell has some built-in commandos.

- `cp`: Copies files.
Expand All @@ -190,30 +195,30 @@ Next to running actual executable like `./myprogram`, `cmake` or `python` the sh
### Syntax

- **Boolean list:** use `&&` or `||` to separate two commands.
- `&&`: if the command before `&&` succeeds continue with the next command.
- `||`: if the command before `||` fails continue with the next command.
- `&&`: if the command before `&&` succeeds continue with the next command.
- `||`: if the command before `||` fails continue with the next command.
- **Sequential lists:** use `;` to run two commands without checking if the first command failed or succeeded.
- **Environment variables:**
- Set env variable using: `export ENV_VAR=value`
- Use env variable using: `$ENV_VAR`
- unset env variable using `unset ENV_VAR`
- Set env variable using: `export ENV_VAR=value`
- Use env variable using: `$ENV_VAR`
- unset env variable using `unset ENV_VAR`
- **Shell variables:** Shell variables are similar to environment variables, but won’t be exported to spawned commands.
- Set them: `VAR=value`
- use them: `VAR=value && echo $VAR`
- Set them: `VAR=value`
- use them: `VAR=value && echo $VAR`
- **Pipelines:** Use the stdout output of a command into the stdin a following command
- `|`: `echo Hello | python receiving_app.py`
- `|&`: use this to also get the stderr as input.
- `|`: `echo Hello | python receiving_app.py`
- `|&`: use this to also get the stderr as input.
- **Command substitution:** `$()` to use the output of a command as input for another command.
- `python main.py $(git rev-parse HEAD)`
- `python main.py $(git rev-parse HEAD)`
- **Negate exit code:** `! ` before any command will negate the exit code from 1 to 0 or visa-versa.
- **Redirects:** `>` to redirect the stdout to a file.
- `echo hello > file.txt` will put `hello` in `file.txt` and overwrite existing text.
- `python main.py 2> file.txt` will put the `stderr` output in `file.txt`.
- `python main.py &> file.txt` will put the `stderr` **and** `stdout` in `file.txt`.
- `echo hello > file.txt` will append `hello` to the existing `file.txt`.
- `echo hello > file.txt` will put `hello` in `file.txt` and overwrite existing text.
- `python main.py 2> file.txt` will put the `stderr` output in `file.txt`.
- `python main.py &> file.txt` will put the `stderr` **and** `stdout` in `file.txt`.
- `echo hello > file.txt` will append `hello` to the existing `file.txt`.
- **Glob expansion:** `*` to expand all options.
- `echo *.py` will echo all filenames that end with `.py`
- `echo **/*.py` will echo all filenames that end with `.py` in this directory and all descendant directories.
- `echo data[0-9].csv` will echo all filenames that have a single number after `data` and before `.csv`
- `echo *.py` will echo all filenames that end with `.py`
- `echo **/*.py` will echo all filenames that end with `.py` in this directory and all descendant directories.
- `echo data[0-9].csv` will echo all filenames that have a single number after `data` and before `.csv`

More info in [`deno_task_shell` documentation](https://deno.land/manual@v1.35.0/tools/task_runner#task-runner).
13 changes: 12 additions & 1 deletion docs/environment.md → docs/features/environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,12 @@ description: The resulting environment of a pixi installation.
---

# Environments

Pixi is a tool to manage virtual environments.
This document explains what an environment looks like and how to use it.

## Structure

A pixi environment is located in the `.pixi/envs` directory of the project.
This location is **not** configurable as it is a specific design decision to keep the environments in the project directory.
This keeps your machine and your project clean and isolated from each other, and makes it easy to clean up after a project is done.
Expand Down Expand Up @@ -39,7 +41,9 @@ Pixi will always make sure the environment is in sync with the `pixi.lock` file.
If this is not the case then all the commands that use the environment will automatically update the environment, e.g. `pixi run`, `pixi shell`.

### Cleaning up

If you want to clean up the environments, you can simply delete the `.pixi/envs` directory, and pixi will recreate the environments when needed.

```shell
# either:
rm -rf .pixi/envs
Expand All @@ -50,20 +54,23 @@ rm -rf .pixi/envs/cuda
```

## Activation

An environment is nothing more than a set of files that are installed into a certain location, that somewhat mimics a global system install.
You need to activate the environment to use it.
In the most simple sense that mean adding the `bin` directory of the environment to the `PATH` variable.
But there is more to it in a conda environment, as it also sets some environment variables.

To do the activation we have multiple options:

- Use the `pixi shell` command to open a shell with the environment activated.
- Use the `pixi shell-hook` command to print the command to activate the environment in your current shell.
- Use the `pixi run` command to run a command in the environment.

Where the `run` command is special as it runs its own cross-platform shell and has the ability to run tasks.
More information about tasks can be found in the [tasks documentation](advanced/advanced_tasks.md).
More information about tasks can be found in the [tasks documentation](advanced_tasks.md).

Using the `pixi shell-hook` in pixi you would get the following output:

```shell
export PATH="/home/user/development/pixi/.pixi/envs/default/bin:/home/user/.local/bin:/home/user/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/home/user/.pixi/bin"
export CONDA_PREFIX="/home/user/development/pixi/.pixi/envs/default"
Expand All @@ -82,6 +89,7 @@ export PIXI_PROMPT="(pixi) "
. "/home/user/development/pixi/.pixi/envs/default/etc/conda/activate.d/libglib_activate.sh"
. "/home/user/development/pixi/.pixi/envs/default/etc/conda/activate.d/rust.sh"
```

It sets the `PATH` and some more environment variables. But more importantly it also runs activation scripts that are presented by the installed packages.
An example of this would be the [`libglib_activate.sh`](https://github.com/conda-forge/glib-feedstock/blob/52ba1944dffdb2d882d824d6548325155b58819b/recipe/scripts/activate.sh) script.
Thus, just adding the `bin` directory to the `PATH` is not enough.
Expand Down Expand Up @@ -130,6 +138,7 @@ python not found
```

## Environment variables

The following environment variables are set by pixi, when using the `pixi run`, `pixi shell`, or `pixi shell-hook` command:

- `PIXI_PROJECT_ROOT`: The root directory of the project.
Expand All @@ -147,6 +156,7 @@ The following environment variables are set by pixi, when using the `pixi run`,
Even though the variables are environment variables these cannot be overridden. E.g. you can not change the root of the project by setting `PIXI_PROJECT_ROOT` in the environment.

## Solving environments

When you run a command that uses the environment, pixi will check if the environment is in sync with the `pixi.lock` file.
If it is not, pixi will solve the environment and update it.
This means that pixi will retrieve the best set of packages for the dependency requirements that you specified in the `pixi.toml` and will put the output of the solve step into the `pixi.lock` file.
Expand All @@ -166,6 +176,7 @@ For this building step, `pixi` requires to first install `python` in the (conda)
This will always be slower than the pure conda solves. So for the best pixi experience you should stay within the `[dependencies]` section of the `pixi.toml` file.

## Caching

Pixi caches the packages used in the environment.
So if you have multiple projects that use the same packages, pixi will only download the packages once.

Expand Down
Loading

0 comments on commit cb4bcbc

Please sign in to comment.