Skip to content

Commit 97ddb24

Browse files
authored
Update developer experience (#918)
1 parent ee7c6ba commit 97ddb24

File tree

17 files changed

+527
-132
lines changed

17 files changed

+527
-132
lines changed

.devcontainer/Dockerfile

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
FROM ghcr.io/astral-sh/uv:latest AS uv
2+
3+
FROM buildpack-deps:24.04-curl AS bin
4+
COPY --chmod=755 --from=uv /uv /usr/local/bin/uv
5+
RUN uv tool install rust-just
6+
7+
FROM mcr.microsoft.com/devcontainers/base:ubuntu-24.04
8+
9+
RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
10+
&& apt-get -y install --no-install-recommends \
11+
gdb \
12+
valgrind \
13+
libasan6 \
14+
&& apt-get autoremove -y \
15+
&& apt-get clean -y \
16+
&& rm -rf /var/lib/apt/lists/*
17+
18+
WORKDIR /workspaces/msgspec
19+
20+
COPY --chmod=755 --from=bin \
21+
/root/.local/bin/just \
22+
/usr/local/bin/
23+
COPY --chmod=755 --from=uv \
24+
/uv \
25+
/uvx \
26+
/usr/local/bin/

.devcontainer/devcontainer.json

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
{
2+
"name": "msgspec",
3+
"build": {
4+
"cacheFrom": "ghcr.io/jcrist/msgspec",
5+
"dockerfile": "Dockerfile"
6+
},
7+
"customizations": {
8+
"vscode": {
9+
"settings": {
10+
"python.defaultInterpreterPath": "/home/vscode/.local/share/msgspec/dev/venvs/test/bin/python",
11+
"python.testing.pytestEnabled": true,
12+
"[python]": {
13+
"editor.codeActionsOnSave": {
14+
"source.fixAll.ruff": "explicit",
15+
"source.organizeImports.ruff": "explicit"
16+
},
17+
"editor.defaultFormatter": "charliermarsh.ruff",
18+
"editor.formatOnSave": true
19+
},
20+
"ruff.configurationPreference": "filesystemFirst",
21+
"ruff.importStrategy": "fromEnvironment",
22+
"ruff.interpreter": ["/home/vscode/.local/share/msgspec/dev/venvs/test/bin/python"]
23+
},
24+
"extensions": [
25+
"charliermarsh.ruff",
26+
"github.vscode-github-actions",
27+
"ms-python.python",
28+
"ms-vscode.cpptools",
29+
"nefrob.vscode-just-syntax",
30+
"tamasfe.even-better-toml",
31+
"yzhang.markdown-all-in-one"
32+
]
33+
}
34+
},
35+
"postCreateCommand": "just env-sync test && just env-sync hooks",
36+
"remoteEnv": {
37+
"PYTHONUNBUFFERED": "1",
38+
"PYTHONDONTWRITEBYTECODE": "1",
39+
"UV_MANAGED_PYTHON": "1"
40+
},
41+
// Defaults for the base Ubuntu image:
42+
// https://github.com/devcontainers/images/blob/main/src/base-ubuntu/.devcontainer/devcontainer.json
43+
"features": {
44+
"ghcr.io/devcontainers/features/common-utils:2": {
45+
"configureZshAsDefaultShell": true
46+
}
47+
}
48+
}

.github/CONTRIBUTING.md

Lines changed: 98 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -7,18 +7,20 @@ contribution is at its best.
77

88
## Setting up your Development Environment
99

10+
> [!TIP]
11+
> If you're using VS Code or Cursor, you can develop inside a container using the provided `.devcontainer` configuration. This provides a pre-configured environment with all dependencies installed. Simply open the project in your editor and select "Reopen in Container" when prompted.
12+
1013
Before getting started, you will need to already have installed:
1114

12-
- Python (3.8+ only), with development headers installed
15+
- [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
16+
- [uv](https://docs.astral.sh/uv/)
1317
- A C compiler (`gcc`, `clang`, and `msvc` are all tested)
14-
- `git`
1518

1619
Once you have those installed, you're ready to:
1720

1821
- Clone the repository
19-
- Install all development dependencies
20-
- Build a development version of `msgspec`
21-
- Install the `pre-commit` hooks
22+
- Install the command runner [`just`](https://just.systems/man/en/)
23+
- (Optional) Install the Git hooks
2224

2325
```bash
2426
# Clone the repository
@@ -27,11 +29,11 @@ git clone https://github.com/jcrist/msgspec.git
2729
# cd into the repo root directory
2830
cd msgspec/
2931

30-
# Build and install msgspec & all dev dependencies
31-
pip install -e ".[dev]"
32+
# Install `just`
33+
uv tool install rust-just
3234

33-
# Install the pre-commit hooks
34-
pre-commit install
35+
# Install the Git hooks
36+
just pre-commit install
3537
```
3638

3739
## Editing and Rebuilding
@@ -41,18 +43,18 @@ can make changes to the `.py` files and test them without requiring a rebuild
4143
of msgspec's C extension. Edit away!
4244

4345
If you do make changes to a `.c` file, you'll need to recompile. You can do
44-
this by running
46+
this by running commands with the `rebuild` variable set to `true` or `1`:
4547

4648
```bash
47-
pip install -e .
49+
just rebuild=1 test
4850
```
4951

5052
By default `msgspec` is built in release mode, with optimizations enabled. To
51-
build a debug build instead (for use with e.g. `gdb` or `lldb`) define the
52-
`MSGSPEC_DEBUG` environment variable before building.
53+
build a debug build instead (for use with e.g. `gdb` or `lldb`) set the
54+
`debug` variable to `true` or `1` when running the command:
5355

5456
```bash
55-
MSGSPEC_DEBUG=1 pip install -e .
57+
just rebuild=1 debug=1 test
5658
```
5759

5860
## Testing
@@ -61,23 +63,36 @@ Tests are located in the `tests/` directory. Any code changes should include
6163
additional tests to ensure correctness. The tests are broken into various
6264
`test_*.py` files specific to the functionality that they're testing.
6365

64-
The tests can be run using `pytest` as follows:
66+
The recipes prefixed with `test-` use `pytest` to run the tests. For example:
67+
68+
```bash
69+
just test-all
70+
```
71+
72+
All such recipes accept additional arguments that are passed to `pytest`. For
73+
example, if you want to run a specific test file:
6574

6675
```bash
67-
pytest
76+
just test tests/test_json.py
6877
```
6978

70-
If you want to run a specific test file, you may specify that file explicitly:
79+
To invoke `pytest` directly, you can use the `test-env` recipe:
7180

7281
```bash
73-
pytest tests/test_json.py
82+
just test-env pytest --help
83+
```
84+
85+
To run tests with a specific version of Python, you can use the `python` variable:
86+
87+
```bash
88+
just python=3.12 test
7489
```
7590

7691
## Linting
7792

78-
We use `pre-commit` to automatically run a few code linters before every
79-
commit. If you followed the development setup above, you should already have
80-
`pre-commit` and all the commit hooks installed.
93+
We use [`prek`](https://github.com/j178/prek) to automatically run a few code
94+
linters before every commit. If you followed the development setup above, you
95+
should already have `prek` and all the Git hooks installed.
8196

8297
These hooks will run whenever you try to commit changes.
8398

@@ -88,20 +103,75 @@ git commit # linters will run automatically here
88103
If you wish to run the linters manually without committing, you can run:
89104

90105
```bash
91-
pre-commit run
106+
just hooks
107+
```
108+
109+
You may also run the same hooks without applying fixes:
110+
111+
```bash
112+
just check
92113
```
93114

94115
## Documentation
95116

96117
The source of the documentation can be found under `docs/source/`. They are
97-
built using `Sphinx` and can be built locally by running the following steps:
118+
built using `Sphinx` and can be built locally by running:
119+
120+
```bash
121+
just doc-build
122+
```
123+
124+
The built HTML documentation can be served locally by running:
125+
126+
```bash
127+
just doc-serve
128+
```
129+
130+
## Benchmarking
131+
132+
To run benchmarks, you can use the `bench-run` recipe:
133+
134+
```bash
135+
just bench-run --libs msgspec
136+
```
137+
138+
To run benchmarks against all libraries, omit the `--libs` argument.
139+
140+
## Dev Container
141+
142+
You can manually use dev containers as long as the official [`devcontainer`](https://github.com/devcontainers/cli) CLI is on PATH.
143+
144+
> [!NOTE]
145+
> If you need to install the `devcontainer` CLI and don't yet have Node.js installed, the easiest way is to use [`mise`](https://github.com/jdx/mise) (mise-en-place).
146+
>
147+
> 1. [Install](https://mise.jdx.dev/installing-mise.html) `mise`
148+
> 2. Add the [shim directory](https://mise.jdx.dev/dev-tools/shims.html#mise-activate-shims) to your PATH in one of the following ways:
149+
> 1. Manually by finding the path in the output of `mise doctor`
150+
> 2. Automatically by running `mise activate [SHELL] --shims` (see the help text for the supported shells)
151+
> 3. Install the `devcontainer` CLI with `mise use -g npm:@devcontainers/cli`
152+
153+
To start the dev container, you can use the `dev-start` recipe:
98154

99155
```bash
100-
cd docs/ # Make sure we are in the docs/ folder
156+
just dev-start
157+
```
158+
159+
To open a shell in the dev container, you can use the `dev-shell` recipe:
160+
161+
```bash
162+
just dev-shell
163+
```
101164

102-
make html # Build the html
165+
To stop the dev container, you can use the `dev-stop` recipe:
103166

104-
# Output can now be found under docs/build/html and can be viewed in the browser
167+
```bash
168+
just dev-stop
169+
```
170+
171+
To remove the dev container, you can use the `dev-remove` recipe:
172+
173+
```bash
174+
just dev-remove
105175
```
106176

107177
## Continuous Integration (CI)
@@ -112,6 +182,6 @@ and fix any issues that come up.
112182

113183
## Code of Conduct
114184

115-
``msgspec`` has a code of conduct that must be followed by all contributors to
185+
`msgspec` has a code of conduct that must be followed by all contributors to
116186
the project. You may read the code of conduct
117-
[here](https://github.com/jcrist/msgspec/blob/main/CODE_OF_CONDUCT.md).
187+
[here](https://github.com/jcrist/msgspec/blob/main/.github/CODE_OF_CONDUCT.md).

.github/env/uv.env

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,6 @@
11
# We want to test a real installation as a user.
22
UV_NO_EDITABLE=true
33

4-
# We manually select the required dependency groups rather than installing
5-
# everything with the default `dev` group and don't want to redundantly
6-
# supply the same flags to `uv run`.
7-
UV_NO_DEV=true
4+
# We manually install dependencies when required so that the `uv run` command
5+
# never modifies environments and the timing of each step is meaningful.
86
UV_NO_SYNC=true

.github/workflows/ci.yml

Lines changed: 17 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -41,21 +41,20 @@ jobs:
4141
- name: Install uv
4242
uses: astral-sh/setup-uv@v7
4343

44-
- name: Run pre-commit hooks
45-
# Run `pre-commit` in an isolated environment to prevent test pollution.
46-
run: uvx --with pre-commit-uv pre-commit run --all-files --show-diff-on-failure
44+
- name: Install command runner
45+
run: uv tool install rust-just
4746

48-
- name: Install project and test dependencies
49-
run: uv sync --group test
47+
- name: Install static analysis dependencies
48+
run: just env-sync hooks
5049

51-
- name: Test Mypy compatibility
52-
run: uv run -- pytest tests/test_mypy.py
50+
- name: Run hooks
51+
run: just hooks-check
5352

54-
- name: Test Pyright compatibility
55-
run: uv run -- pytest tests/test_pyright.py
53+
- name: Install testing dependencies
54+
run: just env-sync test
5655

57-
- name: Run Doctests
58-
run: uv run -- pytest --doctest-modules --pyargs msgspec
56+
- name: Run tests
57+
run: just test-all
5958

6059
- name: Build source distribution
6160
id: build-sdist
@@ -108,23 +107,26 @@ jobs:
108107
- name: Install uv
109108
uses: astral-sh/setup-uv@v7
110109

111-
- name: Install project with sanitizers & coverage
110+
- name: Install command runner
111+
run: uv tool install rust-just
112+
113+
- name: Install testing dependencies
112114
env:
113115
MSGSPEC_SANITIZE: "true"
114116
MSGSPEC_COVERAGE: "true"
115-
run: uv sync --group test-unit
117+
run: just env-sync test
116118

117119
- name: Run tests with sanitizers
118120
env:
119121
PYTHONMALLOC: "malloc"
120122
ASAN_OPTIONS: "detect_leaks=0"
121123
run: >-
122124
LD_PRELOAD=`gcc -print-file-name=libasan.so`
123-
uv run -- coverage run -m pytest -s
125+
just test-env coverage run -m pytest -s
124126
125127
- name: Generate coverage files
126128
run: |-
127-
uv run -- coverage xml
129+
just test-env coverage xml
128130
gcov -abcu `find build/temp*/ -name *.o`
129131
130132
- name: Upload coverage artifacts

.github/workflows/docs.yml

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -24,11 +24,14 @@ jobs:
2424
- name: Install uv
2525
uses: astral-sh/setup-uv@v7
2626

27-
- name: Install project and doc dependencies
28-
run: uv sync --group doc
27+
- name: Install command runner
28+
run: uv tool install rust-just
2929

30-
- name: Build Docs
31-
run: uv run --directory docs -- make html
30+
- name: Install dependencies
31+
run: just env-sync doc
32+
33+
- name: Build documentation
34+
run: just doc-build
3235

3336
- name: Restore link check cache
3437
uses: actions/cache@v4

0 commit comments

Comments
 (0)