Skip to content

Commit 4872786

Browse files
committed
Update developer experience
1 parent 24b7a93 commit 4872786

File tree

17 files changed

+521
-132
lines changed

17 files changed

+521
-132
lines changed

.devcontainer/Dockerfile

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
FROM ghcr.io/astral-sh/uv:latest AS uv
2+
3+
FROM buildpack-deps:24.04-curl AS bin
4+
COPY --chmod=755 --from=uv /uv /usr/local/bin/uv
5+
RUN uv tool install rust-just
6+
7+
FROM mcr.microsoft.com/devcontainers/base:ubuntu-24.04
8+
9+
RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
10+
&& apt-get -y install --no-install-recommends \
11+
gdb \
12+
valgrind \
13+
libasan6 \
14+
&& apt-get autoremove -y \
15+
&& apt-get clean -y \
16+
&& rm -rf /var/lib/apt/lists/*
17+
18+
WORKDIR /workspaces/msgspec
19+
20+
COPY --chmod=755 --from=bin \
21+
/root/.local/bin/just \
22+
/usr/local/bin/
23+
COPY --chmod=755 --from=uv \
24+
/uv \
25+
/uvx \
26+
/usr/local/bin/

.devcontainer/devcontainer.json

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
{
2+
"name": "msgspec",
3+
"build": {
4+
"cacheFrom": "ghcr.io/jcrist/msgspec",
5+
"dockerfile": "Dockerfile"
6+
},
7+
"customizations": {
8+
"vscode": {
9+
"settings": {
10+
"python.defaultInterpreterPath": "/home/vscode/.local/share/msgspec/dev/venvs/test/bin/python",
11+
"python.testing.pytestEnabled": true,
12+
"[python]": {
13+
"editor.codeActionsOnSave": {
14+
"source.fixAll.ruff": "explicit",
15+
"source.organizeImports.ruff": "explicit"
16+
},
17+
"editor.defaultFormatter": "charliermarsh.ruff",
18+
"editor.formatOnSave": true
19+
},
20+
"ruff.configurationPreference": "filesystemFirst",
21+
"ruff.importStrategy": "fromEnvironment",
22+
"ruff.interpreter": ["/home/vscode/.local/share/msgspec/dev/venvs/test/bin/python"]
23+
},
24+
"extensions": [
25+
"charliermarsh.ruff",
26+
"github.vscode-github-actions",
27+
"ms-python.python",
28+
"ms-vscode.cpptools",
29+
"nefrob.vscode-just-syntax",
30+
"tamasfe.even-better-toml",
31+
"yzhang.markdown-all-in-one"
32+
]
33+
}
34+
},
35+
"postCreateCommand": "just env-sync test && just env-sync hooks",
36+
"remoteEnv": {
37+
"PYTHONUNBUFFERED": "1",
38+
"PYTHONDONTWRITEBYTECODE": "1",
39+
"UV_MANAGED_PYTHON": "1"
40+
},
41+
// Defaults for the base Ubuntu image:
42+
// https://github.com/devcontainers/images/blob/main/src/base-ubuntu/.devcontainer/devcontainer.json
43+
"features": {
44+
"ghcr.io/devcontainers/features/common-utils:2": {
45+
"configureZshAsDefaultShell": true
46+
}
47+
}
48+
}

.github/CONTRIBUTING.md

Lines changed: 92 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -7,18 +7,20 @@ contribution is at its best.
77

88
## Setting up your Development Environment
99

10+
> [!TIP]
11+
> If you're using VS Code or Cursor, you can develop inside a container using the provided `.devcontainer` configuration. This provides a pre-configured environment with all dependencies installed. Simply open the project in your editor and select "Reopen in Container" when prompted.
12+
1013
Before getting started, you will need to already have installed:
1114

12-
- Python (3.8+ only), with development headers installed
15+
- [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
16+
- [uv](https://docs.astral.sh/uv/)
1317
- A C compiler (`gcc`, `clang`, and `msvc` are all tested)
14-
- `git`
1518

1619
Once you have those installed, you're ready to:
1720

1821
- Clone the repository
19-
- Install all development dependencies
20-
- Build a development version of `msgspec`
21-
- Install the `pre-commit` hooks
22+
- Install the command runner [`just`](https://just.systems/man/en/)
23+
- (Optional) Install the Git hooks
2224

2325
```bash
2426
# Clone the repository
@@ -27,11 +29,11 @@ git clone https://github.com/jcrist/msgspec.git
2729
# cd into the repo root directory
2830
cd msgspec/
2931

30-
# Build and install msgspec & all dev dependencies
31-
pip install -e ".[dev]"
32+
# Install `just`
33+
uv tool install rust-just
3234

33-
# Install the pre-commit hooks
34-
pre-commit install
35+
# Install the Git hooks
36+
just pre-commit install
3537
```
3638

3739
## Editing and Rebuilding
@@ -41,18 +43,18 @@ can make changes to the `.py` files and test them without requiring a rebuild
4143
of msgspec's C extension. Edit away!
4244

4345
If you do make changes to a `.c` file, you'll need to recompile. You can do
44-
this by running
46+
this by running commands with the `rebuild` variable set to `true` or `1`:
4547

4648
```bash
47-
pip install -e .
49+
just rebuild=1 test
4850
```
4951

5052
By default `msgspec` is built in release mode, with optimizations enabled. To
51-
build a debug build instead (for use with e.g. `gdb` or `lldb`) define the
52-
`MSGSPEC_DEBUG` environment variable before building.
53+
build a debug build instead (for use with e.g. `gdb` or `lldb`) set the
54+
`debug` variable to `true` or `1` when running the command:
5355

5456
```bash
55-
MSGSPEC_DEBUG=1 pip install -e .
57+
just rebuild=1 debug=1 test
5658
```
5759

5860
## Testing
@@ -61,23 +63,30 @@ Tests are located in the `tests/` directory. Any code changes should include
6163
additional tests to ensure correctness. The tests are broken into various
6264
`test_*.py` files specific to the functionality that they're testing.
6365

64-
The tests can be run using `pytest` as follows:
66+
The recipes prefixed with `test-` use `pytest` to run the tests. For example:
67+
68+
```bash
69+
just test-all
70+
```
71+
72+
All such recipes accept additional arguments that are passed to `pytest`. For
73+
example, if you want to run a specific test file:
6574

6675
```bash
67-
pytest
76+
just test tests/test_json.py
6877
```
6978

70-
If you want to run a specific test file, you may specify that file explicitly:
79+
To invoke `pytest` directly, you can use the `test-env` recipe:
7180

7281
```bash
73-
pytest tests/test_json.py
82+
just test-env pytest --help
7483
```
7584

7685
## Linting
7786

78-
We use `pre-commit` to automatically run a few code linters before every
79-
commit. If you followed the development setup above, you should already have
80-
`pre-commit` and all the commit hooks installed.
87+
We use [`prek`](https://github.com/j178/prek) to automatically run a few code
88+
linters before every commit. If you followed the development setup above, you
89+
should already have `prek` and all the Git hooks installed.
8190

8291
These hooks will run whenever you try to commit changes.
8392

@@ -88,20 +97,75 @@ git commit # linters will run automatically here
8897
If you wish to run the linters manually without committing, you can run:
8998

9099
```bash
91-
pre-commit run
100+
just hooks
101+
```
102+
103+
You may also run the same hooks without applying fixes:
104+
105+
```bash
106+
just check
92107
```
93108

94109
## Documentation
95110

96111
The source of the documentation can be found under `docs/source/`. They are
97-
built using `Sphinx` and can be built locally by running the following steps:
112+
built using `Sphinx` and can be built locally by running:
113+
114+
```bash
115+
just doc-build
116+
```
117+
118+
The built HTML documentation can be served locally by running:
119+
120+
```bash
121+
just doc-serve
122+
```
123+
124+
## Benchmarking
125+
126+
To run benchmarks, you can use the `bench-run` recipe:
98127

99128
```bash
100-
cd docs/ # Make sure we are in the docs/ folder
129+
just bench-run --libs msgspec
130+
```
131+
132+
To run benchmarks against all libraries, omit the `--libs` argument.
133+
134+
## Dev Container
135+
136+
You can manually use dev containers as long as the official [`devcontainer`](https://github.com/devcontainers/cli) CLI is on PATH.
137+
138+
> [!NOTE]
139+
> If you need to install the `devcontainer` CLI and don't yet have Node.js installed, the easiest way is to use [`mise`](https://github.com/jdx/mise) (mise-en-place).
140+
>
141+
> 1. [Install](https://mise.jdx.dev/installing-mise.html) `mise`
142+
> 2. Add the [shim directory](https://mise.jdx.dev/dev-tools/shims.html#mise-activate-shims) to your PATH in one of the following ways:
143+
> 1. Manually by finding the path in the output of `mise doctor`
144+
> 2. Automatically by running `mise activate [SHELL] --shims` (see the help text for the supported shells)
145+
> 3. Install the `devcontainer` CLI with `mise use -g npm:@devcontainers/cli`
146+
147+
To start the dev container, you can use the `dev-start` recipe:
148+
149+
```bash
150+
just dev-start
151+
```
101152

102-
make html # Build the html
153+
To open a shell in the dev container, you can use the `dev-shell` recipe:
103154

104-
# Output can now be found under docs/build/html and can be viewed in the browser
155+
```bash
156+
just dev-shell
157+
```
158+
159+
To stop the dev container, you can use the `dev-stop` recipe:
160+
161+
```bash
162+
just dev-stop
163+
```
164+
165+
To remove the dev container, you can use the `dev-remove` recipe:
166+
167+
```bash
168+
just dev-remove
105169
```
106170

107171
## Continuous Integration (CI)
@@ -112,6 +176,6 @@ and fix any issues that come up.
112176

113177
## Code of Conduct
114178

115-
``msgspec`` has a code of conduct that must be followed by all contributors to
179+
`msgspec` has a code of conduct that must be followed by all contributors to
116180
the project. You may read the code of conduct
117-
[here](https://github.com/jcrist/msgspec/blob/main/CODE_OF_CONDUCT.md).
181+
[here](https://github.com/jcrist/msgspec/blob/main/.github/CODE_OF_CONDUCT.md).

.github/env/uv.env

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,6 @@
11
# We want to test a real installation as a user.
22
UV_NO_EDITABLE=true
33

4-
# We manually select the required dependency groups rather than installing
5-
# everything with the default `dev` group and don't want to redundantly
6-
# supply the same flags to `uv run`.
7-
UV_NO_DEV=true
4+
# We manually install dependencies when required so that the `uv run` command
5+
# never modifies environments and the timing of each step is meaningful.
86
UV_NO_SYNC=true

.github/workflows/ci.yml

Lines changed: 17 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -41,21 +41,20 @@ jobs:
4141
- name: Install uv
4242
uses: astral-sh/setup-uv@v7
4343

44-
- name: Run pre-commit hooks
45-
# Run `pre-commit` in an isolated environment to prevent test pollution.
46-
run: uvx --with pre-commit-uv pre-commit run --all-files --show-diff-on-failure
44+
- name: Install command runner
45+
run: uv tool install rust-just
4746

48-
- name: Install project and test dependencies
49-
run: uv sync --group test
47+
- name: Install static analysis dependencies
48+
run: just env-sync hooks
5049

51-
- name: Test Mypy compatibility
52-
run: uv run -- pytest tests/test_mypy.py
50+
- name: Run hooks
51+
run: just hooks-check
5352

54-
- name: Test Pyright compatibility
55-
run: uv run -- pytest tests/test_pyright.py
53+
- name: Install testing dependencies
54+
run: just env-sync test
5655

57-
- name: Run Doctests
58-
run: uv run -- pytest --doctest-modules --pyargs msgspec
56+
- name: Run tests
57+
run: just test-all
5958

6059
- name: Build source distribution
6160
id: build-sdist
@@ -108,23 +107,26 @@ jobs:
108107
- name: Install uv
109108
uses: astral-sh/setup-uv@v7
110109

111-
- name: Install project with sanitizers & coverage
110+
- name: Install command runner
111+
run: uv tool install rust-just
112+
113+
- name: Install testing dependencies
112114
env:
113115
MSGSPEC_SANITIZE: "true"
114116
MSGSPEC_COVERAGE: "true"
115-
run: uv sync --group test-unit
117+
run: just env-sync test
116118

117119
- name: Run tests with sanitizers
118120
env:
119121
PYTHONMALLOC: "malloc"
120122
ASAN_OPTIONS: "detect_leaks=0"
121123
run: >-
122124
LD_PRELOAD=`gcc -print-file-name=libasan.so`
123-
uv run -- coverage run -m pytest -s
125+
just test-env coverage run -m pytest -s
124126
125127
- name: Generate coverage files
126128
run: |-
127-
uv run -- coverage xml
129+
just test-env coverage xml
128130
gcov -abcu `find build/temp*/ -name *.o`
129131
130132
- name: Upload coverage artifacts

.github/workflows/docs.yml

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -24,11 +24,14 @@ jobs:
2424
- name: Install uv
2525
uses: astral-sh/setup-uv@v7
2626

27-
- name: Install project and doc dependencies
28-
run: uv sync --group doc
27+
- name: Install command runner
28+
run: uv tool install rust-just
2929

30-
- name: Build Docs
31-
run: uv run --directory docs -- make html
30+
- name: Install dependencies
31+
run: just env-sync doc
32+
33+
- name: Build documentation
34+
run: just doc-build
3235

3336
- name: Restore link check cache
3437
uses: actions/cache@v4

0 commit comments

Comments
 (0)