[SymForce] Use uv for requirements

Last I tried this, uv didn't support solving for python versions other
than the running interpreter; now they do (except that dependencies that
don't have wheels need to get built with an actual Python interpreter).

I previously had a GitHub action to build requirements for all Python
versions; it was super fast (<1m) and kinda convenient, it'd just make a
PR.  I think I like just having the requirements test do it now though,
now that it's possible to do that way.

Topic: sf-uv

Author: aaron-skydio

.github/workflows/ci.yml

@@ -121,18 +121,11 @@ jobs:
               libgoogle-glog-dev \
               libeigen3-dev
 
-      # NOTE(aaron): Some packages do not have a version that supports py3.8..py3.12
-      - name: Fix py3.12 versions
-        if: ${{ matrix.python == '3.12' }}
-        run: |
-          sed -i 's|numba==0.58.1|numba~=0.59.0|g' dev_requirements.txt
-          sed -i 's|llvmlite==0.41.1|llvmlite~=0.42.0|g' dev_requirements.txt
-          sed -i 's|numpy==1.24.4|numpy~=1.26.0|g' dev_requirements.txt
-          sed -i 's|scipy==1.10.1|scipy~=1.12.0|g' dev_requirements.txt
-          sed -i 's|pandas==2.0.3|pandas~=2.2.0|g' dev_requirements.txt
-
       - name: Install python dependencies
-        run: pip install -r dev_requirements.txt
+        run: |
+          PY_MINOR_VERSION=$(python -c "import sys; print(sys.version_info.minor)")
+          python -m pip install pip==24.0 setuptools==69.5.1
+          python -m pip install -r requirements_dev_py3${PY_MINOR_VERSION}.txt
 
       - name: Run cmake build
         run: |
@@ -144,17 +137,11 @@ jobs:
             -D SYMFORCE_BUILD_BENCHMARKS=ON
           cmake --build build -j $(nproc)
 
-      # - lcmtypes need to be available for tests
-      # - Exact contents of dev_requirements.txt depend on python version. Need to update file to
-      # match current python version to avoid failure of corresponding gen test. symforce needs
-      # to be on the PYTHONPATH to run gen test in this manner.
+      # lcmtypes and symforce need to be available for tests
       - name: Run tests
         run: |
           pip install build/lcmtypes/python2.7
           export PYTHONPATH=$PYTHONPATH:$(pwd)
-          python${{ matrix.python }} test/symforce_requirements_test.py --update
-          echo "Modifications made to requirements:"
-          git diff
           EXIT_CODE=0
           ctest --test-dir build -j $(nproc) || EXIT_CODE=$?
           if [ $EXIT_CODE -ne 0 ]; then

.github/workflows/docs.yml

@@ -34,7 +34,9 @@ jobs:
           python-version: "3.8"
 
       - name: Install python dependencies
-        run: pip install -r dev_requirements.txt
+        run: |
+          PY_MINOR_VERSION=$(python -c "import sys; print(sys.version_info.minor)")
+          python -m pip install -r requirements_dev_py3${PY_MINOR_VERSION}.txt
 
       - name: Run cmake build
         run: |

.github/workflows/test_editable_pip_install.yml

@@ -34,18 +34,10 @@ jobs:
             pip==${{ matrix.pip_version }} \
             setuptools==${{ matrix.setuptools_version }}
 
-      # NOTE(aaron): Some packages do not have a version that supports py3.8..py3.12
-      - name: Fix py3.12 versions
-        if: ${{ matrix.os == 'ubuntu-24.04' }}
+      - name: install requirements
         run: |
-          sed -i 's|numba==0.58.1|numba~=0.59.0|g' dev_requirements.txt
-          sed -i 's|llvmlite==0.41.1|llvmlite~=0.42.0|g' dev_requirements.txt
-          sed -i 's|numpy==1.24.4|numpy~=1.26.0|g' dev_requirements.txt
-          sed -i 's|scipy==1.10.1|scipy~=1.12.0|g' dev_requirements.txt
-          sed -i 's|pandas==2.0.3|pandas~=2.2.0|g' dev_requirements.txt
-
-      - name: install dev_requirements.txt
-        run: python -m pip install -r dev_requirements.txt
+          PY_MINOR_VERSION=$(python -c "import sys; print(sys.version_info.minor)")
+          python -m pip install -r requirements_dev_py3${PY_MINOR_VERSION}.txt
 
       - name: editable install
         run: python -m pip install -v -e .

CMakeLists.txt

@@ -5,7 +5,7 @@
 
 # NOTE(aaron): This is the minimum version for policy range support, not sure if we need newer;
 # certainly no newer than 3.15 required.  This will use NEW policies up to CMake 3.25; this should
-# be the maximum tested CMake version, matching dev_requirements.txt
+# be the maximum tested CMake version, matching requirements_dev.txt
 cmake_minimum_required(VERSION 3.19...3.25)
 
 if(CMAKE_VERSION VERSION_GREATER_EQUAL 3.27)

README.md

@@ -567,7 +567,7 @@ pip install -e .
 
 You should then [verify your installation](#verify-your-installation).
 
-___Note:___ `pip install .` will not install pinned versions of SymForce's dependencies, it'll install any compatible versions.  It also won't install all packages required to run all of the SymForce tests and build all of the targets (e.g. building the docs or running the linters).  If you want all packages required for that, you should `pip install .[dev]` instead (or one of the other groups of extra requirements in our `setup.py`).  If you additionally want pinned versions of our dependencies, which are the exact versions guaranteed by CI to pass all of our tests, you can install them from `pip install -r dev_requirements.txt`.
+___Note:___ `pip install .` will not install pinned versions of SymForce's dependencies, it'll install any compatible versions.  It also won't install all packages required to run all of the SymForce tests and build all of the targets (e.g. building the docs or running the linters).  If you want all packages required for that, you should `pip install .[dev]` instead (or one of the other groups of extra requirements in our `setup.py`).  If you additionally want pinned versions of our dependencies, which are the exact versions guaranteed by CI to pass all of our tests, you can install them from `pip install -r requirements_dev_py3<version>.txt`.
 
 _Note: Editable installs as root with the system python on Ubuntu (and other Debian derivatives) are broken on `setuptools<64.0.0`.  This is a [bug in Debian](https://ffy00.github.io/blog/02-python-debian-and-the-install-locations/), not something in SymForce that we can fix.  If this is your situation, either use a virtual environment, upgrade setuptools to a version `>=64.0.0`, or use a different installation method._
 
@@ -576,11 +576,13 @@ _Note: Editable installs as root with the system python on Ubuntu (and other Deb
 If you'll be modifying the C++ parts of SymForce, you should build with CMake directly instead - this method will not install
 SymForce into your Python environment, so you'll need to add it to your PYTHONPATH separately.
 
-Install python requirements:
+Install dependencies required to build and run SymForce:
 ```bash
-pip install -r dev_requirements.txt
+pip install -r requirements_build.txt
 ```
 
+___Note:___ `requirements_build` contains only packages required to build and run symforce, but not everything recommended to develop symforce, like to run the SymForce tests and linters.  For that, install the full pinned requirements using `pip install -r requirements_dev_py3<version>.txt` for your Python version.
+
 Build SymForce (requires C++14 or later):
 ```bash
 mkdir build

pyproject.toml

@@ -199,7 +199,6 @@ module = [
   "argh.*",
   "clang_format",
   "graphviz.*",
-  "matplotlib.*",
   "mpl_toolkits.*",
   "numba.*",
   "ruff.*",
@@ -215,3 +214,12 @@ ignore_missing_imports = true
 [[tool.mypy.overrides]]
 module = "lcmtypes"
 follow_imports = "silent"
+
+# matplotlib has some overly restrictive / wrong annotations, on animation.FuncAnimation.  It also
+# requires annotating Axes3d to get proper typing
+[[tool.mypy.overrides]]
+module = "matplotlib.*"
+follow_imports = "skip"
+follow_imports_for_stubs = true
+# the py3.8 version of matplotlib has no type stubs, so we fully ignore it
+ignore_missing_imports = true

requirements_build.txt

@@ -0,0 +1,32 @@
+# This file was autogenerated by uv via the following command:
+#    python test/symforce_requirements_test.py --update
+clang-format
+    # via symforce (pyproject.toml)
+cmake
+    # via symforce (pyproject.toml)
+cython
+    # via symforce (pyproject.toml)
+graphviz
+    # via symforce (pyproject.toml)
+jinja2
+    # via symforce (pyproject.toml)
+numpy
+    # via symforce (pyproject.toml)
+pip
+    # via symforce (pyproject.toml)
+ruff
+    # via symforce (pyproject.toml)
+scipy
+    # via symforce (pyproject.toml)
+setuptools
+    # via symforce (pyproject.toml)
+setuptools-scm
+    # via symforce (pyproject.toml)
+file:./third_party/skymarshal
+    # via symforce (pyproject.toml)
+file:./gen/python
+    # via symforce (pyproject.toml)
+sympy
+    # via symforce (pyproject.toml)
+wheel
+    # via symforce (pyproject.toml)