Test against Python 3.13 pre-release

[simonw]

I learned about allow-prerelease: true from https://github.com/dateutil/dateutil/pull/1337/files

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.69%. Comparing base (17eb818) to head (ddf5ce1).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #619   +/-   ##
=======================================
  Coverage   95.69%   95.69%           
=======================================
  Files           8        8           
  Lines        2854     2854           
=======================================
  Hits         2731     2731           
  Misses        123      123           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[simonw]

Tests failed attempting to install numpy I think:

      Message:
      CPU Optimization Options
        baseline:
          Requested : min
          Enabled   : SSE SSE2 SSE3
        dispatch:
          Requested : max -xop -fma4
          Enabled   : SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F AVX512CD AVX512_KNL AVX512_KNM AVX512_SKX AVX512_CLX AVX512_CNL AVX512_ICL
      
      Library m found: YES
      Run-time dependency scipy-openblas found: NO (tried pkgconfig)
      Run-time dependency mkl found: NO (tried pkgconfig and system)
      Run-time dependency mkl found: NO (tried pkgconfig and system)
      Run-time dependency accelerate found: NO (tried system)
      Found CMake: /usr/local/bin/cmake (3.28.3)
      WARNING: CMake Toolchain: Failed to determine CMake compilers state
      
      ../numpy/meson.build:124:13: ERROR: Unhandled python exception
      
          This is a Meson bug and should be reported!
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

[simonw]

https://github.com/simonw/sqlite-utils/actions/runs/8290227555/job/22687937164

A bunch of new warnings:

tests/test_insert_files.py::test_insert_files[True]
  /home/runner/work/sqlite-utils/sqlite-utils/sqlite_utils/cli.py:3195: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
    "mtime_iso": lambda p: datetime.utcfromtimestamp(p.stat().st_mtime).isoformat(),
tests/test_insert_files.py::test_insert_files[False]
tests/test_insert_files.py::test_insert_files[False]
tests/test_insert_files.py::test_insert_files[False]
tests/test_insert_files.py::test_insert_files[True]
tests/test_insert_files.py::test_insert_files[True]
tests/test_insert_files.py::test_insert_files[True]
  /home/runner/work/sqlite-utils/sqlite-utils/sqlite_utils/cli.py:3196: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
    "ctime_iso": lambda p: datetime.utcfromtimestamp(p.stat().st_ctime).isoformat(),
tests/test_recipes.py::test_dateparse_errors[None-parsedate]
  /opt/hostedtoolcache/Python/3.13.0-alpha.5/x64/lib/python3.13/site-packages/_pytest/unraisableexception.py:80: PytestUnraisableExceptionWarning: Exception ignored in: <function Table.convert.<locals>.convert_value at 0x7a02ebf3ab60>
  
  Traceback (most recent call last):
    File "/home/runner/work/sqlite-utils/sqlite-utils/sqlite_utils/db.py", line 2841, in convert_value
      return jsonify_if_needed(fn(v))
                               ~~^^^
    File "/home/runner/work/sqlite-utils/sqlite-utils/tests/test_recipes.py", line 77, in <lambda>
      fresh_db["example"].convert("dt", lambda value: getattr(recipes, fn)(value))
                                                      ~~~~~~~~~~~~~~~~~~~~^^^^^^^
    File "/home/runner/work/sqlite-utils/sqlite-utils/sqlite_utils/recipes.py", line 19, in parsedate
      parser.parse(value, dayfirst=dayfirst, yearfirst=yearfirst)
      ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/opt/hostedtoolcache/Python/3.13.0-alpha.5/x64/lib/python3.13/site-packages/dateutil/parser/_parser.py", line 1368, in parse
      return DEFAULTPARSER.parse(timestr, **kwargs)
             ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
    File "/opt/hostedtoolcache/Python/3.13.0-alpha.5/x64/lib/python3.13/site-packages/dateutil/parser/_parser.py", line 643, in parse
      raise ParserError("Unknown string format: %s", timestr)
  dateutil.parser._parser.ParserError: Unknown string format: invalid
  
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
tests/test_recipes.py::test_dateparse_errors[None-parsedatetime]
  /opt/hostedtoolcache/Python/3.13.0-alpha.5/x64/lib/python3.13/site-packages/_pytest/unraisableexception.py:80: PytestUnraisableExceptionWarning: Exception ignored in: <function Table.convert.<locals>.convert_value at 0x7a02ebeb8a40>
  
  Traceback (most recent call last):
    File "/home/runner/work/sqlite-utils/sqlite-utils/sqlite_utils/db.py", line 2841, in convert_value
      return jsonify_if_needed(fn(v))
                               ~~^^^
    File "/home/runner/work/sqlite-utils/sqlite-utils/tests/test_recipes.py", line 77, in <lambda>
      fresh_db["example"].convert("dt", lambda value: getattr(recipes, fn)(value))
                                                      ~~~~~~~~~~~~~~~~~~~~^^^^^^^
    File "/home/runner/work/sqlite-utils/sqlite-utils/sqlite_utils/recipes.py", line 42, in parsedatetime
      return parser.parse(value, dayfirst=dayfirst, yearfirst=yearfirst).isoformat()
             ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/opt/hostedtoolcache/Python/3.13.0-alpha.5/x64/lib/python3.13/site-packages/dateutil/parser/_parser.py", line 1368, in parse
      return DEFAULTPARSER.parse(timestr, **kwargs)
             ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
    File "/opt/hostedtoolcache/Python/3.13.0-alpha.5/x64/lib/python3.13/site-packages/dateutil/parser/_parser.py", line 643, in parse
      raise ParserError("Unknown string format: %s", timestr)
  dateutil.parser._parser.ParserError: Unknown string format: invalid
  
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

Weirdly the tests failed on cog:

Checking README.md
Checking docs/changelog.rst
Checking docs/cli-reference.rst  (changed)

[simonw]

I installed Python 3.13 locally like this:

pyenv install 3.13-dev
cd /tmp
mkdir hello
cd hello
pyenv local 3.13-dev
cat .python-version
eval "$(pyenv init -)"

git clone git@github.com:simonw/sqlite-utils
cd sqlite-utils
python -m venv venv
source venv/bin/activate
pip install -e '.[docs]'
cog -r docs/*.rst

Now if I run cog -r docs/*.rst it produces this diff:

diff --git a/docs/cli-reference.rst b/docs/cli-reference.rst
index 55517c9..04c4a59 100644
--- a/docs/cli-reference.rst
+++ b/docs/cli-reference.rst
@@ -615,25 +615,25 @@ See :ref:`cli_convert`.
 
       r.jsonsplit(value, delimiter=',', type=<class 'str'>)
 
-          Convert a string like a,b,c into a JSON array ["a", "b", "c"]
+      Convert a string like a,b,c into a JSON array ["a", "b", "c"]
 
       r.parsedate(value, dayfirst=False, yearfirst=False, errors=None)
 
-          Parse a date and convert it to ISO date format: yyyy-mm-dd
-         
-          - dayfirst=True: treat xx as the day in xx/yy/zz
-          - yearfirst=True: treat xx as the year in xx/yy/zz
-          - errors=r.IGNORE to ignore values that cannot be parsed
-          - errors=r.SET_NULL to set values that cannot be parsed to null
+      Parse a date and convert it to ISO date format: yyyy-mm-dd
+     
+      - dayfirst=True: treat xx as the day in xx/yy/zz
+      - yearfirst=True: treat xx as the year in xx/yy/zz
+      - errors=r.IGNORE to ignore values that cannot be parsed
+      - errors=r.SET_NULL to set values that cannot be parsed to null
 
       r.parsedatetime(value, dayfirst=False, yearfirst=False, errors=None)
 
-          Parse a datetime and convert it to ISO datetime format: yyyy-mm-ddTHH:MM:SS
-         
-          - dayfirst=True: treat xx as the day in xx/yy/zz
-          - yearfirst=True: treat xx as the year in xx/yy/zz
-          - errors=r.IGNORE to ignore values that cannot be parsed
-          - errors=r.SET_NULL to set values that cannot be parsed to null
+      Parse a datetime and convert it to ISO datetime format: yyyy-mm-ddTHH:MM:SS
+     
+      - dayfirst=True: treat xx as the day in xx/yy/zz
+      - yearfirst=True: treat xx as the year in xx/yy/zz
+      - errors=r.IGNORE to ignore values that cannot be parsed
+      - errors=r.SET_NULL to set values that cannot be parsed to null
 
       You can use these recipes like so:
 

[simonw]

Yeah there's a difference. In this screenshot the left is Python 3.10 and the right is Python 3.13-dev:

[simonw]

Weird, there is now a visible character in that README when it is displayed in the help:

https://github.com/simonw/sqlite-utils/blob/995446bc944b2322f3feb707dc7f44e9cbc192be/docs/cli-reference.rst#convert

[simonw]

It doesn't show if I just run sqlite-utils convert --help on the terminal:

But it DOES if I run that through sqlite-utils convert --help | pbcopy:

Usage: sqlite-utils convert [OPTIONS] DB_PATH TABLE COLUMNS... CODE

  Convert columns using Python code you supply. For example:

  sqlite-utils convert my.db mytable mycolumn \
      '"\n".join(textwrap.wrap(value, 10))' \
      --import=textwrap

  "value" is a variable with the column value to be converted.

  Use "-" for CODE to read Python code from standard input.

  The following common operations are available as recipe functions:

  r.jsonsplit(value, delimiter=',', type=<class 'str'>)

  Convert a string like a,b,c into a JSON array ["a", "b", "c"]

  r.parsedate(value, dayfirst=False, yearfirst=False, errors=None)

  Parse a date and convert it to ISO date format: yyyy-mm-dd
  �
  - dayfirst=True: treat xx as the day in xx/yy/zz
  - yearfirst=True: treat xx as the year in xx/yy/zz
  - errors=r.IGNORE to ignore values that cannot be parsed
  - errors=r.SET_NULL to set values that cannot be parsed to null

  r.parsedatetime(value, dayfirst=False, yearfirst=False, errors=None)

  Parse a datetime and convert it to ISO datetime format: yyyy-mm-ddTHH:MM:SS
  �
  - dayfirst=True: treat xx as the day in xx/yy/zz
  - yearfirst=True: treat xx as the year in xx/yy/zz
  - errors=r.IGNORE to ignore values that cannot be parsed
  - errors=r.SET_NULL to set values that cannot be parsed to null

  You can use these recipes like so:

  sqlite-utils convert my.db mytable mycolumn \
      'r.jsonsplit(value, delimiter=":")'

Options:
  --import TEXT                   Python modules to import
  --dry-run                       Show results of running this against first
                                  10 rows
  --multi                         Populate columns for keys in returned
                                  dictionary
  --where TEXT                    Optional where clause
  -p, --param <TEXT TEXT>...      Named :parameters for where clause
  --output TEXT                   Optional separate column to populate with
                                  the output
  --output-type [integer|float|blob|text]
                                  Column type to use for the output column
  --drop                          Drop original column afterwards
  --no-skip-false                 Don't skip falsey values
  -s, --silent                    Don't show a progress bar
  --pdb                           Open pdb debugger on first error
  -h, --help                      Show this message and exit.

[simonw]

That weird visible character is a \x08 which is the same thing as a \b:

'\n  \x08\n  - dayfirst=True: treat xx

>>> '\x08' == '\b'
True

[simonw]

https://github.com/simonw/sqlite-utils/blob/fd803d7043121f8618932c6c45888da60282ff5b/docs/cli-reference.rst#convert displays without visible glitch characters now.