CI migration and refresh #41

bjlittle · 2021-05-18T15:53:21Z

This PR shows some much needed love to python-stratify.

Namely, it performs the following:

migrates the code py2 to py3
migrates the CI from travis-ci to cirrus-ci
- supports linux and ~~osx~~
rationalises the setup.py into the setup.cfg
adopts isort
adopts pre-commit git hooks to primarily perform black, flake8, isort
uses setuptools-scm for version management

bjlittle · 2021-05-18T15:54:08Z

closes #40

jamesp

Looks good!

I'm a little confused by the versioning system. Where is the version number actually stored/recalled?

jamesp · 2021-05-18T16:00:05Z

.cirrus.yml

+    populate_script:
+      - conda-lock --mamba --platform linux-64 --file ${CIRRUS_WORKING_DIR}/requirements/py$(echo ${PY_VER} | tr -d ".").yml
+      - mamba create --name py${PY_VER} --quiet --file conda-linux-64.lock
+      - cp conda-linux-64.lock ${HOME}/mambaforge/envs/py${PY_VER}


Nice, so you're caching the lock info in the CI 👍

jamesp · 2021-05-18T16:03:53Z

.gitignore

@@ -1,3 +1,4 @@
+_version.py


What is this file and why explicitly ignored?

This is used by scm and you do not place it under source code control.

It's like a scratch file that scm uses, as per your configuration, to dump its versioning metadata, see pyproject.toml:line+14.

This is then parsed by stratify.init:line+2

jamesp · 2021-05-18T16:14:36Z

stratify/_bounded_vinterp.py

    fz_src_orig = list(fz_src_reshaped.shape)
    shape = (
-        int(np.product(fz_src_reshaped.shape[:len(bdims)])),
+        int(np.product(fz_src_reshaped.shape[: len(bdims)])),


Why space here? seems less readable to me

This is blacks doing, not me guv

This is gross. https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html?#slices

jamesp · 2021-05-18T16:15:36Z

stratify/_bounded_vinterp.py

+        int(np.product(fz_src_reshaped.shape[: len(bdims)])),
        fz_src_reshaped.shape[len(bdims)],
-        int(np.product(fz_src_reshaped.shape[len(bdims)+1:])))
+        int(np.product(fz_src_reshaped.shape[len(bdims) + 1 :])),


I would prefer [(len(bdims) + 1):], adding spaces around slice op is not idiomatic

All these changes to the actual code are a result of black auto-fornatting. I trying not to touch the code, other than py2 migration

But yeah... what can I say 😱

bjlittle · 2021-05-18T16:49:02Z

The magic of versioning is handled by setuptools-scm. Perhaps scan through https://github.com/pypa/setuptools_scm/#pyprojecttoml-usage, and that should answer your questions

bjlittle · 2021-05-18T16:50:42Z

I'm going to push setuptools-scm into iris next btw

bjlittle · 2021-05-18T16:53:17Z

pyproject.toml

+build-backend = "setuptools.build_meta"
+
+[tool.setuptools_scm]
+write_to = "stratify/_version.py"


Tell scm where to dump its metadata - this nominated file must not be kept under source code control

bjlittle · 2021-05-18T16:55:11Z

stratify/__init__.py

-
-
-__version__ = '0.2.dev0'
+from ._version import version as __version__  # noqa: F401


Configure scm

Now we can do python setup.py --version and also get stratify.__version__ as usual

ok, so setuptools_scm is taking the version number from the latest tag and then applying logic to calculate a new one? Sorry, this is all new to me, I'm sure it makes sense once you've done it a couple of times but seems like magic! If I was to download the package without the .git repo, would it still be able to build/install from source?

Yup, you've essentially got it. This following section in the pyproject.toml file is like a nominated scratch area for setuptools-scm:

[tool.setuptools_scm] write_to = "stratify/_version.py"

It can be anything that you want. Also, you can control how scm generates the offset from the latest tag to create your version string, but what you get out the box is good enough for me.

Note that, there is also this plugin setuptools-scm-git-archive which might address your question, but I think the use case having the source code outside git is a bit rare - particularly when it comes to packaging IMHO.

@jamesp I actually stumbled across the actual answer to your question regarding the necessity to have access to the .git repo... as I had to jump through that hoop when attempting to build the conda-forge package for the 0.2.post0 release.

You do need access to a .git repo and the associated artifacts for setuptools-scm to do its job of fathoming out what the version of the package. However, if that's not possible, for example this is exactly the case when attempting to conda-build your package from a .tar.gz archive from GitHub, then you need to instead hit the associated PyPI URL for your package as this contain the necessary metadata for setuptools-scm to do it's job i.e., see https://github.com/conda-forge/python-stratify-feedstock/blob/master/recipe/meta.yaml#L9.

Bingo. Just works 👍

Yep, this is the use-case I was thinking about. Good to see there is a reasonable workaround, although having conda-forge hanging off of pypi does seem a little Rube Goldberg!

codecov · 2021-05-18T21:29:35Z

Codecov Report

❗ No coverage uploaded for pull request base (master@a6749e6). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #41   +/-   ##
=========================================
  Coverage          ?   98.33%           
=========================================
  Files             ?        5           
  Lines             ?      480           
  Branches          ?        0           
=========================================
  Hits              ?      472           
  Misses            ?        8           
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a6749e6...e09d203. Read the comment docs.

bjlittle · 2021-05-19T06:28:37Z

stratify/_vinterp.pyx

-    cpdef public z_target, orig_shape, axis, _zp_reshaped, _fp_reshaped
-    cpdef public _result_working_shape, result_shape
+    cdef public z_target, orig_shape, axis, _zp_reshaped, _fp_reshaped
+    cdef public _result_working_shape, result_shape


Changed these from cpdef to cdef to avoid cython future warnings, but functionally a nop change.

jamesp · 2021-05-19T06:57:14Z

All approved, but I don't have write access to this repo! @bjlittle feel free to merge

bjlittle · 2021-05-19T07:47:46Z

@jamesp Let me fix that for you...

bjlittle · 2021-05-19T09:00:31Z

@jamesp You should now have write access to the repo 👍

bjlittle · 2021-05-19T09:48:47Z

@jamesp Awesome, thanks 🎉

CI migration and refresh

4bf2c24

jamesp reviewed May 18, 2021

View reviewed changes

bjlittle commented May 18, 2021

View reviewed changes

update

18a86c8

drop osx, add ci linting

e09d203

bjlittle force-pushed the update branch from 131f7dd to e09d203 Compare May 18, 2021 22:37

bjlittle commented May 19, 2021

View reviewed changes

jamesp merged commit d7c939d into SciTools:master May 19, 2021

This was referenced May 19, 2021

Remove cython from install_requires #40

Closed

reinstate .coveragerc #43

Merged

bjlittle deleted the update branch April 25, 2023 20:23



		__version__ = '0.2.dev0'
		from ._version import version as __version__ # noqa: F401

CI migration and refresh #41

CI migration and refresh #41

Uh oh!

Conversation

bjlittle commented May 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bjlittle commented May 18, 2021

Uh oh!

jamesp left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bjlittle May 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bjlittle May 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bjlittle commented May 18, 2021

Uh oh!

bjlittle commented May 18, 2021

Uh oh!

bjlittle May 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bjlittle May 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented May 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jamesp commented May 19, 2021

Uh oh!

bjlittle commented May 19, 2021

Uh oh!

bjlittle commented May 19, 2021

Uh oh!

bjlittle commented May 19, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

bjlittle commented May 18, 2021 •

edited

Loading

bjlittle May 18, 2021 •

edited

Loading

bjlittle May 19, 2021 •

edited

Loading

bjlittle May 18, 2021 •

edited

Loading

bjlittle May 18, 2021 •

edited

Loading

codecov bot commented May 18, 2021 •

edited

Loading