Skip to content

Conversation

@VeckoTheGecko
Copy link
Contributor

@VeckoTheGecko VeckoTheGecko commented Sep 26, 2025

Summary

This PR restructures the Parcels package

Main changes:

  • Creates a namespace parcels.kernels for all kernels that are shipped with parcels
  • Creates a namespace parcels.interpolators for all interpolators that are shipped with parcels
  • Makes private some items that should have been public
  • Restructures the package to have core elements in "_core", while ancillary items outside
    • ancillary items can be deprecation helpers, Python version compatability utils, other general Python utils (isinstance_noimport to avoid circular imports)
  • Removes all wildcard imports "*"
  • Made a function parcels._python.isinstance_noimport to avoid a circular import - not sure what "best practice" is here, but I've seen code in production (in zarr iirc) so I don't think its too obscure

see commit log for full list of changes. @erikvansebille I think this is easiest to just review the end result, or one commit at a time (I don't think the diff is useful to look at). Let's review together f2f.

Fixes #1965

Note this doesn't fix. gh-2216 as it wasn't as straight forward as expected

Future work:

  • Restructure test suite as needed.
    • Some test files used to match the filenames, but now they don't. This is part of a bit of a different discussion on how we want to organise the test suite (topical or by file name). I think a mix is good, but worth discussing later
  • Remove v3 code that is no longer relevant
    • This PR was big enough as is, let's track that separately

Structure with comments

After

parcels
├── __init__.py
├── _compat.py
├── _core
│   ├── basegrid.py
│   ├── constants.py
│   ├── converters.py
│   ├── field.py
│   ├── fieldset.py
│   ├── index_search.py
│   ├── kernel.py
│   ├── particle.py
│   ├── particlefile.py
│   ├── particleset.py
│   ├── spatialhash.py
│   ├── statuscodes.py
│   ├── utils
│   │   ├── __init__.py
│   │   ├── array.py
│   │   ├── time.py
│   │   └── unstructured.py
│   ├── uxgrid.py
│   ├── warnings.py
│   └── xgrid.py
├── _datasets
│   ├── __init__.py
│   ├── structured
│   │   ├── __init__.py
│   │   ├── circulation_models.py
│   │   ├── generated.py
│   │   └── generic.py
│   ├── unstructured
│   │   ├── __init__.py
│   │   └── generic.py
│   └── utils.py
├── _decorators.py
├── _interpolation.py  <--- v3 code that should likely be removed/moved into core in some way
├── _logger.py
├── _python.py
├── _reprs.py
├── _tutorial.py
├── _typing.py
├── _v3to4.py
├── _version.py
├── interaction   <--- v3 code that we're just keeping here for now
│   ├── __init__.py
│   ├── interactionkernel.py
│   └── neighborsearch
│       ├── __init__.py
│       ├── base.py
│       ├── basehash.py
│       ├── bruteforce.py
│       ├── distanceutils.py
│       ├── hashflat.py
│       ├── hashspherical.py
│       └── kdtreeflat.py
├── interpolators.py
├── kernels
│   ├── __init__.py
│   ├── advection.py
│   ├── advectiondiffusion.py
│   ├── EOSseawaterproperties.py
│   ├── interaction.py
│   └── TEOSseawaterdensity.py
└── utils
    ├── __init__.py
    ├── _helpers.py   <--- v3 code that should likely be removed/moved into core in some way
    ├── interpolation_utils.py   <--- v3 code that should likely be removed
    └── timer.py   <--- v3 code that should likely be removed

Before

parcels
├── __init__.py
├── _compat.py
├── _core
│   └── utils
│       ├── __init__.py
│       ├── common.py
│       ├── structured.py
│       ├── time.py
│       └── unstructured.py
├── _datasets
│   ├── __init__.py
│   ├── structured
│   │   ├── __init__.py
│   │   ├── circulation_models.py
│   │   ├── generated.py
│   │   └── generic.py
│   ├── unstructured
│   │   ├── __init__.py
│   │   └── generic.py
│   └── utils.py
├── _index_search.py
├── _interpolation.py
├── _reprs.py
├── _typing.py
├── _version.py
├── application_kernels
│   ├── __init__.py
│   ├── advection.py
│   ├── advectiondiffusion.py
│   ├── EOSseawaterproperties.py
│   ├── interaction.py
│   ├── interpolation.py
│   └── TEOSseawaterdensity.py
├── basegrid.py
├── field.py
├── fieldset.py
├── interaction
│   ├── __init__.py
│   ├── interactionkernel.py
│   └── neighborsearch
│       ├── __init__.py
│       ├── base.py
│       ├── basehash.py
│       ├── bruteforce.py
│       ├── distanceutils.py
│       ├── hashflat.py
│       ├── hashspherical.py
│       └── kdtreeflat.py
├── kernel.py
├── particle.py
├── particlefile.py
├── particleset.py
├── spatialhash.py
├── tools
│   ├── __init__.py
│   ├── _helpers.py
│   ├── _v3to4.py
│   ├── converters.py
│   ├── exampledata_utils.py
│   ├── interpolation_utils.py
│   ├── loggers.py
│   ├── statuscodes.py
│   ├── timer.py
│   └── warnings.py
├── uxgrid.py
└── xgrid.py

Style guidelines

  • When writing package code, we import directly from the file (i.e., prefer from parcels._core.field import Field over from parcels import Field). This is to make clear the import path
  • When writing testing code, prefer to import from the parcels namespace for public API (of course, import from
  • The parcels and parcels/kernels are the only __init__.py files we need to currate. We don't need to specify more __init__.py files

I think that these guidelines provide a consistent way of working with things that communicates package structure to developers (while also helping refactoring tools), while also communicating (i.e., via the tests) how we expect user to use the code and namespaces. Also somewhat mimicking what I've seen in the xarray codebase.

Should we document these guidelines somewhere?

@VeckoTheGecko VeckoTheGecko merged commit 5ccc0bc into Parcels-code:v4-dev Sep 29, 2025
10 checks passed
@github-project-automation github-project-automation bot moved this from Backlog to Done in Parcels development Sep 29, 2025
@VeckoTheGecko VeckoTheGecko deleted the package-restructure branch September 29, 2025 07:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Package structure

2 participants