Skip to content

Remove under the hood combining of kernels into single function #2107

@VeckoTheGecko

Description

@VeckoTheGecko

Part of #2106

In v3 (and previously in Parcels) concatenated kernels are actually (under the hood) combined into a single function1. This leads to behaviour like the following:

# test_kernel_execution.py
@pytest.mark.parametrize("mode", ["scipy"])
def test_kernel_var_defining(mode):
    fieldset = FieldSet.from_data(
        {"U": [[0, 1], [2, 3]], "V": np.ones((2, 2))}, {"lon": [0, 2], "lat": [0, 2]}, mesh="flat"
    )

    def DefineA(particle, fieldset, time):
        a = 2

    def Update_dlon(particle, fieldset, time):
        particle_dlon += a

    kernels = [DefineA, Update_dlon]

    """
    Final generated kernel is effectively
        # Setcoords
        particle_dlon = 0  # noqa
        particle_dlat = 0  # noqa
        particle_ddepth = 0  # noqa
        particle.lon = particle.lon_nextloop
        particle.lat = particle.lat_nextloop
        particle.depth = particle.depth_nextloop
        particle.time = particle.time_nextloop

        # DefineA
        a = 2

        # Update_dlon
        particle_dlon += a

        # Updatecoords
        particle.lon_nextloop = particle.lon + particle_dlon
        particle.lat_nextloop = particle.lat + particle_dlat
        particle.depth_nextloop = particle.depth + particle_ddepth
        particle.time_nextloop = particle.time + particle.dt
    """

    pset = ParticleSet(fieldset, pclass=ptype[mode], lon=0, lat=0)
    pset.execute(kernels, endtime=1, dt=1)

    # if we reverse the kernels, the variable a would not be defined
    pset = ParticleSet(fieldset, pclass=ptype[mode], lon=0, lat=0)
    pset.execute(kernels[::-1], endtime=1, dt=1)  # Errors :/

This bespoke behaviour deviates from a lot of assumptions people have about Python code, which negatively affects:

  • Static analysers
    • Ruff reports errors (which needs to be manually silenced via noqa comments):
      • Local variable particle_dlon is assigned to but never used (F841)
      • Undefined name particle_dlon (F821)
  • Users
    • They need to get familiar with this behaviour
    • They encounter less informative error messages (since the code being run is AST (i.e., not defined at a file location), there is no line number where the error occured <ast>:2: UnboundLocalError)
  • JIT code compilers
    • numba doesn't know how to handle this

I propose that instead we do away with:

  • The combining of kernels internally (no manipulating AST)
  • Setcoords and UpdateCoords kernels (instead, kernels will be run as part of a larger evaluation loop which handles this setting and updating of coords - similar to what was done in the JIT C code)

These changes would address all the concerns above.

Breaking changes:

  • Kernels will have to be updated:
    • particle_dlon, particle_dlon, and particle_dlat would become particle.dlon, particle.dlon, and particle.dlat
    • Setting a variable in one kernel to be accessed in another wouldn't work anymore.
  • Sharing of state between kernels would need to done via the particle itself.

Notes:

  • This would result in dlon and dlat and ddepth needing to be added to the particle data.

Footnotes

  1. This is evident from the kernel.py:Kernel.merge() function where the AST is added together. These user provided kernels are then (in Scipy mode) pre-pended and appended with Setcoords and UpdateCoords within Kernel.add_scipy_positionupdate_kernels().

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions