-
Notifications
You must be signed in to change notification settings - Fork 168
Closed
Description
Part of #2106
In v3 (and previously in Parcels) concatenated kernels are actually (under the hood) combined into a single function1. This leads to behaviour like the following:
# test_kernel_execution.py
@pytest.mark.parametrize("mode", ["scipy"])
def test_kernel_var_defining(mode):
fieldset = FieldSet.from_data(
{"U": [[0, 1], [2, 3]], "V": np.ones((2, 2))}, {"lon": [0, 2], "lat": [0, 2]}, mesh="flat"
)
def DefineA(particle, fieldset, time):
a = 2
def Update_dlon(particle, fieldset, time):
particle_dlon += a
kernels = [DefineA, Update_dlon]
"""
Final generated kernel is effectively
# Setcoords
particle_dlon = 0 # noqa
particle_dlat = 0 # noqa
particle_ddepth = 0 # noqa
particle.lon = particle.lon_nextloop
particle.lat = particle.lat_nextloop
particle.depth = particle.depth_nextloop
particle.time = particle.time_nextloop
# DefineA
a = 2
# Update_dlon
particle_dlon += a
# Updatecoords
particle.lon_nextloop = particle.lon + particle_dlon
particle.lat_nextloop = particle.lat + particle_dlat
particle.depth_nextloop = particle.depth + particle_ddepth
particle.time_nextloop = particle.time + particle.dt
"""
pset = ParticleSet(fieldset, pclass=ptype[mode], lon=0, lat=0)
pset.execute(kernels, endtime=1, dt=1)
# if we reverse the kernels, the variable a would not be defined
pset = ParticleSet(fieldset, pclass=ptype[mode], lon=0, lat=0)
pset.execute(kernels[::-1], endtime=1, dt=1) # Errors :/This bespoke behaviour deviates from a lot of assumptions people have about Python code, which negatively affects:
- Static analysers
- Ruff reports errors (which needs to be manually silenced via
noqacomments):- Local variable
particle_dlonis assigned to but never used (F841) - Undefined name
particle_dlon(F821)
- Local variable
- Ruff reports errors (which needs to be manually silenced via
- Users
- They need to get familiar with this behaviour
- They encounter less informative error messages (since the code being run is AST (i.e., not defined at a file location), there is no line number where the error occured
<ast>:2: UnboundLocalError)
- JIT code compilers
- numba doesn't know how to handle this
I propose that instead we do away with:
- The combining of kernels internally (no manipulating AST)
SetcoordsandUpdateCoordskernels (instead, kernels will be run as part of a larger evaluation loop which handles this setting and updating of coords - similar to what was done in the JIT C code)
These changes would address all the concerns above.
Breaking changes:
- Kernels will have to be updated:
particle_dlon,particle_dlon, andparticle_dlatwould becomeparticle.dlon,particle.dlon, andparticle.dlat- Setting a variable in one kernel to be accessed in another wouldn't work anymore.
- Sharing of state between kernels would need to done via the particle itself.
Notes:
- This would result in
dlonanddlatandddepthneeding to be added to the particle data.
Footnotes
-
This is evident from the
kernel.py:Kernel.merge()function where the AST is added together. These user provided kernels are then (in Scipy mode) pre-pended and appended withSetcoordsandUpdateCoordswithinKernel.add_scipy_positionupdate_kernels(). ↩
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
Status
Done