Skip to content

Performance hit due to array conversion from C to Fortran layout #520

@c84925

Description

@c84925

It looks like most, or maybe all, of the spherical harmonic coefficient storage objects use Numpy's default C-contiguous memory layout. This seems to be causing a significant performance hit when the python interface calls through to Fortran code. I'm not sure why; Numpy's documentation would seem to suggest the array should be passed directly through in most cases (https://numpy.org/doc/stable/f2py/python-usage.html#array-arguments) [edit: I was misreading the Numpy docs, looks like they're just saying C-to-F layout conversion is automatically done with a copy, causing the overhead which is observed]. Users can mitigate the issue by manually converting coefficients to Fortran layout, as I show below. Maybe something about how the functions are being wrapped could be causing an unnecessary copy, or might be worth considering making all coefficient arrays use Fortran layout when they are created.

Sample script, using the default EGM2008 and MakeGravGridPoint evaluated at a single query point:

import pyshtools as pysh
import numpy as np
import timeit
egm2008 = pysh.datasets.Earth.EGM2008()

def do_profile(lmax):
    pysh.gravmag.MakeGravGridPoint(egm2008.coeffs, egm2008.gm, egm2008.r0, lmax=lmax, r=8e6, lat=45, lon=45) # take care of any one-time costs.
    t = timeit.timeit(
        'pysh.gravmag.MakeGravGridPoint(egm2008.coeffs, egm2008.gm, egm2008.r0, lmax=%i, r=8e6, lat=45, lon=45)' % lmax,
        number=10, globals=globals())
    print('lmax=',lmax,'\ttime=',t)

def do_profile_set():
    print('egm2008.coeffs.flags.c_contiguous is ', egm2008.coeffs.flags.c_contiguous)
    print('egm2008.coeffs.flags.f_contiguous is ', egm2008.coeffs.flags.f_contiguous)

    do_profile(10)
    do_profile(100)
    do_profile(1000)

print('Starting with egm2008 in default C-contiguous format, ie')
do_profile_set()

egm2008.coeffs = np.asfortranarray(egm2008.coeffs)

print()
print('Now with egm2008 in F-contiguous format, ie')
do_profile_set()

Output from this script:

Starting with egm2008 in default C-contiguous format, ie
egm2008.coeffs.flags.c_contiguous is  True
egm2008.coeffs.flags.f_contiguous is  False
lmax= 10        time= 0.3619217239320278
lmax= 100       time= 0.3648493140935898
lmax= 1000      time= 0.4514749152585864

Now with egm2008 in F-contiguous format, ie
egm2008.coeffs.flags.c_contiguous is  False
egm2008.coeffs.flags.f_contiguous is  True
lmax= 10        time= 2.2382475435733795e-05
lmax= 100       time= 0.00029553286731243134
lmax= 1000      time= 0.03709914069622755

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions