FsGrid & fieldsolver refactoring

[cscjlan]

FsGrid

Vlasiator has been updated to use the refactored FsGrid. This is visible everywhere FsGrid was used in. Some notable differences are dicussed below.

First, there's only a single FsGrid object and it's called fsgrid. It no longer stores any data. The data that used to be stored by the different grids are now owned by a structure called FsData, one for each ex-grid. The data is not passed around by passing the FsData structures, but rather by passing an std::span. You can get an std::span from FsData with the view() method: auto technicalSpan = technicalData.view();

updateGhostCells now takes in an std::span: fsgrid.updateGhostCells(technical); instead of technicalGrid.updateGhostCells().

Data is no longer accessed through the grid.get(i, j, k) method, but rather by contructing an FsStencil, and using it to compute a 1D index from a 3D index:

auto stencil = fsgrid.makeStencil(i, j, k);
// Get the center element ('ooo'), i.e. the one corresponding to i, j, k
const auto center = technical[stencil.ooo()];

// The element in the -x direction from the center, i.e. i - 1, j, k
const auto x_left = technical[stencil.moo()];

This has the benefit that some values that are constant throughout the simulation can be computed once and stored in the fsgrid object. A single FsStencil is constructed for each iteration of a loop (i.e. one for each i, j, k). The constant values can be accessed by the FsStencil when it computes the 1D value from the 3D index it was constructed with.

Fieldsolver

Fieldsolver has been refactored to use the updated FsGrid. Some files have undergone heavier refactoring (ldz_electric_field.cpp), some less heavy. The underlying reason for all of the refactoring is to make it easier to change the fieldsolver to use the updated FsGrid: with less code repetition there's less work to be done (and thus fewer bugs) when changing the code to use the stencil and the spans. An additional benefit was a speed up especially in the electric field computation.

derivatives.cpp and ldz_volume.cpp use the newly added fsgrid.parallel_for, which is an architecture/processor independent way of doing the desired computation. It takes in a lambda, which it calls for each local cell on the fsgrid.

[cscjlan]

Note that some of the diffs are completely unintelligible due to clang-format. To get a clearer diff, it's best to format some of the files and view the diffs locally.

Files:

• datareduction/datareducer.cpp
• fieldtracing/fieldtracing.cpp
• fieldtracing/fieldtracing.h
• grid.cpp
• ioread.cpp
• sysboundary/copysphere.cpp
• sysboundary/inflow.cpp
• sysboundary/ionosphere.cpp
• sysboundary/outflow.cpp
• tools/vlsvdiff.cpp

This script creates a complete diff locally:

#!/bin/bash

set -exu

git clone http://github.com/fmihpc/vlasiator.git
cd vlasiator/
git remote add csc https://github.com/cschpc/vlasiator.git
git fetch csc
git checkout -b clang-format-dev origin/dev

declare -a to_format=(
    "datareduction/datareducer.cpp"
    "fieldtracing/fieldtracing.cpp"
    "fieldtracing/fieldtracing.h"
    "grid.cpp"
    "ioread.cpp"
    "sysboundary/copysphere.cpp"
    "sysboundary/inflow.cpp"
    "sysboundary/ionosphere.cpp"
    "sysboundary/outflow.cpp"
    "tools/vlsvdiff.cpp"
)

for file in "${to_format[@]}"; do
    clang-format --style=file:.clang-format -i "${file}"
done

git add .
git commit -m "format some files"
git diff HEAD csc/fsgrid_merge_without_override > git_diff

Save in a file, execute it, then view the file vlasiator/git_diff.