Replace predictor-corrector by analytic integration and multigrid solve

[MaxThevenet]

This PR proposes to implement the analytic calculation for Bx and By as done in https://arxiv.org/abs/2012.00881 and https://github.com/tianhongg/WAND-PIC. The user now has the option to add hipace.explicit = 1 (default 0) to use this instead of the predictor-corrector loop to calculate Bx and By.

This explicit integration required the following main changes:

• deposition of jxx, jxy and jyy quantities
• Calculation of source term
• Solve the non-homogeneous Helmholtz equation with non-constant coefficients using AMReX' multigrid solver.
• Add a CI test

A more detailed description can be found in issue #356.

[MaxThevenet]

@SaVicente
Function Hipace::SolveBxBy is very hard to review, I found a few mistakes and I am sure there are more. Some things help a lot, for instance it is very useful to define MF n_star_gamma as you did. Could you make the names more systematic? In particular, you could:

• create multifabs n_star_ax and n_star_ay, similar to your n_start_gamma
• Re-organize the code so you:
  • compute the n*/(1+psi) factor
  • compute n_star_gamma
  • compute n_star_ay
  • compute sy
  • call the Bx field solver (currently, just a comment line)
  • do the same with n_star_ax sx and By
• have a consistent naming for temporary arrays. Actually, you probably only need 2 or 3 temporary arrays, so don't allocate too many (they are expensive and, if the names are not clear, that is not helping), you could give them dummy names and explain what you're doing in comments
• Fix the comments, they are currently wrong (the equation numbers are wrong, it contains some "TO DO" for things already implement, etc.)

Thanks!

[SaVicente]

@MaxThevenet

Yes, I can try to clean up everything. I tried to save as much space as possible, but I had to change some things in the code on the go and that caused some dirty parts. Can you tell me what part is wrong to also work on it?

Thank you!

[MaxThevenet]

Sure, my message above applied to all code in SolveBxBy. If you could have a first round and apply the points I mentioned, that would be a great step to make the code easier to read and review. In the meantime, we clarified how to call the solver in #373, so as soon as you are confident with the source terms we can actually call the solver.

Additionally, this commit 945a40a proposes to define a few variables to replace very verbose calls with much shorter ones, for things called over and over:

const int isl = WhichSlice::This;
const amrex::MultiFab& slicemf = m_fields.getSlices(lev, isl);
const amrex::BoxArray ba = slicemf.boxArray();
const amrex::DistributionMapping dm = slicemf.DistributionMap();
const amrex::IntVect ngv = slicemf.nGrowVect();

In the commit, I used these shorter versions in the first few lines, to show how they can help. Do you think you could apply them to all SolveBxBy? I believe this would greatly help readability. If you have other ideas along those lines, feel free to try and let me know.
Thanks!

[MaxThevenet]

Just updated the description and the title, this PR is ready for review. Once @SeverinDiederichs confirms that the results are correct in the presence of hosing we can merge it.

[SeverinDiederichs]

Great job! See few minor comments and suggestions below

[SeverinDiederichs]

For consistency, could you please also change this to using the explicit solver?

[SeverinDiederichs]

Suggested change

	rm -rf si_data
	rm -rf si_data_fixed_weight
	rm -rf normalized_data

[MaxThevenet]

Thanks @SeverinDiederichs I addressed your comments in the last commit.

[SeverinDiederichs]

Thanks for adding the suggestions.

Note: the explicit approach still has a bug, which shows at strong blowout cases, where it significantly differs from the predictor-corrector solver.