Fix bug in momentum-conserving interpolation function with MR (ECP-Wa…

…rpX#1624)

* Fix bug in momentum-conserving interpolation function

* Reset checksum benchmark for CI test momentum-conserving-gather

Unused Params: Check after Step 0 (ECP-WarpX#1596)

An early check for unused parameters after the first step.

Add Superparticle version of getParticlePosition. (ECP-WarpX#1640)

* Add Superparticle version of GetParticlePosition.
* move unpack_particle to a free function and rename
* no longer need SuperPType
* Update Source/Particles/Pusher/GetAndSetPosition.H
* remove templating from get_particle_position
* remove template
* Add missing include

Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>

Distribution mapping and cost plotting (ECP-WarpX#1444)

* Distribution mapping and cost plotting

Cleanup

Cleanup

Cleanup

EOL

Unused import

* Cost initializes to 0.0

* plot slices of 3D

* WIP

* WIP docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* EOL

SpeciesProperties: C, N, O, Cu (ECP-WarpX#1638)

* SpeciesProperties: C, N, O, Cu

Add more ionic species as pre-defined particle species.

* Cu: Add to Ionization Energies

Fix 2D: ParticleHistogram (ECP-WarpX#1635)

Fix out-of-bounds access in particle histogram in non-3D.

Fix FilterFunctor in non-3D (ECP-WarpX#1633)

Fix an out-of-bounds access to positions in 2D & RZ for
filter functors.

openPMD: ionizationLevel (ECP-WarpX#1622)

* openPMD: ionizationLevel

Write out the ionizationLevel with openPMD.

* openPMD Record Repetitions: Use Lambdas

openPMD: Particle Filter (Container) (ECP-WarpX#1632)

* openPMD: Particle Filter (Container)

A fresh implementation of particle filters for openPMD diagnostics
using the generalized `PhysicalParticleContainer` approach that we
also use in plotfiles 🎉

* Ion Example: Filter & Coarsen

Test diagnostics filter & coarsen functions in CI.
Test reduced diagnostics (histograms).

Define: _OPENMP -> AMREX_USE_OMP (ECP-WarpX#1520)

* Define: _OPENMP -> AMREX_USE_OMP

Replace the define check of `_OPENMP` with the explicit
backend control of `AMREX_USE_OMP` for parallel constructs.

Doing so avoids that we accidentially turn on OpenMP, e.g. if a dependency
pulls it in for linear algebra, I/O, etc. This can led to confusion if the
user explicitly requested a serial build. Also, we might want to use OpenMP
functionality here and there for auxiliary functions w/o having to use the
AMReX OpenMP backend, i.e. because we compile for GPUs.

* Add missing amrex::Gpu::notInLaunchRegion

Start: PerformanceHints After Init (ECP-WarpX#1630)

Start a helper routine that gives performance hints after
initialization of the simulation.

Fix number of guard cells in PML with 2D PSATD (ECP-WarpX#1648)

Remove unused lines about particle filters (ECP-WarpX#1646)

Add parameter for default galilean velocity (ECP-WarpX#1097)

* Add parameter for default galilean velocity

* Apply suggestions from code review

Co-authored-by: Edoardo Zoni <59625522+EZoni@users.noreply.github.com>

* Fix bug in constructor of PhysicalParticleContainer

* Use new input parameter in CI test

Co-authored-by: Edoardo Zoni <59625522+EZoni@users.noreply.github.com>
Co-authored-by: Edoardo Zoni <ezoni@lbl.gov>

Cleaned up field diagnostic in picmi interface (ECP-WarpX#1649)

Use pinned memory for tmp particles in diags. (ECP-WarpX#1644)

* Use pinned memory for tmp particles in diags.

* openPMD: pinned memory (tmp particles)

Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>

Replace an always false test with an abort (ECP-WarpX#1655)

* replace always false test with abort

* fix bug

[mini-PR] Broken link to scripts (ECP-WarpX#1651)

CI: Ubuntu-latest -> 20.04 (ECP-WarpX#1654)

Start Embedded Boundary development (ECP-WarpX#1641)

* Start Embedded Boundary development

Modify the build system for embedded boundary (EB) support.  Currently EB is
a compile time option that must be explicitly enabled.  For GNU Make, one
can enable it with `USE_EB=TRUE`, whereas for CMake, `-DAMReX_EB=ON`.  Later
we could decide to enable EB by default with all regular geometry.

Add a simple geometry initialization function, WarpX::InitEB.  By default,
the geometry is all regular.  A few basic types such as box, cylinder,
plane, sphere, etc. are supported via ParmParse runtime parameters.  See
`amrex/Src/EB/AMReX_EB2.cpp` for more details.  Later, we could build more
complex geometry using constructive solid geometry (CSG).  (
https://en.wikipedia.org/wiki/Constructive_solid_geometry ) See
`amrex/Tutorials/EB/GeometryGeneration` for an example of CSG.  There is
also a STL ( https://en.wikipedia.org/wiki/STL_(file_format) ) approach
under development in AMReX.

Add a new member, m_factory, to WarpX class.  This object can be used to
obtain geometry information such as whether a cell is cut, edge centroids,
etc.  Currently we are not using these factories to build MultiFabs for
field data to embed the geometry information into the data containers.  We
could do that later if it is needed or it makes things more convenient.
Nevertheless, this should be sufficient to start the EB development of the
field solver.

It's not clear to me yet how many ghost cells are needed for the geometry
information.  It's currently one, and can be adjusted.

In the future, when particle and embedded boundary interaction is
considered, we can use `amrex::FillSignedDistance` function to obtain signed
distance function on the nodes.  With that information, one should be able
to determine where and when a particle collides with the embedded boundary.

* resize factory vector

* CMake: Require AMReX_EB

for embedded boundaries

* Add AMReX_Config.H et al.

Explicit includes are most robust to make sure defines are set with
future refactorings.

* rename Factory fieldFactory

Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>

Add filter for particle histogram (ECP-WarpX#1643)

* Add filter for particle histogram

* Update benchmarks

* Update benchmark again

* Use serialize_ics in test + proper usage of RandomEngine

* Laser-Ion Acc. (2D3V): Test Hist Filter

Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>

Take time step into account to compute guard cells for J and rho (ECP-WarpX#1607)

* Use IntVect for ng_J and ng_rho

* Compute guard cells for J and rho based on dt

* Reset some CI benchmarks

* Fix rebase commit

* Add back +1 cell for rho: fix remaining out-of-bound accesses

* Simplify ASSERTS using new interface of amrex::numParticlesOutOfRange

Modern setup.py: CMake-Driven, Multi-Dimensional (ECP-WarpX#1647)

This adds a new, project-centric setup.py file.
With this file, all dimensions (2D, 3D, RZ) of WarpX can be built
and packaged at once, using the CMake build logic.

Build & install:
```bash
pip wheel -v .
pip install *whl
```

Updates for electrostatic solver (ECP-WarpX#1604)

* Fix electrostatic solver with momentum conservation

* Fix electrostatic, adding call to FillBoundaryAux

* For electrostatic, removed unneeded extra calls to UpdateAux and FillBoundary

* For electrostatic, calculate fields at the end of the time step

* Updated ElectrostaticSphere analysis script to use fields from end of time step

Reset broken benchmark for CI test initial_distribution (ECP-WarpX#1661)

make sure we redefine the tmp particle tiles when we load balance. (ECP-WarpX#1658)

[mini-PR] Add particle tiling documentation (ECP-WarpX#1665)

* tiling wip

* do_tiling documentation

* Update Docs/source/running_cpp/parameters.rst

Co-authored-by: Luca Fedeli <luca.fedeli@for.unipi.it>

Co-authored-by: Luca Fedeli <luca.fedeli@for.unipi.it>

fix inconsistent formatting (ECP-WarpX#1666)

parameter.rst fixes (ECP-WarpX#1663)

requirements.txt: fix overspecification (ECP-WarpX#1668)

I accidentally added the patch-level for version-compatible matching

This removes the patch-level for `~=` matching for the packages that
have a >=1 major version already.

scipy 1.5+ (ECP-WarpX#1670)

Sufficient since it was last release in december and works well.
The 3.5 release series still builds wheels for Python 3.6 for Ubuntu
oldstable (18.04).

Remove redundant "do_qed" option in inputfile (ECP-WarpX#1667)

* removed redundant do_qed option in inputfile

* fixed bug

remove semicolons (ECP-WarpX#1662)

fixTypo for pml has particles parameter for docs (ECP-WarpX#1671)

load balance efficiency reduced diagnostic

eol

minor

eol

.github/workflows/source.yml

@@ -11,7 +11,7 @@ on: [push, pull_request]
 jobs:
   style:
 
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-20.04
 
     steps:
     - uses: actions/checkout@v2

.github/workflows/source/hasNonASCII

@@ -28,6 +28,7 @@ for i in $(find . \
                 -not -path "*wp_parse*"        \
                 -not -path "./tmp_build_dir/*" \
                 -not -path "./Docs/*"          \
+                -not -path "./setup.py"        \
                 -type f | \
            grep -E "${pattern}")
 do

.github/workflows/ubuntu.yml

@@ -41,7 +41,7 @@ jobs:
 # intel-basekit intel-hpckit are too large in size
   build_icc:
     name: oneAPI ICC SP&DP [Linux]
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-20.04
     steps:
     - uses: actions/checkout@v2
     - name: install dependencies

CMakeLists.txt

@@ -44,6 +44,7 @@ set_default_install_dirs()
 #
 option(WarpX_APP           "Build the WarpX executable application"     ON)
 option(WarpX_ASCENT        "Ascent in situ diagnostics"                 OFF)
+option(WarpX_EB            "Embedded boundary support"                  OFF)
 option(WarpX_LIB           "Build WarpX as a shared library"            OFF)
 option(WarpX_MPI           "Multi-node support (message-passing)"       ON)
 option(WarpX_OPENPMD       "openPMD I/O (HDF5, ADIOS)"                  OFF)
@@ -181,6 +182,7 @@ endif()
 
 add_subdirectory(Source/BoundaryConditions)
 add_subdirectory(Source/Diagnostics)
+add_subdirectory(Source/EmbeddedBoundary)
 add_subdirectory(Source/Evolve)
 add_subdirectory(Source/FieldSolver)
 add_subdirectory(Source/Filter)

Docs/source/building/cmake.rst

@@ -8,13 +8,15 @@ Until we have transitioned our documentation and functionality completely, pleas
 
 Progress status: `see on GitHub <https://github.com/ECP-WarpX/WarpX/projects/10>`_
 
+
 Introduction to CMake
 =====================
 
 If you are new to CMake, `this short tutorial <https://hsf-training.github.io/hsf-training-cmake-webpage/>`_ from the HEP Software foundation is the perfect place to get started with it.
 
 If you just want to use CMake to build the project, jump into sections *1. Introduction*, *2. Building with CMake* and *9. Finding Packages*.
 
+
 Dependencies
 ============
 
@@ -37,6 +39,7 @@ Optional dependencies include:
   - see `optional I/O backends <https://github.com/openPMD/openPMD-api#dependencies>`_
 - `CCache <https://ccache.dev>`_: to speed up rebuilds (needs 3.7.9+ for CUDA)
 
+
 Install Dependencies
 ====================
 
@@ -75,6 +78,7 @@ or macOS/Linux:
 
 Now, ``cmake --version`` should be at version 3.14.0 or newer.
 
+
 Configure your compiler
 =======================
 
@@ -92,6 +96,7 @@ If you also want to select a CUDA compiler:
    export CUDACXX=$(which nvcc)
    export CUDAHOSTCXX=$(which g++)
 
+
 Build & Test
 ============
 
@@ -123,6 +128,7 @@ CMake Option                  Default & Values                             Descr
 ``WarpX_ASCENT``              ON/**OFF**                                   Ascent in situ visualization
 ``WarpX_COMPUTE``             NOACC/**OMP**/CUDA/SYCL/HIP                  On-node, accelerated computing backend
 ``WarpX_DIMS``                **3**/2/RZ                                   Simulation dimensionality
+``WarpX_EB``                  ON/**OFF**                                   Embedded boundary support
 ``WarpX_LIB``                 ON/**OFF**                                   Build WarpX as a shared library
 ``WarpX_MPI``                 **ON**/OFF                                   Multi-node support (message-passing)
 ``WarpX_MPI_THREAD_MULTIPLE`` **ON**/OFF                                   MPI thread-multiple support, i.e. for ``async_io``
@@ -142,16 +148,62 @@ Assuming AMReX' source is located in ``$HOME/src/amrex`` and changes are committ
 
 For developers, WarpX can be configured in further detail with options from AMReX, which are `documented in the AMReX manual <https://amrex-codes.github.io/amrex/docs_html/BuildingAMReX.html#customization-options>`_.
 
+
 Run
 ===
 
 An executable WarpX binary with the current compile-time options encoded in its file name will be created in ``build/bin/``.
 
 Additionally, a `symbolic link <https://en.wikipedia.org/wiki/Symbolic_link>`_ named ``warpx`` can be found in that directory, which points to the last built WarpX executable.
 
-Python Wrapper
-==============
 
+Python Wrapper (New Scripts)
+============================
+
+Build and install a wheel from the root of the WarpX source tree:
+
+.. code-block:: bash
+
+   python3 -m pip wheel -v .
+   python3 -m pip install *whl
+
+Maintainers might also want to generate a self-contained source package that can be distributed to exotic architectures:
+
+.. code-block:: bash
+
+   python setup.py sdist --dist-dir .
+   pip wheel -v pywarpx-*.tar.gz
+   python3 -m pip install *whl
+
+The above steps can also be executed in one go to build from source on a machine:
+
+.. code-block:: bash
+
+   python setup.py sdist --dist-dir .
+   pip install -v pywarpx-*.tar.gz
+
+Environment variables can be used to control the build step:
+
+============================= ============================================ =======================================================
+Environment Variable          Default & Values                             Description
+============================= ============================================ =======================================================
+``WarpX_COMPUTE``             NOACC/**OMP**/CUDA/SYCL/HIP                  On-node, accelerated computing backend
+``WarpX_DIMS``                ``"2;3;RZ"``                                 Simulation dimensionalities (semicolon-separated list)
+``WarpX_MPI``                 ON/**OFF**                                   Multi-node support (message-passing)
+``WarpX_OPENPMD``             ON/**OFF**                                   openPMD I/O (HDF5, ADIOS)
+``WarpX_PSATD``               ON/**OFF**                                   Spectral solver
+``WarpX_QED``                 ON/**OFF**                                   PICSAR QED (requires Boost and PICSAR)
+``BUILD_PARALLEL``            ``2``                                        Number of threads to use for parallel builds
+``BUILD_SHARED_LIBS``         ON/**OFF**                                   Build shared libraries for dependencies
+``HDF5_USE_STATIC_LIBRARIES`` ON/**OFF**                                   Prefer static libraries for HDF5 dependency (openPMD)
+``ADIOS_USE_STATIC_LIBS``     ON/**OFF**                                   Prefer static libraries for ADIOS1 dependency (openPMD)
+============================= ============================================ =======================================================
+
+
+Python Wrapper (Legacy Scripts)
+===============================
+
+This is a manual workflow using the GNUmake-like Python install logic.
 The Python wrapper library can be built by pre-building WarpX into one or more shared libraries.
 For full functionality in 2D, 3D and RZ geometry, the following workflow can be executed:
 

Docs/source/building/quartz.rst

@@ -89,6 +89,6 @@ The other :ref:`general compile-time options <building-cmake>` apply as usual.
 Running
 -------
 
-Please see :ref:`our example job scripts <running-cpp-quartz>` on how to run WarpX on Quartz.
+Please see :ref:`our example job scripts <running-cpp-quartz-CPUs>` on how to run WarpX on Quartz.
 
 See :doc:`../visualization/yt` for more information on how to visualize the simulation results.

Docs/source/running_cpp/parameters.rst

@@ -119,7 +119,7 @@ Setting up the field mesh
 
     Note: in development; currently, ``2`` is supported.
 
-* ``amr.ref_ratio_vect`` (3 `integer`s for x,y,z per refined level)
+* ``amr.ref_ratio_vect`` (`3 integers for x,y,z per refined level`)
     When using mesh refinement, this can be used to set the refinement ratio per direction and level, relative to the previous level.
 
     Example: for three levels, a value of ``2 2 4 8 8 16`` refines the first level by 2-fold in x and y and 4-fold in z compared to the coarsest level (level 0/mother grid); compared to the first level, the second level is refined 8-fold in x and y and 16-fold in z.
@@ -285,7 +285,7 @@ WarpX provides a math parser that reads expressions in the input file.
 It can be used in all input parameters that consist in one real number.
 
 WarpX constants
-###############
+^^^^^^^^^^^^^^^
 
 WarpX provides a few pre-defined constants, that can be used for any parameter that consists in one real number.
 
@@ -301,7 +301,7 @@ pi       math constant pi
 See ``Source/Utils/WarpXConst.H`` for the values.
 
 User-defined constants
-######################
+^^^^^^^^^^^^^^^^^^^^^^
 
 Users can define their own constants in the input file.
 These constants can be used for any parameter that consists in one real number.
@@ -314,7 +314,7 @@ For example, parameters ``a0`` and ``z_plateau`` can be specified with:
 * ``my_constants.z_plateau = 150.e-6``
 
 Coordinates
-###########
+^^^^^^^^^^^
 
 Besides, for profiles that depend on spatial coordinates (the plasma momentum distribution or the laser field, see below `Particle initialization` and `Laser initialization`), the parser will interpret some variables as spatial coordinates. These are specified in the input parameter, i.e., ``density_function(x,y,z)`` and ``field_function(X,Y,t)``.
 
@@ -323,7 +323,7 @@ The parser reads python-style expressions between double quotes, for instance
 user-defined constant (see below) and ``x`` and ``y`` are spatial coordinates. The names are case sensitive. The factor
 ``(x>0)`` is ``1`` where ``x>0`` and ``0`` where ``x<=0``. It allows the user to
 define functions by intervals.
-The parser reads mathematical functions into an `abstract syntax tree (AST) <https://en.wikipedia.org/wiki/Abstract_syntax_tree>`_, which supports a maximum depth (see :ref:`build options <building-cmake>`_).
+The parser reads mathematical functions into an `abstract syntax tree (AST) <https://en.wikipedia.org/wiki/Abstract_syntax_tree>`_, which supports a maximum depth (see :ref:`build options <building-cmake>`).
 Additional terms in a function can create a level of depth in the AST, e.g. ``a+b+c+d`` is parsed in groups of ``[+ a [+ b [+ c [+ d]]]]`` (depth: 4).
 A trick to reduce this depth for the parser, e.g. when reaching the limit, is to group expliclity, e.g. via ``(a+b)+(c+d)``, which is parsed in groups of ``[+ [+ a b] [+ c d]]`` (depth: 2).
 
@@ -355,6 +355,10 @@ Particle initialization
     particles are pushed in a standard way, using the specified pusher.
     (see the parameter ``<species_name>.zinject_plane`` below)
 
+* ``particles.do_tiling`` (`bool`) optional (default `true` if WarpX is compiled for GPUs, `false` otherwise)
+    Controls whether tiling ('cache blocking') transformation is used for particles.
+    Tiling should be on when using OpenMP and off when using GPUs.
+
 * ``<species_name>.species_type`` (`string`) optional (default `unspecified`)
     Type of physical species, ``"electron"``, ``"positron"``, ``"photon"``, ``"hydrogen"``.
     Either this or both ``mass`` and ``charge`` have to be specified.
@@ -623,20 +627,15 @@ Particle initialization
     must be either electrons or positrons. Boris pusher must be used for the
     simulation
 
-* ``<species>.do_qed`` (`int`) optional (default `0`)
-    If `<species>.do_qed = 0` all the QED effects are disabled for this species.
-    If `<species>.do_qed = 1` QED effects can be enabled for this species (see below).
-    **This feature requires to compile with QED=TRUE**
-
 * ``<species>.do_qed_quantum_sync`` (`int`) optional (default `0`)
-    It only works if `<species>.do_qed = 1`. Enables Quantum synchrotron emission for this species.
+    Enables Quantum synchrotron emission for this species.
     Quantum synchrotron lookup table should be either generated or loaded from disk to enable
     this process (see "Lookup tables for QED modules" section below).
     `<species>` must be either an electron or a positron species.
     **This feature requires to compile with QED=TRUE**
 
 * ``<species>.do_qed_breit_wheeler`` (`int`) optional (default `0`)
-    It only works if `<species>.do_qed = 1`. Enables non-linear Breit-Wheeler process for this species.
+    Enables non-linear Breit-Wheeler process for this species.
     Breit-Wheeler lookup table should be either generated or loaded from disk to enable
     this process (see "Lookup tables for QED modules" section below).
     `<species>` must be a photon species.
@@ -1160,6 +1159,7 @@ Numerics and algorithms
 
     - ``backwardeuler`` is a fully-implicit, first-order in time scheme for E-update (default).
     - ``laxwendroff`` is the semi-implicit, second order in time scheme for E-update.
+
     Comparing the two methods, Lax-Wendroff is more prone to developing oscillations and requires a smaller timestep for stability. On the other hand, Backward Euler is more robust but it is first-order accurate in time compared to the second-order Lax-Wendroff method.
 
 * ``macroscopic.sigma_function(x,y,z)``, ``macroscopic.epsilon_function(x,y,z)``, ``macroscopic.mu_function(x,y,z)`` (`string`)
@@ -1340,7 +1340,7 @@ Numerics and algorithms
     Non-zero `v_galilean` activates Galilean algorithm, which suppresses the Numerical Cherenkov instability
     in boosted-frame simulation. This requires the code to be compiled with `USE_PSATD=TRUE`.
     (see the sub-section Numerical Stability and alternate formulation
-    in a Galilean frame in :doc:`../theory/boosted-frame`).
+    in a Galilean frame in :doc:`../theory/boosted_frame`).
     It also requires the use of the `direct` current deposition option
     `algo.current_deposition = direct` (does not work with Esirkepov algorithm).
 
@@ -1358,14 +1358,14 @@ Numerics and algorithms
     boxes, an instability can occur if they have too different values.
     This option makes sure that they are synchronized periodically.
 
-* ``warpx.use_hybrid_QED`` ('bool'; default: 0)
+* ``warpx.use_hybrid_QED`` (`bool`; default: 0)
     Will use the Hybird QED Maxwell solver when pushing fields: a QED correction is added to the
     field solver to solve non-linear Maxwell's equations, according to [Quantum Electrodynamics
     vacuum polarization solver, P. Carneiro et al., `ArXiv 2016 <https://arxiv.org/abs/1607.04224>`__].
     Note that this option can only be used with the PSATD build. Furthermore,
     warpx.do_nodal must be set to `1` which is not its default value.
 
- * ``warpx.quantum_xi`` ('float'; default: 1.3050122.e-52)
+ * ``warpx.quantum_xi`` (`float`; default: 1.3050122.e-52)
      Overwrites the actual quantum parameter used in Maxwell's QED equations. Assigning a
      value here will make the simulation unphysical, but will allow QED effects to become more apparent.
      Note that this option will only have an effect if the ``warpx.use_Hybrid_QED`` flag is also triggered.
@@ -1405,13 +1405,13 @@ Boundary conditions
     Whether to create the PML inside the simulation area or outside. If inside,
     it allows the user to propagate particles in PML and to use extended PML
 
-* ``warpx.do_pml_has_particles`` (`int`; default: 0)
+* ``warpx.pml_has_particles`` (`int`; default: 0)
     Whether to propagate particles in PML or not. Can only be done if PML are in simulation domain,
     i.e. if `warpx.do_pml_in_domain = 1`.
 
 * ``warpx.do_pml_j_damping`` (`int`; default: 0)
     Whether to damp current in PML. Can only be used if particles are propagated in PML,
-    i.e. if `warpx.do_pml_has_particles = 1`.
+    i.e. if `warpx.pml_has_particles = 1`.
 
 * ``warpx.do_pml_Lo`` (`2 ints in 2D`, `3 ints in 3D`; default: `1 1 1`)
     The directions along which one wants a pml boundary condition for lower boundaries on mother grid.
@@ -1538,8 +1538,7 @@ In-situ capabilities can be used by turning on Sensei or Ascent (provided they a
     Lower corner of the output fields (if smaller than ``warpx.dom_lo``, then set to ``warpx.dom_lo``). Currently, when the ``diag_lo`` is different from ``warpx.dom_lo``, particle output is disabled.
 
 * ``<diag_name>.diag_hi`` (list `float`, 1 per dimension) optional (default `+infinity +infinity +infinity`)
-    Higher corner of the output fields (if larger than ``warpx.dom_hi``, then set to ``warpx.dom_hi``). Currently, when the ``diag_hi`` is different from ``warpx.dom_hi``, particle output i
-s disabled.
+    Higher corner of the output fields (if larger than ``warpx.dom_hi``, then set to ``warpx.dom_hi``). Currently, when the ``diag_hi`` is different from ``warpx.dom_hi``, particle output is disabled.
 
 * ``<diag_name>.write_species`` (`0` or `1`) optional (default `1`)
    Whether to write species output or not. For checkpoint format, always set this parameter to 1.
@@ -1873,6 +1872,20 @@ Reduced Diagnostics
             the macroparticle weight will be used to compute
             the histogram, and no normalization will be done.
 
+        * ``<reduced_diags_name>.filter_function(t,x,y,z,ux,uy,uz)`` (`string`) optional
+            Users can provide an expression returning a boolean for whether a particle is taken
+            into account when calculating the histogram (the exact test is whether the return
+            value is `> 0.5`).
+            `t` represents the physical time in seconds during the simulation.
+            `x, y, z` represent particle positions in the unit of meter.
+            `ux, uy, uz` represent particle velocities in the unit of
+            :math:`\gamma v/c`, where
+            :math:`\gamma` is the Lorentz factor,
+            :math:`v/c` is the particle velocity normalized by the speed of light.
+            E.g. If provided `(x>0.0)*(uz<10.0)` only those particles located at
+            positions `x` greater than `0`, and those having velocity `uz` less than 10,
+            will be taken into account when calculating the histogram.
+
         The output columns are
         values of the 1st bin, the 2nd bin, ..., the nth bin.
         An example input file and a loading pything script of
@@ -1914,16 +1927,15 @@ Reduced Diagnostics
     The default separator is a whitespace.
 
 Lookup tables and other settings for QED modules
------------------------------------------------------------------------------
+------------------------------------------------
 
 Lookup tables store pre-computed values for functions used by the QED modules.
-**This feature requires to compile with QED=TRUE (and also with QED_TABLE_GEN=TRUE for table generation) **
+**This feature requires to compile with QED=TRUE (and also with QED_TABLE_GEN=TRUE for table generation)**
 
 * ``qed_bw.lookup_table_mode`` (`string`)
     There are three options to prepare the lookup table required by the Breit-Wheeler module:
 
-    * ``builtin``:  a built-in table is used (Warning: the table gives reasonable results but its resolution
-    is quite low).
+    * ``builtin``:  a built-in table is used (Warning: the table gives reasonable results but its resolution is quite low).
 
     * ``generate``: a new table is generated. This option requires Boost math library
       (version >= 1.66) and to compile with ``QED_TABLE_GEN=TRUE``. All
@@ -1958,8 +1970,7 @@ Lookup tables store pre-computed values for functions used by the QED modules.
 * ``qed_qs.lookup_table_mode`` (`string`)
     There are three options to prepare the lookup table required by the Quantum Synchrotron module:
 
-    * ``builtin``: a built-in table is used (Warning: the table gives reasonable results but its resolution
-    is quite low).
+    * ``builtin``: a built-in table is used (Warning: the table gives reasonable results but its resolution is quite low).
 
     * ``generate``: a new table is generated. This option requires Boost math library
       (version >= 1.66) and to compile with ``QED_TABLE_GEN=TRUE``. All