Training

Basics

Atomic self-energies

Energies obtained using quantum mechanics typically contain an atomic energy offset, often called atomic self-energies. These self-energies are the energies that remain if a system loses every interatomic interaction (e.g., by increasing the interatomic distances to infinity (or, more realistically, above the interaction cutoff)). It has been observed that removing this offset helps with learning rates. Removing the offset does not change the width of the dataset energy distribution, i.e. (E_min - E_max) == (E_min_offset - E_max_offset), but moves the distribution closer to zero.

Besides faster training convergence, this is also relevant for increasing numerical precision: typically, the training is performed with a single precision, and a large offset might limit the number of relevant bits. The offset is removed from the dataset --- and energies are provided in double precision. After removing the offset, the remaining energy is cast to a PyTorch tensor and single precision.

The atomic self-energies depend on the QM method used to calculate the data points of the training set. Typically, these are provided for each element in each dataset. If this information is missing, modelforge will estimate the self-energies using linear regression.

Loss

If only energies are considered for the loss, the loss function is as follows

$$ L = \frac{1}{N} (\hat{E} - E)^{2} $$

with $N$ as the number of atoms, and $\hat{E}$ as the predicted energy for a given system. if we want to include the forces, we extend this to the following expression

$$ L = \frac{1}{N} (\hat{E_{i}} - \hat{E_{i}})^{2} \frac{1}{3N} \sum\limits_{i=1}^N \sum\limits_{\alpha=1}^{3} || -\frac{\partial \hat{U}}{\partial r_{i, \alpha}} - F_{i, \alpha}||^2 $$

with $\alpha$ as the number of cartesian coordinates (3) and, therefore, $F_{i, \alpha}$ as the force component of atom $i$ at dimension $\alpha$.