Log sum exp signed

[andrjohns]

Summary

This PR introduces a function for performing log_sum_exp while respecting the signs of the input variables and tracking the sign of the returned sum. The function returns a two-element tuple, where the first element is the sum (on the log-scale) and the second is the sign. This isn't intended to be exposed to users, so I think the tuple return is fine.

We currently have to exponentiate and apply the signs to each argument (so that the values are correctly added or subtracted) and then extract the sign of the result and return the log(fabs()) of the sum:

double a = 2.1;
double b = 54.21;
int a_sign = 1;
int b_sign = -1;

double sum_exp_scale = a_sign * exp(a) + b_sign * exp(b);
double sum_log_scale = log(fabs(sum_exp_scale));
int sum_log_scale_sign = sign(sum_exp_scale);

However, this means that we lose the over-/under-flow robustness of the log_sum_exp and log_diff_exp functions. The log_sum_exp_signed function uses the provided signs to determine whether a log_sum_exp or log_diff_exp should be applied and also what the returned sign should be:

std::tuple<double, int> sum_new = log_sum_exp_signed(a, a_sign, b, b_sign);

This will be used in the hypergeometric_* functions and their derivatives which accumulate an infinite sum on the log-scale while tracking the input and output signs.

Tests

prim tests are added to ensure that different combinations of positive and negative arguments, as well as different magnitudes of arguments (i.e., a > b vs b > a) are correctly handled, as well as mix tests for the gradients

Side Effects

N/A

Release notes

Added log_sum_exp_signed function for computing log_sum_exp while respecting signs of arguments and tracking the sign of the result

Checklist

• Math issue Add a signed log_sum_exp #2592

• Copyright holder: Andrew Johnson

  The copyright holder is typically you or your assignee, such as a university or company. By submitting this pull request, the copyright holder is agreeing to the license the submitted work under the following licenses:
  - Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
  - Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

• the basic tests are passing

  • unit tests pass (to run, use: ./runTests.py test/unit)
  • header checks pass, (make test-headers)
  • dependencies checks pass, (make test-math-dependencies)
  • docs build, (make doxygen)
  • code passes the built in C++ standards checks (make cpplint)

• the code is written in idiomatic C++ and changes are documented in the doxygen

• the new changes are tested

[spinkney]

thanks @andrjohns, this will be nice to have!

[rok-cesnovar]

The C++ side of this is good and make sense based on the specified issue. As for the utilization of this in Stan, I will defer to others that know more than me there :)