Skip to content

Commit

Permalink
Merge pull request #29 from SpeysideHEP/bugfix
Browse files Browse the repository at this point in the history
Bugfix in $\sigma_\mu$ computation
  • Loading branch information
jackaraz authored Jan 25, 2024
2 parents 9d1f56c + 3c8ec41 commit de5f8e4
Show file tree
Hide file tree
Showing 4 changed files with 48 additions and 45 deletions.
12 changes: 9 additions & 3 deletions docs/releases/changelog-v0.1.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Release notes v0.1.4
# Release notes v0.1

## New features since last release

Expand All @@ -25,6 +25,9 @@
before approximating through $q_{\mu,A}$.
[#25](https://github.com/SpeysideHEP/spey/pull/25)

* Update clarification on text based keyword arguments.
([#30](https://github.com/SpeysideHEP/spey/pull/30))

## Bug Fixes

* In accordance to the latest updates ```UnCorrStatisticsCombiner``` has been updated with
Expand All @@ -36,8 +39,11 @@

* Numeric problem rising from `==` which has been updated to `np.isclose`
see issue [#23](https://github.com/SpeysideHEP/spey/issues/23).
[#25](https://github.com/SpeysideHEP/spey/pull/25)

([#25](https://github.com/SpeysideHEP/spey/pull/25))

* Typofix during computation of $\sigma_\mu$.
([#29](https://github.com/SpeysideHEP/spey/pull/29))

## Contributors

This release contains contributions from (in alphabetical order):
Expand Down
4 changes: 2 additions & 2 deletions docs/tutorials.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Tutorials
=========
Kitchen Sink
============

.. toctree::
:maxdepth: 2
Expand Down
2 changes: 1 addition & 1 deletion src/spey/_version.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
"""Version number (major.minor.patch[-label])"""

__version__ = "0.1.4"
__version__ = "0.1.5"
75 changes: 36 additions & 39 deletions src/spey/base/hypotest_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import warnings
from abc import ABC, abstractmethod
from functools import partial
from typing import Any, Callable, Dict, List, Optional, Text, Tuple, Union
from typing import Any, Callable, Dict, List, Literal, Optional, Text, Tuple, Union

import numpy as np
import tqdm
Expand Down Expand Up @@ -153,7 +153,7 @@ def asimov_likelihood(
poi_test: float = 1.0,
expected: ExpectationType = ExpectationType.observed,
return_nll: bool = True,
test_statistics: Text = "qtilde",
test_statistics: Literal["qtilde", "q", "q0"] = "qtilde",
**kwargs,
) -> float:
r"""
Expand All @@ -173,9 +173,9 @@ def asimov_likelihood(
* :obj:`~spey.ExpectationType.apriori`: Computes the expected p-values with via pre-fit
prescription which means that the SM will be assumed to be the truth.
return_nll (:obj:`bool`, default :obj:`True`): If ``True`` returns negative log-likelihood,
return_nll (`bool`, default `True`): If ``True`` returns negative log-likelihood,
else likelihood value.
test_statistics (:obj:`Text`, default :obj:`"qtilde"`): test statistics.
test_statistics (`Text`, default `"qtilde"`): test statistics.
* ``'qtilde'``: (default) performs the calculation using the alternative test statistic,
:math:`\tilde{q}_{\mu}`, see eq. (62) of :xref:`1007.1727`
Expand Down Expand Up @@ -206,7 +206,7 @@ def maximize_asimov_likelihood(
self,
return_nll: bool = True,
expected: ExpectationType = ExpectationType.observed,
test_statistics: Text = "qtilde",
test_statistics: Literal["qtilde", "q", "q0"] = "qtilde",
**kwargs,
) -> Tuple[float, float]:
r"""
Expand All @@ -227,18 +227,18 @@ def maximize_asimov_likelihood(
* :obj:`~spey.ExpectationType.apriori`: Computes the expected p-values with via pre-fit
prescription which means that the SM will be assumed to be the truth.
test_statistics (:obj:`Text`, default :obj:`"qtilde"`): test statistics.
test_statistics (`Text`, default `"qtilde"`): test statistics.
* ``'qtilde'``: (default) performs the calculation using the alternative test statistic,
:math:`\tilde{q}_{\mu}`, see eq. (62) of :xref:`1007.1727`
(:func:`~spey.hypothesis_testing.test_statistics.qmu_tilde`).
.. warning::
Note that this assumes that :math:`\hat\mu\geq0`, hence :obj:`allow_negative_signal`
assumed to be :obj:`False`. If this function has been executed by user, :obj:`spey`
Note that this assumes that :math:`\hat\mu\geq0`, hence `allow_negative_signal`
assumed to be `False`. If this function has been executed by user, `spey`
assumes that this is taken care of throughout the external code consistently.
Whilst computing p-values or upper limit on :math:`\mu` through :obj:`spey` this
Whilst computing p-values or upper limit on :math:`\mu` through `spey` this
is taken care of automatically in the backend.
* ``'q'``: performs the calculation using the test statistic :math:`q_{\mu}`, see
Expand Down Expand Up @@ -354,14 +354,13 @@ def chi2(
)

return 2.0 * (
self.likelihood(poi_test=poi_test, expected=expected, **kwargs)
- denominator
self.likelihood(poi_test=poi_test, expected=expected, **kwargs) - denominator
)

def _prepare_for_hypotest(
self,
expected: ExpectationType = ExpectationType.observed,
test_statistics: Text = "qtilde",
test_statistics: Literal["qtilde", "q", "q0"] = "qtilde",
**kwargs,
) -> Tuple[
Tuple[float, float],
Expand All @@ -385,18 +384,18 @@ def _prepare_for_hypotest(
* :obj:`~spey.ExpectationType.apriori`: Computes the expected p-values with via pre-fit
prescription which means that the SM will be assumed to be the truth.
test_statistics (:obj:`Text`, default :obj:`"qtilde"`): test statistics.
test_statistics (`Text`, default `"qtilde"`): test statistics.
* ``'qtilde'``: (default) performs the calculation using the alternative test statistic,
:math:`\tilde{q}_{\mu}`, see eq. (62) of :xref:`1007.1727`
(:func:`~spey.hypothesis_testing.test_statistics.qmu_tilde`).
.. warning::
Note that this assumes that :math:`\hat\mu\geq0`, hence :obj:`allow_negative_signal`
assumed to be :obj:`False`. If this function has been executed by user, :obj:`spey`
Note that this assumes that :math:`\hat\mu\geq0`, hence `allow_negative_signal`
assumed to be `False`. If this function has been executed by user, :obj:`spey`
assumes that this is taken care of throughout the external code consistently.
Whilst computing p-values or upper limit on :math:`\mu` through :obj:`spey` this
Whilst computing p-values or upper limit on :math:`\mu` through `spey` this
is taken care of automatically in the backend.
* ``'q'``: performs the calculation using the test statistic :math:`q_{\mu}`, see
Expand Down Expand Up @@ -449,7 +448,7 @@ def sigma_mu(
self,
poi_test: float,
expected: ExpectationType = ExpectationType.observed,
test_statistics: Text = "qtilde",
test_statistics: Literal["qtilde", "q", "q0"] = "qtilde",
**kwargs,
) -> float:
r"""
Expand Down Expand Up @@ -477,18 +476,18 @@ def sigma_mu(
* :obj:`~spey.ExpectationType.apriori`: Computes the expected p-values with via pre-fit
prescription which means that the SM will be assumed to be the truth.
test_statistics (:obj:`Text`, default :obj:`"qtilde"`): test statistics.
test_statistics (`Text`, default `"qtilde"`): test statistics.
* ``'qtilde'``: (default) performs the calculation using the alternative test statistic,
:math:`\tilde{q}_{\mu}`, see eq. (62) of :xref:`1007.1727`
(:func:`~spey.hypothesis_testing.test_statistics.qmu_tilde`).
.. warning::
Note that this assumes that :math:`\hat\mu\geq0`, hence :obj:`allow_negative_signal`
assumed to be :obj:`False`. If this function has been executed by user, :obj:`spey`
Note that this assumes that :math:`\hat\mu\geq0`, hence `allow_negative_signal`
assumed to be `False`. If this function has been executed by user, `spey`
assumes that this is taken care of throughout the external code consistently.
Whilst computing p-values or upper limit on :math:`\mu` through :obj:`spey` this
Whilst computing p-values or upper limit on :math:`\mu` through `spey` this
is taken care of automatically in the backend.
* ``'q'``: performs the calculation using the test statistic :math:`q_{\mu}`, see
Expand All @@ -512,7 +511,7 @@ def sigma_mu(
poi_test=poi_test, expected=expected, **kwargs
)
except MethodNotAvailable:
warnings.warm(
warnings.warn(
"Hessian implementation is not available for this backend, "
"continuing with the approximate method."
)
Expand All @@ -539,7 +538,7 @@ def exclusion_confidence_level(
poi_test: float = 1.0,
expected: ExpectationType = ExpectationType.observed,
allow_negative_signal: bool = False,
calculator: Text = "asymptotic",
calculator: Literal["asymptotic", "toy", "chi_square"] = "asymptotic",
poi_test_denominator: Optional[float] = None,
**kwargs,
) -> List[float]:
Expand Down Expand Up @@ -587,12 +586,12 @@ def exclusion_confidence_level(
allow_negative_signal (``bool``, default ``False``): If ``True`` :math:`\hat\mu`
value will be allowed to be negative.
calculator (``Text``, default ``asymptotic``): Chooses the computation basis for hypothesis
testing
calculator (``Literal["asymptotic", "toy", "chi_square"]``, default ``"asymptotic"``):
Chooses the computation basis for hypothesis testing
* ``asymptotic``: Uses asymptotic hypothesis testing to compute p-values.
* ``toy``: Uses generated toy samples to compute p-values.
* ``chi_square``: Computes p-values via chi-square;
* ``"asymptotic"``: Uses asymptotic hypothesis testing to compute p-values.
* ``"toy"``: Uses generated toy samples to compute p-values.
* ``"chi_square"``: Computes p-values via chi-square;
:math:`\chi^2=-2\log\frac{\mathcal{L}(1,\theta_1)}{\mathcal{L}(0,\theta_0)}`.
poi_test_denominator (``float``, default ``None``): Set the POI value for the null hypothesis.
Expand Down Expand Up @@ -783,9 +782,7 @@ def significance(
logpdf,
maximum_asimov_likelihood,
logpdf_asimov,
) = self._prepare_for_hypotest(
expected=expected, test_statistics="q0", **kwargs
)
) = self._prepare_for_hypotest(expected=expected, test_statistics="q0", **kwargs)

sqrt_q0, sqrt_q0A, delta_teststat = compute_teststatistics(
0.0,
Expand All @@ -805,9 +802,9 @@ def poi_upper_limit(
self,
expected: ExpectationType = ExpectationType.observed,
confidence_level: float = 0.95,
low_init: Optional[float] = None,
hig_init: Optional[float] = None,
expected_pvalue: Text = "nominal",
low_init: Optional[float] = 1.0,
hig_init: Optional[float] = 1.0,
expected_pvalue: Literal["nominal", "1sigma", "2sigma"] = "nominal",
maxiter: int = 10000,
optimiser_arguments: Optional[Dict[Text, Any]] = None,
) -> Union[float, List[float]]:
Expand All @@ -834,25 +831,25 @@ def poi_upper_limit(
confidence_level (``float``, default ``0.95``): Determines the confidence level of the upper
limit i.e. the value of :math:`1-CL_s`. It needs to be between ``[0,1]``.
low_init (``Optional[float]``, default ``None``): Lower limit for the search algorithm to start
low_init (``Optional[float]``, default ``1.0``): Lower limit for the search algorithm to start
If ``None`` it the lower limit will be determined by :math:`\hat\mu + 1.5\sigma_{\hat\mu}`.
.. note::
:math:`\sigma_{\hat\mu}` is determined via
:func:`~spey.base.hypotest_base.HypothesisTestingBase.sigma_mu` function.
hig_init (``Optional[float]``, default ``None``): Upper limit for the search algorithm to start
hig_init (``Optional[float]``, default ``1.0``): Upper limit for the search algorithm to start
If ``None`` it the upper limit will be determined by :math:`\hat\mu + 2.5\sigma_{\hat\mu}`.
.. note::
:math:`\sigma_{\hat\mu}` is determined via
:func:`~spey.base.hypotest_base.HypothesisTestingBase.sigma_mu` function.
expected_pvalue (``Text``, default ``"nominal"``): In case of :obj:`~spey.ExpectationType.aposteriori`
and :obj:`~spey.ExpectationType.apriori` expectation, gives the choice to find excluded upper
limit for statistical deviations as well.
expected_pvalue (``Literal["nominal", "1sigma", "2sigma"]``, default ``"nominal"``):
In case of :obj:`~spey.ExpectationType.aposteriori` and :obj:`~spey.ExpectationType.apriori`
expectation, gives the choice to find excluded upper limit for statistical deviations as well.
* ``"nominal"``: only find the upper limit for the central p-value. Returns a single value.
* ``"1sigma"``: find the upper limit for central p-value and :math:`1\sigma` fluctuation from
Expand Down

0 comments on commit de5f8e4

Please sign in to comment.