Skip to content

Commit 6e6b86c

Browse files
committed
fix(results): exact fractional confidence-level labels family-wide
Every text surface that names a confidence level computed the percent with int((1 - alpha) * 100) (truncation), int(round(...)) (rounding), or :.0f, so alpha=0.025 printed "97%" or "98%" over a 97.5% interval. All 34 sites now route through one shared formatter in results_base (_coverage_level / _coverage_pct, plus _alpha_pct for the dCDH "Significant at" line, which printed 2% for alpha=0.025). Default-alpha output is byte-identical. BusinessReport headline.ci_level carries the exact level: int when integral (95 unchanged), float otherwise (97.5). No schema-version bump (REPORTING.md Note). tests/test_coverage_label.py pins the formatter, the representative surfaces at alpha=0.025, and adds a source guard against reintroducing an inline percent computation. Retires the TODO.md row.
1 parent e11d414 commit 6e6b86c

31 files changed

Lines changed: 362 additions & 62 deletions

‎TODO.md‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,6 @@ Related tracking surfaces:
5757
| `WooldridgeDiD` does not apply the W2025 Sec 5.4 `D_{G_max} x X` covariate normalization, and three sibling covariate rank deficiencies are pre-existing. Measured with the period range pinned and only the never-treated units toggled: (1) time-invariant `exovar` is absorbed by the unit FE, 4 of 26 columns, IDENTICALLY with and without never-treated units; (2) `xgvar`'s cell x covariate block, 19 of 41, identical on both panels; (3) `xtvar` under `demean_covariates=False` does exhibit the `sum_g D_g x = x` dependency that the default demeaning removes; (4) the newly-reachable case -- time-VARYING data passed through `exovar`, which its own docstring reserves for time-invariant covariates -- where the paper's `dT_i` rule would give a deterministic `D_{G_max} x X` drop instead of QR's arbitrary pick (coefficients unaffected, `1.35e-14`; `rank_deficient_action="error"` raises). REGISTRY's narrowed Sec 5.4 note cross-references this row. **Trap for whoever takes it:** `xtvar` under the DEFAULT `demean_covariates=True` is FULL RANK -- the raw block carries demeaned values while `D_g x X` carries raw ones -- and forcing the drop there moves `overall_att` 1.11903 -> 1.46269. Pinned as-is by `TestComparisonSupportFiltering::test_cells_derived_groups_did_not_leak_into_the_design`. | `diff_diff/wooldridge.py` | #729-followup | Heavy | Medium |
5858
| `WooldridgeDiD.n_control_units` counts never-treated UNITS on `control_group="never_treated"` regardless of method, but on the nonlinear paths (`logit`/`poisson`) treated units' pre-treatment rows ARE the identifying comparison -- only the OLS path absorbs them into their own cells. So the reported count under-states the comparison pool exactly where the REGISTRY control-pool asymmetry note applies. Widen to `not_yet_treated or (never_treated and method != "ols")`, or document the count as never-treated-units-by-definition. Behavior is PRE-EXISTING; documented for now in the REGISTRY control-pool Note rather than changed, because widening moves a public results field and wants its own ledger row and test matrix. | `diff_diff/wooldridge.py` | #729-followup | Mid | Low |
5959
| `WooldridgeDiD` has no opt-out for comparison-support period filtering: a user who would rather see the refusal than a reduced sample cannot ask for it. Adding one means a constructor parameter (`get_params`/`set_params` propagation, transactional validation), a ledger row, and a test matrix across both predicate branches and all three `rank_deficient_action` modes -- deliberately out of scope for the change that introduced the filter. The always-on warning is the interim answer. | `diff_diff/wooldridge.py` | #729-followup | Mid | Low |
60-
| Fractional confidence-level display unification: every `summary()` header computes `int((1 - alpha) * 100)` (~19 truncating sites incl. the shipped M-146 staggered family), so a fit `alpha=0.025` prints "97% Confidence Interval" over a 97.5% interval; `plot_dose_response`'s band label now renders exact fractional coverage via its `_coverage_label` helper - unify the summary surfaces on the same formatter (cross-family display change: moves every summary golden/doctest that pins a header, so it wants one sweep with its own test recapture, not a per-estimator drip) | `diff_diff/results_base.py`, `diff_diff/results.py` | alpha-guard review | Quick | Low |
6160

6261
### Performance
6362

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
### Fixed
2+
- **Exact fractional confidence-level labels, family-wide**: every text surface
3+
that names a confidence level now prints the exact coverage (`97.5%` for
4+
`alpha=0.025`; previously truncated to `97%` by `int((1 - alpha) * 100)` or
5+
rounded to `98%` by `int(round(...))` / `:.0f`) via one shared
6+
`results_base._coverage_pct` formatter: the 14 `summary()` headers
7+
(DiD/TWFE/MultiPeriod/SyntheticDiD, CallawaySantAnna, staggered and 2x2x2
8+
TripleDifference, StackedDiD, ImputationDiD, TwoStageDiD, EfficientDiD,
9+
ContinuousDiD, dCDH, SunAbraham, TROP), the `EventStudyResults` / HAD / RDD /
10+
ETWFE / LWDiD / LPDiD / ChangesInChanges table headers, the CS and dCDH sup-t
11+
band labels, the dCDH HonestDiD block (whose "Significant at" line printed
12+
`2%` for `alpha=0.025`; now `2.5%`), `WildBootstrapResults` and the LWDiD
13+
wild-cluster-bootstrap summaries, `HonestDiDResults` / `PlaceboTestResults`
14+
summaries, and BusinessReport / DiagnosticReport prose. The BusinessReport
15+
headline `ci_level` field carries the exact level as an `int` when integral
16+
(`95` is byte-unchanged) and a `float` otherwise (`97.5`); no schema-version
17+
bump (REPORTING.md Note). Default-alpha output is byte-identical. A source
18+
guard (`tests/test_coverage_label.py`) rejects any reintroduced inline
19+
percent computation.

‎diff_diff/business_report.py‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@
4848

4949
from diff_diff._reporting_helpers import describe_target_parameter
5050
from diff_diff.diagnostic_report import DiagnosticReport, DiagnosticReportResults
51-
from diff_diff.results_base import Diagnostic
51+
from diff_diff.results_base import Diagnostic, _coverage_level, _coverage_pct
5252

5353
BUSINESS_REPORT_SCHEMA_VERSION = "2.0"
5454

@@ -534,7 +534,7 @@ def _build_schema(self) -> Dict[str, Any]:
534534
"ci_upper": None,
535535
"alpha_was_honored": True,
536536
"alpha_override_caveat": None,
537-
"ci_level": int(round((1.0 - self._context.alpha) * 100)),
537+
"ci_level": _coverage_level(self._context.alpha),
538538
"p_value": None,
539539
"is_significant": False,
540540
"near_significance_threshold": False,
@@ -683,7 +683,7 @@ def _extract_headline(self, dr_schema: Optional[Dict[str, Any]]) -> Dict[str, An
683683
f"for the confidence interval because this fit uses "
684684
f"{inference_label} inference; the displayed CI remains "
685685
f"at the fit's native level "
686-
f"({int(round((1.0 - result_alpha) * 100))}%). The "
686+
f"({_coverage_pct(result_alpha)}%). The "
687687
f"significance phrasing still uses the requested alpha."
688688
)
689689

@@ -700,7 +700,7 @@ def _extract_headline(self, dr_schema: Optional[Dict[str, Any]]) -> Dict[str, An
700700
)
701701
if att is None or not np.isfinite(att):
702702
sign = "undefined"
703-
ci_level = int(round((1.0 - display_alpha) * 100))
703+
ci_level = _coverage_level(display_alpha)
704704
# bool(...) coerces away numpy bool_ — when ``p`` is a numpy NaN (e.g.
705705
# SyntheticControl, whose analytical p_value is always NaN), ``np.isfinite``
706706
# yields a numpy bool that is NOT JSON-serializable in the schema.
@@ -2073,7 +2073,7 @@ def _significance_phrase(p: Optional[float], alpha: float) -> str:
20732073
"""
20742074
if p is None or not np.isfinite(p):
20752075
return "statistical significance cannot be assessed (p-value unavailable)"
2076-
ci_level = int(round((1.0 - alpha) * 100))
2076+
ci_level = _coverage_pct(alpha)
20772077
if p < 0.001:
20782078
return "the direction of the effect is strongly supported by the data"
20792079
if p < 0.01:

‎diff_diff/chaisemartin_dhaultfoeuille_results.py‎

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,13 @@
3434
from diff_diff._deprecation import deprecated_field_property
3535
from diff_diff.aggregation import AggregationMixin, AggregationResult
3636
from diff_diff.results import _get_significance_stars
37-
from diff_diff.results_base import BaseResults, _require_fit_alpha, build_event_study_surface
37+
from diff_diff.results_base import (
38+
BaseResults,
39+
_alpha_pct,
40+
_coverage_pct,
41+
_require_fit_alpha,
42+
build_event_study_surface,
43+
)
3844

3945
__all__ = [
4046
"ChaisemartinDHaultfoeuilleResults",
@@ -935,7 +941,7 @@ def summary(self, alpha: Optional[float] = None) -> str:
935941
decomposition diagnostic, and a footer of significance codes.
936942
"""
937943
alpha = _require_fit_alpha(alpha, self.alpha)
938-
conf_level = int((1 - alpha) * 100)
944+
conf_level = _coverage_pct(alpha)
939945
width = 85
940946
sep = "=" * width
941947
thin = "-" * width
@@ -1590,7 +1596,7 @@ def _render_path_effects_section(
15901596
if self.path_sup_t_bands is not None and path in self.path_sup_t_bands:
15911597
crit_p = self.path_sup_t_bands[path].get("crit_value", np.nan)
15921598
if np.isfinite(crit_p):
1593-
conf_level = int((1 - self.alpha) * 100)
1599+
conf_level = _coverage_pct(self.alpha)
15941600
lines.append(
15951601
f" Sup-t critical value: {crit_p:.4f} "
15961602
f"(simultaneous {conf_level}% bands)"
@@ -1605,7 +1611,7 @@ def _render_honest_did_section(self, lines: List[str], width: int, thin: str) ->
16051611
method_label = hd.method.replace("_", " ").title()
16061612
m_val = hd.M
16071613
sig_label = "Yes" if hd.is_significant else "No"
1608-
conf_pct = int((1 - hd.alpha) * 100)
1614+
conf_pct = _coverage_pct(hd.alpha)
16091615
lines.extend(
16101616
[
16111617
thin,
@@ -1625,7 +1631,7 @@ def _render_honest_did_section(self, lines: List[str], width: int, thin: str) ->
16251631
f"{'Identified set:':<35} " f"[{_fmt_float(hd.lb)}, {_fmt_float(hd.ub)}]",
16261632
f"{'Robust ' + str(conf_pct) + '% CI:':<35} "
16271633
f"[{_fmt_float(hd.ci_lb)}, {_fmt_float(hd.ci_ub)}]",
1628-
f"{'Significant at ' + str(int(hd.alpha * 100)) + '%:':<35} " f"{sig_label:>10}",
1634+
f"{'Significant at ' + _alpha_pct(hd.alpha) + '%:':<35} " f"{sig_label:>10}",
16291635
thin,
16301636
"",
16311637
]

‎diff_diff/changes_in_changes_results.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
import pandas as pd
88

99
from diff_diff._deprecation import deprecated_field_property
10-
from diff_diff.results_base import BaseResults
10+
from diff_diff.results_base import BaseResults, _coverage_pct
1111

1212
_ESTIMATOR_TITLES = {
1313
"cic": "Changes-in-Changes (Athey & Imbens 2006) Results",
@@ -159,7 +159,7 @@ def summary(self) -> str:
159159
"""Fixed-width text summary: headline ATT block plus the quantile-effects table."""
160160
from diff_diff.results import _get_significance_stars
161161

162-
ci_pct = int(round((1 - self.alpha) * 100))
162+
ci_pct = _coverage_pct(self.alpha)
163163
width = 88
164164
bar = "=" * width
165165
dash = "-" * width

‎diff_diff/continuous_did_results.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
from diff_diff.results_base import (
1919
_SUMMARY_ALPHA_MESSAGE,
2020
BaseResults,
21+
_coverage_pct,
2122
_require_fit_alpha,
2223
build_event_study_surface,
2324
)
@@ -274,7 +275,7 @@ def summary(self, alpha: Optional[float] = None) -> str:
274275
or relabeling. Re-fit at the desired alpha instead.
275276
"""
276277
alpha = _require_fit_alpha(alpha, self.alpha, message=_SUMMARY_ALPHA_MESSAGE)
277-
conf_level = int((1 - alpha) * 100)
278+
conf_level = _coverage_pct(alpha)
278279
w = 85
279280

280281
lines = [

‎diff_diff/diagnostic_report.py‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@
4949
import pandas as pd
5050

5151
from diff_diff._reporting_helpers import describe_target_parameter # noqa: E402 (top-level import)
52-
from diff_diff.results_base import Diagnostic
52+
from diff_diff.results_base import Diagnostic, _coverage_pct
5353

5454
DIAGNOSTIC_REPORT_SCHEMA_VERSION = "2.0"
5555

@@ -4461,9 +4461,9 @@ def _render_overall_interpretation(schema: Dict[str, Any], labels: Dict[str, str
44614461
# stays consistent with the rendered interval when alpha != 0.05.
44624462
headline_alpha = headline.get("alpha") if isinstance(headline, dict) else None
44634463
if isinstance(headline_alpha, (int, float)) and 0 < headline_alpha < 1:
4464-
ci_level = int(round((1.0 - headline_alpha) * 100))
4464+
ci_level = _coverage_pct(headline_alpha)
44654465
else:
4466-
ci_level = 95
4466+
ci_level = "95"
44674467
ci_finite = (
44684468
isinstance(ci, (list, tuple))
44694469
and len(ci) == 2

‎diff_diff/diagnostics.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@
2020
from diff_diff._deprecation import NOT_SUPPLIED, require_arg, resolve_renamed_kwarg
2121
from diff_diff.estimators import DifferenceInDifferences
2222
from diff_diff.results import _get_significance_stars
23-
from diff_diff.results_base import Diagnostic
23+
from diff_diff.results_base import Diagnostic, _coverage_pct
2424
from diff_diff.utils import safe_inference, validate_binary
2525

2626

@@ -87,7 +87,7 @@ def significance_stars(self) -> str:
8787

8888
def summary(self) -> str:
8989
"""Generate formatted summary of placebo test results."""
90-
conf_level = int((1 - self.alpha) * 100)
90+
conf_level = _coverage_pct(self.alpha)
9191

9292
lines = [
9393
"=" * 65,

‎diff_diff/efficient_did_results.py‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,12 @@
2121
from diff_diff.efficient_did_aggregation import _EfficientAggregationMixin
2222
from diff_diff.efficient_did_bootstrap import EfficientDiDBootstrapMixin
2323
from diff_diff.results import _format_survey_block, _get_significance_stars
24-
from diff_diff.results_base import BaseResults, _require_fit_alpha, build_event_study_surface
24+
from diff_diff.results_base import (
25+
BaseResults,
26+
_coverage_pct,
27+
_require_fit_alpha,
28+
build_event_study_surface,
29+
)
2530

2631
if TYPE_CHECKING:
2732
from diff_diff.efficient_did_bootstrap import EDiDBootstrapResults
@@ -647,7 +652,7 @@ def summary(self, alpha: Optional[float] = None) -> str:
647652
never recomputed or relabeled - re-fit at the desired alpha).
648653
"""
649654
alpha = _require_fit_alpha(alpha, self.alpha)
650-
conf_level = int((1 - alpha) * 100)
655+
conf_level = _coverage_pct(alpha)
651656

652657
lines = [
653658
"=" * 85,

‎diff_diff/had.py‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,7 @@
9090
BiasCorrectedFit,
9191
bias_corrected_local_linear,
9292
)
93-
from diff_diff.results_base import BaseResults, build_event_study_surface
93+
from diff_diff.results_base import BaseResults, _coverage_pct, build_event_study_surface
9494
from diff_diff.survey import (
9595
SurveyMetadata,
9696
compute_survey_metadata,
@@ -444,7 +444,7 @@ def __repr__(self) -> str:
444444
def summary(self) -> str:
445445
"""Formatted summary table."""
446446
width = 72
447-
conf_level = int((1 - self.alpha) * 100)
447+
conf_level = _coverage_pct(self.alpha)
448448
lines = [
449449
"=" * width,
450450
"HeterogeneousAdoptionDiD Estimation Results".center(width),
@@ -884,7 +884,7 @@ def __repr__(self) -> str:
884884
def summary(self) -> str:
885885
"""Formatted per-horizon summary table."""
886886
width = 80
887-
conf_level = int((1 - self.alpha) * 100)
887+
conf_level = _coverage_pct(self.alpha)
888888
lines = [
889889
"=" * width,
890890
"HeterogeneousAdoptionDiD Event-Study Results".center(width),

0 commit comments

Comments
 (0)