Add an abstract type HypothesisTest

[ararslan]

This is currently defined in HypothesisTests. Defining the common supertype in this package would allow other packages to define their own hypothesis test types that are too specific and/or have too many dependencies to upstream to HypothesisTests. This could include, for example, the log-rank test for comparing Kaplan-Meier survival curves in Survival and the likelihood ratio test in StatsModels.

[codecov-commenter]

Codecov Report

Patch and project coverage have no change.

Comparison is base (1efca7d) 100.00% compared to head (88a2f08) 100.00%.

Additional details and impacted files

@@            Coverage Diff            @@
##              main       #23   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            3         3           
  Lines           37        37           
=========================================
  Hits            37        37           

Impacted Files	Coverage Δ
src/StatsAPI.jl	100.00% <ø> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[nalimilan]

Makes sense, but how about documenting the expected API? Implementations should define pvalue and confint, right?

[ararslan]

how about documenting the expected API?

Good idea. It didn't occur to me to do so; I basically just mimicked the structure of the StatisticalModel docstring.

Implementations should define pvalue and confint, right?

Yep. Anything else you can think of? Maybe dof?

[aplavin]

Defining the common supertype in this package would allow other packages to define their own hypothesis test types

Why would one need that, what are the benefits of having this common supertype?

[ararslan]

I'm not sure I understand the question. The benefits are the same as any other abstract supertype, e.g. useful fallback methods and documented interface expectations. Like how packages that define regression models use the type defined here.

[Datseris]

Implementations should define pvalue and confint, right?

Most implementations in HypothesisTests.jl only define pvalue. In TimeserriesSUrrogates.jl we also only define pvalue for our test.

[ararslan]

We could say that pvalue is required and confint, nobs, and dof are recommended if applicable. HypothesisTests has its own pretty printing framework for tests but I don't know that that needs to be part of the expectation for every subtype across packages.

[aplavin]

HypothesisTests takes the approach that the "HypothesisTest struct" contains already computed test statistics, and pvalue(test)/confint(test) are very cheap - it's the construction that is potentially expensive.
A perfectly reasonable alternative approach is like pvalue(test, values).

Would be nice if these design decisions by HypothesisTest weren't encoded in StatsAPI.

[ararslan]

In what way are those assumptions are encoded here as-is? Or if you just mean that it isn't encoded currently but shouldn't be, I'll note that I definitely don't intend to add language to the docstring that e.g. prescribes a particular structure for HypothesisTest subtypes or anything like that.

[aplavin]

I'm talking about the comment

Implementations should define pvalue and confint, right?

I thought this meant stating pvalue(test) as a part of the interface.

[ararslan]

I figured I would just phrase it as something like "implementations should define a method for pvalue [and whatever else]," leaving out the precise signature.