Skip to content

Commit

Permalink
- bug: update preregistration deviation section for yi standardisat…
Browse files Browse the repository at this point in the history
…ion / transformation #63
  • Loading branch information
egouldo committed Sep 2, 2024
1 parent 41886b6 commit 8064da7
Showing 1 changed file with 9 additions and 5 deletions.
14 changes: 9 additions & 5 deletions index.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -2192,15 +2192,19 @@ All values (predictions) were first transformed to the original scale along with
We used the square of the SE associated with predicted values as the sampling variance in the meta-analyses described below, and we planned to analyze these predicted values in exactly the same ways as we analyzed $Z_r$ in the following analyses.
::: {.callout-note appearance="simple"}
## Preregistration Deviation:
## Preregistration Deviation:
**1. Standardizing blue tit out-of-sample predictions** $y_i$
Because analysts of blue tit data chose different dependent variables on different scales, after transforming out-of-sample values to the original scales, we standardized all values as z scores ('standard scores') to put all dependent variables on the same scale and make them comparable.
This involved taking each relevant value on the original scale (whether a predicted point estimate or a SE associated with that estimate) and subtracting the value in question from the mean value of that dependent variable derived from the full dataset and then dividing this difference by the standard deviation, SD, corresponding to the mean from the full dataset (@eq-Z-VZ).
Thus, all our out-of-sample prediction values from the blue tit data are from a distribution with the mean of 0 and SD of 1.
Because analysts of blue tit data chose different dependent variables on different scales, after transforming out-of-sample values to the original scales, we standardized all values as z scores ('standard scores') to put all dependent variables on the same scale and make them comparable. This involved taking each relevant value on the original scale (whether a predicted point estimate or a SE associated with that estimate) and subtracting the value in question from the mean value of that dependent variable derived from the full dataset and then dividing this difference by the standard deviation, SD, corresponding to the mean from the full dataset (@eq-Z-VZ). Thus, all our out-of-sample prediction values from the blue tit data are from a distribution with the mean of 0 and SD of 1.
Note that we were unable to standardise some analyst-constructed variables, so these analyses were excluded from the final out-of-sample estimates meta-analysis, see @sec-excluded-yi for details and explanation.
We did not add this step for the *Eucalyptus* data because (a) all responses were on the same scale (counts of *Eucalyptus* stems) and were thus comparable and (b) these data, with many zeros and high skew, are poorly suited for z scores.
**2. Log-transforming *Eucalyptus* out-of-sample predictions** $y_i$
All analyses of the *Eucalyptus* data chose dependent variables that were on the same scale, that is, *Eucalyptus* seedling counts. Although analysts may have used different size-classes of *Eucalyptus* seedlings for their dependent variable, we considered these choices to be akin to subsetting, rather than as different response variables, since changing the size-class of the dependent variable ultimately results in observations being omitted or included. Consequently, we did not standardise *Eucalyptus* out-of-sample predictions.
We were unable to fit quasi-Poisson or Poisson meta-regressions, as desired [@ohara2010], because available meta-analysis packages (e.g. `metafor::` and `metainc::`) do not provide implementation for outcomes as estimates-only, methods are only provided for outcomes as ratios or rate-differences between two groups. Consequently, we log-transformed the out-of-sample predictions for the *Eucalyptus* data and use the mean estimate for each prediction scenario as the dependent variable in our meta-analysis with the associated SE as the sampling variance in the meta-analysis [@nakagawa2023, Table 2].
:::
We plotted individual effect size estimates ($Z_r$) and predicted values of the dependent variable ($y_i$) and their corresponding 95$\%$confidence / credible intervals in forest plots to allow visualization of the range and precision of effect size and predicted values.
Expand Down

0 comments on commit 8064da7

Please sign in to comment.