Feature: add bootstrap CI to the rwa method by martinctc · Pull Request #6 · martinctc/rwa

martinctc · 2021-02-04T22:05:16Z

Summary

This branch adds bootstrap confidence intervals for Relative Weights Analysis, enabling statistical significance testing of predictor importance - a major enhancement addressing a long-standing limitation in RWA methodology.

Changes

The changes made in this PR are:

Bootstrap Infrastructure: Added complete bootstrap functionality (R/bootstrap_rwa.R) with BCA, percentile, and basic CI methods
Enhanced Main Function: Extended rwa() with bootstrap parameters (bootstrap, n_bootstrap, conf_level, focal, comprehensive, include_rescaled_ci)
Statistical Rigor: Implemented proper handling of compositional data constraints for rescaled weights with user warnings
Comprehensive Vignette: Created detailed documentation (vignettes/introduction-to-rwa.Rmd) showcasing methodology, examples, and best practices
Package Compliance: Updated DESCRIPTION with proper dependencies, fixed CRAN notes, and modernized authorship format
CI/CD Improvements: Enhanced GitHub Actions workflow with Pandoc support and streamlined testing
Documentation Updates: Refreshed README with vignette information and bootstrap feature highlights

Check

Bootstrap confidence intervals work correctly for raw weights
Proper warnings implemented for rescaled weight CIs due to compositional constraints
Comprehensive vignette demonstrates all functionality with real examples
Package builds successfully with vignettes
All dependencies properly declared in DESCRIPTION
CRAN notes addressed (Authors@R field, maintainer redundancy)
GitHub Actions R CMD checks pass with Pandoc support
Backward compatibility maintained - all existing functionality preserved
Internal functions properly documented with @Keywords internal and @nord

Key Features Added

🎯 Bootstrap Confidence Intervals

Raw Weight CIs: Statistically valid confidence intervals for relative importance
Multiple CI Methods: BCA (preferred), percentile, and basic bootstrap methods with automatic fallback
Significance Testing: Automatic flagging of significant predictors (CIs not containing zero)
Comprehensive Analysis: Optional focal variable comparisons and random variable benchmarking

📊 Statistical Methodology

Compositional Data Awareness: Proper handling of rescaled weights with user warnings about interpretation limitations
Bootstrap Sample Control: User-configurable sample sizes with sensible defaults (1000 samples)
Confidence Level Flexibility: Adjustable confidence levels (default 95%)

📚 Documentation Excellence

Comprehensive Vignette: 400+ line tutorial covering methodology, interpretation, and best practices
Real-World Examples: Diamond price analysis and mtcars demonstrations
Academic References: Proper citations to Tonidandel & LeBreton (2015), Johnson (2000), and related literature
Troubleshooting Guide: Common issues and solutions for users

🔧 Technical Implementation

Robust Error Handling: Graceful fallbacks when BCA method fails
Memory Efficient: Optimized bootstrap sampling for large datasets
Tidyverse Integration: Seamless compatibility with dplyr pipes and modern R workflows

Impact and Significance

This enhancement addresses a critical gap identified by Tonidandel et al. (2009):

"The difficulty in determining the statistical significance of relative weights stems from the fact that the exact (or small sample) sampling distribution of relative weights is unknown."

The bootstrap implementation provides an empirical solution to this long-standing methodological challenge, making the rwa package one of the most comprehensive tools for relative importance analysis in R.

Usage Examples

# Basic bootstrap analysis
result <- mtcars %>%
  rwa(outcome = "mpg", 
      predictors = c("cyl", "disp", "hp"),
      bootstrap = TRUE, 
      n_bootstrap = 1000)

# View confidence intervals
result$bootstrap$ci_results$raw_weights

# Comprehensive analysis with focal comparison
comprehensive <- diamonds %>%
  rwa(outcome = "price",
      predictors = c("carat", "depth", "table"),
      bootstrap = TRUE,
      comprehensive = TRUE,
      focal = "carat")

Note

This major feature addition maintains full backward compatibility while providing cutting-edge statistical capabilities for relative weights analysis. The implementation follows best practices for R package development and provides a foundation for future enhancements to the relative importance analysis ecosystem.

This fixes #1.

work in progress code - does not work yet

stonid · 2021-06-30T18:16:40Z

Hey Martin, The code in there for review doesn't seem to match any of the material I sent for implementing the bootstrap. How can I best help integrate that? Best, Scott T

…

On Tue, Jun 29, 2021 at 6:16 AM Martin Chan ***@***.***> wrote: @martinctc <https://github.com/martinctc> requested your review on: #6 <#6> Feature: add bootstrap CI to the rwa method. — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#6 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIT53LWAU2G7XD5VX5BQBADTVGMOLANCNFSM4XDUG75A> .

martinctc · 2021-06-30T20:02:23Z

Hey Martin,

The code in there for review doesn't seem to match any of the material I
sent for implementing the bootstrap. How can I best help integrate that?

Best,
Scott T

On Tue, Jun 29, 2021 at 6:16 AM Martin Chan @.***>
wrote:

@martinctc https://github.com/martinctc requested your review on: #6
#6 Feature: add bootstrap CI to
the rwa method.

—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
#6 (comment), or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIT53LWAU2G7XD5VX5BQBADTVGMOLANCNFSM4XDUG75A
.

Hey @stonid this pull request is still a draft and I haven't properly coded anything yet! Just wanted to add you to this, will ping you when it is ready for review.

stonid · 2021-06-30T20:04:32Z

Got it. I got a ping that I was supposed to review and was confused.

…

On Wed, Jun 30, 2021, 4:02 PM Martin Chan ***@***.***> wrote: Hey Martin, The code in there for review doesn't seem to match any of the material I sent for implementing the bootstrap. How can I best help integrate that? Best, Scott T On Tue, Jun 29, 2021 at 6:16 AM Martin Chan *@*.***> wrote: @martinctc <https://github.com/martinctc> https://github.com/martinctc requested your review on: #6 <#6> #6 <#6> Feature: add bootstrap CI to the rwa method. — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub #6 (comment) <#6 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIT53LWAU2G7XD5VX5BQBADTVGMOLANCNFSM4XDUG75A . Hey @stonid <https://github.com/stonid> this pull request is still a draft and I haven't properly coded anything yet! Just wanted to add you to this, will ping you when it is ready for review. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIT53LRCXHVXSKZ2ETKINDDTVNZ6JANCNFSM4XDUG75A> .

…al functions

…alysis

…d enforce suggest checks

…e R package checks

… additional metrics

…tandardizing OS to ubuntu-latest

Copilot

Pull Request Overview

This PR introduces bootstrap confidence interval functionality into the rwa() function, enabling statistical inference on relative weights. It also updates documentation, vignettes, and CI/CD to support the new feature.

Added full bootstrap infrastructure with multiple CI methods in R/bootstrap_rwa.R and integrated it into rwa()
Extended documentation (man/*.Rd, vignettes/*.Rmd, README.md) to cover bootstrap parameters and examples
Updated CI/CD workflow and package metadata for Pandoc support, CRAN compliance, and new dependencies

Reviewed Changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
R/rwa.R	Integrated `bootstrap`, `n_bootstrap`, `conf_level`, `focal`, `comprehensive`, `include_rescaled_ci` parameters and logic
R/bootstrap_rwa.R	Implemented bootstrap sampling and CI extraction functions
man/rwa.Rd	Updated Rd usage, arguments, and examples for bootstrap
man/run_rwa_bootstrap.Rd	Added internal documentation for bootstrap helper
man/extract_ci.Rd	Added internal documentation for CI extraction
vignettes/introduction-to-rwa.Rmd	New detailed vignette covering basic and bootstrap features
vignettes/README.md	Added instructions to access the vignette
README.md	Updated to highlight bootstrap CIs and documentation
DESCRIPTION	Bumped version, modernized Authors field, added new imports
NAMESPACE	Imported new functions from `boot`, `purrr`, `utils`
.github/workflows/R-CMD-check.yaml	Switched to Ubuntu runner, added Pandoc setup and artifact upload
.Rbuildignore	Added `_dev` to ignore list

Comments suppressed due to low confidence (1)

R/rwa.R:20

The new bootstrap parameters and logic are not covered by any unit tests. Please add tests for rwa(..., bootstrap = TRUE) and for the underlying bootstrap helper functions (e.g., run_rwa_bootstrap, extract_ci) to ensure correct behavior.

#' @param bootstrap Logical value specifying whether to calculate bootstrap confidence intervals. Defaults to `FALSE`.

Copilot · 2025-07-13T23:06:55Z

R/bootstrap_rwa.R

+#' @keywords internal
+#' Bootstrap statistic function for basic RWA weights
+#' @noRd
+rwa_boot_statistic <- function(data, indices, outcome, predictors) {


[nitpick] The functions rwa_boot_statistic and rwa_boot_statistic_rescaled share nearly identical data preparation and eigen decomposition code. Consider extracting the common steps into a helper to reduce duplication and simplify future maintenance.

Copilot · 2025-07-13T23:06:55Z

R/bootstrap_rwa.R

+  ci_results <- purrr::map_dfr(1:n_weights, function(i) {
+    tryCatch({
+      # Try BCA first
+      ci <- boot::boot.ci(boot_object, type = "bca", index = i, conf = conf_level)


[nitpick] Extracting CIs by calling boot::boot.ci inside a loop for each weight can be slow for large bootstrap samples. Consider offering an option to use a faster percentile‐only method or parallelizing the CI extraction to improve performance.

feat: add bootstrap CI

e8ccd51

work in progress code - does not work yet

martinctc self-assigned this Feb 4, 2021

martinctc requested a review from stonid June 29, 2021 10:15

martinctc added the enhancement New feature or request label Jun 29, 2021

stonid marked this pull request as ready for review June 30, 2021 17:48

This was referenced Dec 10, 2023

95%CI of rescaled weights #16

Closed

weights and missings #12

Open

martinctc added 11 commits May 30, 2024 11:54

chore: update .gitignore

71cc0d1

feat: implement bootstrap analysis for RWA with confidence intervals

1d4a6d7

feat: enhance bootstrap analysis with additional utilities and intern…

855d49d

…al functions

feat: add vignettes for comprehensive usage examples and bootstrap an…

f9eeaef

…alysis

feat: update GitHub Actions workflow to use latest action versions an…

50d0ffd

…d enforce suggest checks

feat: enhance GitHub Actions workflow for multi-OS support and improv…

866bcf8

…e R package checks

feat: enhance confidence interval interpretation in RWA vignette with…

7b56d85

… additional metrics

feat: simplify R-CMD-check workflow by removing matrix strategy and s…

5f2ebaa

…tandardizing OS to ubuntu-latest

feat: update author information format in DESCRIPTION file

9f1bc3f

feat: remove maintainer information from DESCRIPTION file

7d86869

Merge branch 'master' into feature/add-bootstrap-ci

b560a80

martinctc requested a review from Copilot July 13, 2025 23:05

Copilot AI reviewed Jul 13, 2025

View reviewed changes

martinctc merged commit 591727e into master Jul 14, 2025
2 checks passed

martinctc deleted the feature/add-bootstrap-ci branch July 16, 2025 14:19

Copilot AI mentioned this pull request Jan 21, 2026

Add weight variable and missing data handling options to rwa() #22

Open

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature: add bootstrap CI to the rwa method#6

Feature: add bootstrap CI to the rwa method#6
martinctc merged 12 commits intomasterfrom
feature/add-bootstrap-ci

martinctc commented Feb 4, 2021 •

edited

Loading

Uh oh!

stonid commented Jun 30, 2021 via email

Uh oh!

martinctc commented Jun 30, 2021

Uh oh!

stonid commented Jun 30, 2021 via email

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jul 13, 2025

Uh oh!

Copilot AI Jul 13, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

martinctc commented Feb 4, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Check

Key Features Added

🎯 Bootstrap Confidence Intervals

📊 Statistical Methodology

📚 Documentation Excellence

🔧 Technical Implementation

Impact and Significance

Usage Examples

Note

Uh oh!

stonid commented Jun 30, 2021 via email

Uh oh!

martinctc commented Jun 30, 2021

Uh oh!

stonid commented Jun 30, 2021 via email

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Jul 13, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 13, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

martinctc commented Feb 4, 2021 •

edited

Loading