Skip to content

Conversation

@ck37
Copy link
Owner

@ck37 ck37 commented Jun 5, 2025

🎯 Overview

This PR addresses Issue #8 by implementing continuous outcome rescaling to transform variable importance estimates from the [0,1] scale back to the original outcome scale.

🔧 Problem

Variable importance estimates for continuous outcomes were being reported on the [0,1] scale instead of the original outcome scale, making results difficult to interpret. For example, if the original outcome ranged from 10-50, estimates were being reported as values between 0-1 rather than meaningful values in the 10-50 range.

💡 Solution

Core Changes

Modified estimate_pooled_results():

  • Added Qbounds and map_to_ystar parameters to accept outcome bounds information
  • Implemented transformation logic to convert theta estimates from [0,1] scale back to original scale
  • Added proper error handling for the transformation process

Updated vim_numerics() and vim_factors():

  • Modified to pass Qbounds and map_to_ystar parameters to estimate_pooled_results()
  • Ensures bounds information is available for the rescaling transformation
  • Maintains backward compatibility for binary outcomes

Technical Implementation

The rescaling uses the formula:

# Transform from [0,1] scale back to original scale
rescaled_estimate = estimate * (upper_bound - lower_bound) + lower_bound

Where upper_bound and lower_bound are the original outcome variable's range.

📝 Files Changed

  • R/estimate_pooled_results.R - Core rescaling logic
  • R/vim-factors.R - Pass bounds for factor variable analysis
  • R/vim-numerics.R - Pass bounds for numeric variable analysis
  • tests/testthat/test-continuous-outcome-scale.R - Comprehensive test suite
  • NEWS.md - Documentation of the fix

🧪 Testing

Added comprehensive test suite that verifies:

  • ✅ Continuous outcomes are rescaled to original scale
  • ✅ Binary outcomes continue to work correctly
  • ✅ Edge cases and error conditions are handled
  • ⚠️ Note: Tests currently fail due to VIM calculation issues that need further investigation

⚠️ Known Issues

The continuous outcome rescaling logic is implemented but currently has test failures where vim$results_all is NULL. This suggests the rescaling changes may be interfering with VIM calculations and requires additional debugging to ensure the transformation doesn't break the core functionality.

🎯 Next Steps

  1. Debug why VIM calculations are failing with the rescaling changes
  2. Ensure the transformation preserves all existing functionality
  3. Validate that rescaled estimates are mathematically correct

📚 Related

… scale

- Modified estimate_pooled_results() to accept Qbounds and map_to_ystar parameters
- Added transformation of final theta estimates from [0,1] scale back to original scale
- Updated vim_numerics.R and vim_factors.R to pass bounds information
- Added test script and documentation demonstrating the fix
- Ensures continuous outcome estimates are reported on original scale, not [0,1]
- Added NEWS.md with entry for Issue #8 fix
- Created comprehensive testthat test in test-continuous-outcome-scale.R
- Test verifies estimates are on original scale, not [0,1] scale
- Includes test for both continuous and binary outcomes
- Removed old standalone test script
…mation

The condition 'family == "binomial" && length(unique(Y)) > 2' was logically
incorrect since binomial family should have binary outcomes (length(unique(Y)) <= 2).
Simplified the condition to only check for 'family == "gaussian"' for continuous
outcome transformation.
…nuous [0,1] outcomes

The original condition 'family == "binomial" && length(unique(Y)) > 2' was correct.
Binomial family can be applied to continuous outcomes in [0,1] range (quasibinomial).
When there are >2 unique values, it indicates continuous outcome needing scale transformation.
- Allow both binomial (for continuous [0,1] outcomes) and gaussian families
- Ensures continuous outcome rescaling works for both family types
- Addresses feedback on original family condition logic
@openhands-ai
Copy link

openhands-ai bot commented Jun 5, 2025

Looks like there are a few issues preventing this PR from being merged!

  • GitHub Actions are failing:
    • R-CMD-check

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #30

Feel free to include any additional details that might help me get this PR into a better state.

You can manage your notification settings

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Transform continuous outcome estimates

3 participants