TUe Improving Statistical Questions

Eindhoven University of Technology (TU/e) course "Improving Your Statistical Questions" by Daniël Lakens on Coursera (completed December 14, 2022).

Instructor

Daniël Lakens - Associate Professor in the Human-Technology interaction group at Eindhoven University of Technology (TU/e)

All the code is contained in R files

This course examines many statistical concepts through simulations or calculations in the free software R.

Licence

The work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. https://creativecommons.org/licenses/by-nc-sa/4.0/

Programming Language

R version 4.1.2 (2021-11-01) -- "Bird Hippie"

The following packages (libraries) need to be installed

 (1.) TOSTER - Two One-Sided Tests (TOST) Equivalence Testing  You can install TOSTER in R using: install.packages('TOSTER'). 
      Alternatively, you can download a spreadsheet to perform these calculations: https://osf.io/qzjaj/

 (2.) ANOVA_power: Superpower (version 0.1.2)  - Simulation function used to estimate power  
      Alternatively, you can use a shiny app to perform these calculations: https://shiny.ieis.tue.nl/anova_power/

 (3.) METAFOR Package: A Meta-Analysis Package for R. 

 (4.) MBESS Package:   Methods for the Behavioral, Educational, and Social Sciences: An R package

The following Web-Applications are used

statcheck 1.3.0 https://rpubs.com/michelenuijten/statcheckmanual

• Automatically extracts statistics from reasearch articles and recomputes their p-values, as long as statistics are reported following guidelines from the American Psychological Association (APA). Upload a PDF, word document, or HTML file.
• Manual: https://michelenuijten.shinyapps.io/statcheck-web/

Likelihood Ratio for Mixed Results https://shiny.ieis.tue.nl/mixed_results_likelihood/

• Shiny app accompanying: Lakens, D., & Etz, A. J. (2017). Too true to be bad: When sets of studies with significant and non-significant findings are probably true. Social Psychological and Personality Science.

Positive Predictive Value (PPV) of a p-value https://shiny.ieis.tue.nl/PPV/

• When does a significant p-value indicate a true effect?

The following software is used

G*Power

• Conduct statistical power analysis and calculate probabilities as well as some more test cases
  https://download.cnet.com/G-Power/3000-2054_4-10647044.html

Course Description

This course aims to help you to ask better statistical questions when performing empirical research. We will discuss how to design informative studies, both when your predictions are correct, as when your predictions are wrong. We will question norms, and reflect on how we can improve research practices to ask more interesting questions. In practical hands on assignments you will learn techniques and tools that can be immediately implemented in your own research, such as thinking about the smallest effect size you are interested in, justifying your sample size, evaluate findings in the literature while keeping publication bias into account, performing a meta-analysis, and making your analyses computationally reproducible.

If you have the time, it is recommended that you complete my course 'Improving Your Statistical Inferences' before enrolling in this course, although this course is completely self-contained.

Week 1: Introduction 
        1.1. Introduction
        1.2. Do you really want to test a hypothesis?
        1.3. Risky predictions
            - Assignment 1.1.: testing Range Predictions
            
Week 2: Falsifying Predictions
        2.1. Falsifying Predictions in Theory
        2.2. Setting the Smallest Effect Size of Interest (SESOI)
             - Assignment 2.1.: The Smallest Telescope Approach to Setting a SESOI
             - Assignment 2.1.: Setting a SESOI based on Resources
        2.3. Falsifying Predictions in Practice
             - Assignment 2.3.: Equivalence testing
        
Week 3: Designing Informative Studies
        3.1. Justifying Error Rates
             - Assignemnt 3.1.: Confidence Intervals for Standard Deviations
        3.2. Power Analysis
             - Assignemnt 3.2.: Power Analysis for ANOVA Designs
        3.3. Simulation     
             
Week 4: Meta-Analysis and Bias Detection
        4.1. Mixed Results
             - Assignemnt 4.1.: Likelihood of Significant Findings
        4.2. Intro to Meta-Analysis
             - Assignment 4.2.: Introduction to Meta-Analysis
        4.3. Bias Detection     
             - Assignment 4.3.: Detecting Publication Bias
             - Assignment 4.4.: Checking your Stats
             
Week 5: Computational Reproductibility, Philosophy of Science, and Science Integrity
        5.1. Computational Reproducibility
             - Assignment 5.1.: Computational Reproducibility
        5.2. Philosophy of Science in Practice
             - Assignment 5.2.: Does your Philosophy of Science matters in Practice?
        5.3. Scientific Integrity in Practice
             - Assignment 5.3.: Applied research Ethics
        
Week 6: Final Exam

Eindhoven University of Technology

Eindhoven University of Technology (TU/e) is a young university, founded in 1956 by industry, local government and academia. Today, their spirit of collaboration is still at the heart of the university community. We foster an open culture where everyone feels free to exchange ideas and take initiatives.

We offer academic education that is driven by fundamental and applied research. Our educational philosophy is based on personal attention and room for individual ambitions and talents. Our research meets the highest international standards of quality. We push the limits of science, which puts us at the forefront of rapidly emerging areas of research.