Skip to content

Commit

Permalink
minor edits to Shiny apps
Browse files Browse the repository at this point in the history
  • Loading branch information
fcgrolleau committed Aug 13, 2024
1 parent 0fb73b5 commit ed51143
Show file tree
Hide file tree
Showing 5 changed files with 12 additions and 12 deletions.
4 changes: 2 additions & 2 deletions scripts/DevWorkshop/ROC-power/shinyapp/basic-app/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
from numpy import random
from three_panel import *

app_ui = ui.page_fixed(ui.panel_title("Sample Size for Comparing Models' Area Under the ROC Curve:", "Sample Size for Comparing AUROCs"),
app_ui = ui.page_fluid(ui.panel_title("Sample Size for Comparing Models' Area Under the ROC Curve:", "Sample Size for Comparing AUROCs"),
ui.h4("Specifying parameters of two joint distributions"),
ui.row(
ui.column(
Expand Down Expand Up @@ -33,7 +33,7 @@
ui.input_numeric("ss", label="- Sample Size", value=260, min=100, max=100000),
ui.input_slider("prev", label="- Prevalence", value=.1, step=.01, min=1e-2, max=1-1e-2),
ui.input_slider("alpha_t", label="- Alpha threshold", value=.05, step=.01, min=1e-2, max=1-1e-2),
ui.input_select("n_sim", label=ui.markdown("**Choose no. of iterations to run the simulations**"), choices={0: "Zero (for parameter selection)", 100: "100 simulations (fastest, least accurate)", 500: "500 simulations (intermediate)", 2000: "2000 simulations (slowest, most accurate)"}),
ui.input_select("n_sim", label=ui.markdown("**Run the simulations**"), choices={0: "No iteration (for parameter selection)", 100: "100 iterations (fastest, least accurate)", 500: "500 iterations (intermediate)", 2000: "2000 iterations (slowest, most accurate)"}),
)
),
ui.row(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"app_url": "https://francois-grolleau.shinyapps.io/prec_auroc1/",
"app_id": 12432652,
"app_guid": null,
"title": "prec_auroc",
"title": "prec_auroc1",
"app_mode": "python-shiny",
"app_store_version": 1
}
Expand Down
12 changes: 6 additions & 6 deletions scripts/DevWorkshop/ROC-power/shinyapp/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@
margin-top: 10px; /* Add space between list items */
}
.dash-list li::before {
content: "-";
content: "\2192";
margin-right: 8px;
}
.zoom-link {
Expand All @@ -79,19 +79,19 @@
<div class="content">
<div id="intro_page">
<h2>powerROC v0.1</h2>
<p>For the evaluation of clinical prediction models, the <a href="https://www.bmj.com/content/385/bmj-2023-078378" target="_blank">TRIPOD+AI statement</a> requires that researchers report how the sample size was arrived at and justify that this sample size was sufficient to answer the research question.</p>
<p>For the evaluation of clinical prediction models, the <a href="https://www.bmj.com/content/385/bmj-2023-078378" target="_blank">TRIPOD+AI statement</a> mandates reporting the process of determining the sample size and justifying its sufficiency in addressing the research question.</p>
<p>powerROC is a web application that helps researchers determine the sample size required to estimate the area under the receiver operating characteristic curve (AUROC) with a desired level of precision or to compare the AUROCs of two models with a desired level of power.</p>

<ul class="dash-list">
<li><b>For evaluating a single prediction model</b></li>
<p style="text-indent: 40px;">click <a href="#" onclick="showContent('page1')" class="zoom-link">Sample size to precisely estimate the AUROC</a> </p> <p style="text-indent: 40px;">to find the sample size required for accurate AUROC estimation.</p>
<p style="text-indent: 40px;">click <a href="#" onclick="showContent('page1')" class="zoom-link">Sample size to precisely estimate the AUROC.</a></p>
<li><b>For comparing two prediction models</b>
<p style="text-indent: 20px;">With acess to a pilot test set (or a validation set from which you will vary the prevalence),</p><p style="text-indent: 40px;">click <a href="#" onclick="showContent('page2')" class="zoom-link">Sample size for comparing AUROCs: using a pilot test set</a> </p> <p style="text-indent: 40px;">to determine sample size based on pilot data.</p>
<p style="text-indent: 20px;">Without acess to a pilot test set or a validation set,</p><p style="text-indent: 40px;">click <a href="#" onclick="showContent('page3')" class="zoom-link">Sample size for comparing AUROCs: specifying joint distributions</a> </p> <p style="text-indent: 40px;">to calculate sample size without needing preliminary data.</p>
<p style="text-indent: 20px;">- With acess to a pilot test set (or a validation set from which you may want to vary the prevalence),</p><p style="text-indent: 40px;">click <a href="#" onclick="showContent('page2')" class="zoom-link">Sample size for comparing AUROCs: using a pilot test set</a> </p> <p style="text-indent: 40px;">to determine sample size based on pilot data.</p>
<p style="text-indent: 20px;">- Without acess to a pilot test set or a validation set,</p><p style="text-indent: 40px;">click <a href="#" onclick="showContent('page3')" class="zoom-link">Sample size for comparing AUROCs: specifying joint distributions</a> </p> <p style="text-indent: 40px;">to calculate sample size without needing preliminary data.</p>
</li>
</ul>
<br>
For any inquiries or further assistance, please contact François Grolleau at grolleau@stanford.edu.<br>
For any inquiries or further assistance, please contact François Grolleau at grolleau [ a t ] stanford.edu.<br>
© 2024 PowerROC. All rights reserved.
</div>
<div id="page1" class="hidden">
Expand Down
4 changes: 2 additions & 2 deletions scripts/DevWorkshop/ROC-power/shinyapp/pilot/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,15 @@

app_ui = ui.page_fluid(ui.panel_title("Sample Size for Comparing Models' Area Under the ROC Curve:", "Sample Size for Comparing AUROCs"),
ui.h4("Using a pilot test set"), ui.br(),
ui.input_file("file1", "Upload a pilot test set as a CSV file", accept=[".csv"], multiple=False),
ui.input_file("file1", "Upload a pilot test set, with at least three events and three nonevents, as a CSV file", accept=[".csv"], multiple=False),
ui.output_text("text"),
ui.output_table("summary_csv"),
ui.input_switch("change_prev", "Change prevalence", False),
ui.output_ui("ui_prev"),
ui.row(
ui.input_numeric("ss", label="Sample Size", value=1060, min=100, max=100000),
ui.input_slider("alpha_t", label="Alpha threshold", value=.05, step=.01, min=1e-2, max=1-1e-2),
ui.input_select("n_sim", label=ui.markdown("**Choose no. of iterations to run the simulations**"), choices={100: "100 simulations (fastest, least accurate)", 500: "500 simulations (intermediate)", 2000: "2000 simulations (slowest, most accurate)"}),
ui.input_select("n_sim", label=ui.markdown("**Run the simulations**"), choices={100: "100 iteration (fastest, least accurate)", 500: "500 iterations (intermediate)", 2000: "2000 iterations (slowest, most accurate)"}),
),
ui.row(ui.output_plot("int_plot", width='1000px', height='750px'),),
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ def three_panel_pilot(data,
if change_prev:
plt.title(f'Assuming the distribution from the provided pilot test set with a prevalence of {100*prev:.0f}%,\n the true AUROCs are {auc_A:.2f} for Model A and {auc_B:.2f} for Model B,\nbased on {n_sim} simulations, the estimated power to detect a difference in AUROC is:', fontsize=14, y=1.15)
else:
plt.title(f'Assuming the distribution from the provided pilot test set,\n the true AUROCs are {auc_A:.2f} for Model A and {auc_B:.2f} for Model B,\nbased on {n_sim} simulations, the estimated power to detect a difference in AUROC is:', fontsize=14, y=1.15)
plt.title(f'Assuming the distribution from the provided pilot test set,\n the true AUROCs are {auc_A:.2f} for Model A and {auc_B:.2f} for Model B,\nand based on {n_sim} simulations, the estimated power to detect a difference in AUROC is:', fontsize=14, y=1.15)

# add power as text
plt.text(sample_sizes[0], 1.02, f'Power is {powers[0]*100:.1f}%\nAt n={sample_sizes[0]}', ha='center', va='bottom', transform=plt.gca().get_xaxis_transform())
Expand Down

0 comments on commit ed51143

Please sign in to comment.