Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add validation workflow from MC & add 2023 files #91

Merged
merged 40 commits into from
Mar 28, 2024
Merged

feat: add validation workflow from MC & add 2023 files #91

merged 40 commits into from
Mar 28, 2024

Conversation

Ming-Yan
Copy link
Collaborator

New content

  • testfile/notebook : add new notebook to extract ROC from validation workflow
  • workflow/utils: add validation workflows & btag mask
  • doc/condor:add new modules condor/missingFiles.py
  • data, utils: include 2023 conditions

Bug fix

  • ci: change certificate file permission
  • script: fix template file name
  • script: fix plot code
  • workflow : correct syst setup

Other changes

  • util : change the implementation of jet veto

hhsia and others added 24 commits October 6, 2023 11:41
test file for QCD
4f96701
QCD Pt binned samples for QCD workflow
79d8360
Data samples for QCD workflow
4de6c26
add QCD workflow selection
9a36495
add UL18 files for QCD
49e1301
add histograms for QCD workflow
5f86e77
PU weights for QCD workflow
6ecf437
HLT_PFJet140 prescales
84e59e6
add variables for QCD workflow
6b6f962
add QCD Pt binned samples xsection
a3c3ef4
QCD workflow, need debug
1db83bf
add QCD workflow
c936cf9
add QCD plotting
266e762
prescales for HLT_PFJet140
8453835
script to prepare json file for prescales
f99dc0f
csv file contains HLT_PFJet140 prescales information
50575b9
@uttiyasarkar
Resolved conflicts
5a1298d
@Ming-Yan
fix : add new files & change tagger axis & minor fixes
b437150 
- metadata : modified file list
- data: add jetveto ap
- wf : remove outlier config
- scripts: fixes for plotting code for comparison plot
@Ming-Yan
feat: add condor resubmission scripts & fix DY hists
fd79a80
@hsinweihsia
Merge pull request #2 from uttiyasarkar/devhsin
f2a5db0 
Conflicts resolved. New feature of QCD from Hsin
@Ming-Yan
feat : add validation workflow & ROC/efficiency script
3047c9b
@Ming-Yan
Merge branch 'master' of github.com:cms-btv-pog/BTVNanoCommissioning
5ac19b9 
Conflicts:
	.sites_map.json
	metadata/data_Summer22EE_Run3_2022_btagmu_BTV_Comm_v2_NanoV12.json
	metadata/data_Summer22EE_Run3_2022_jetmet_BTV_Comm_v2_NanoV12.json
	metadata/data_Summer22_Run3_2022_btagmu_BTV_Comm_v2_NanoV12.json
	metadata/data_Summer22_Run3_2022_jetmet_BTV_Comm_v2_NanoV12.json
	setup.cfg
	src/BTVNanoCommissioning/utils/correction.py
@Ming-Yan
fix: correction
73ffd99
@Ming-Yan
fix: correction
668d775
@review-notebook-app Review Notebook App
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

@Ming-Yan
Copy link
Collaborator Author

@adeiorio can you please take a look whether the changes of the condor script included?

@adeiorio
Copy link
Contributor

@adeiorio can you please take a look whether the changes of the condor script included?

Hi @Ming-Yan the changes in the condor script are ok. Thanks!

@Ming-Yan
Merge branch 'master' into QCD
8526451 
Conflicts:
	src/BTVNanoCommissioning/utils/AK4_parameters.py
	src/BTVNanoCommissioning/utils/histogrammer.py
	src/BTVNanoCommissioning/utils/selection.py
@Ming-Yan
fix: compatible with current changes
5fbffff
@Ming-Yan
fix: working QCD
5967ba9
@Ming-Yan
feat: fixed QCD
ee43ed6
@Ming-Yan
feat: add veto
0331cfa
@Ming-Yan
feat: add Summer23 info
f5ae16c
@Ming-Yan
fix : format
1c9bbc8
uttiyasarkar
uttiyasarkar reviewed Mar 12, 2024
View reviewed changes
Copy link
Collaborator

@uttiyasarkar uttiyasarkar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following samples still seem to be missing in the JSON metadata:

SingleElectron:
data_Summer22_2022_e_BTV_Run3_2022_Comm_MINIAODv4_NanoV12.json
data_Summer22EE_2022_e_BTV_Run3_2022_Comm_MINIAODv4_NanoV12.json
data_Summer23_2023_e_BTV_Run3_2023_Comm_MINIAODv4_NanoV12.json
data_Summer23BPix_2023_e_BTV_Run3_2023_Comm_MINIAODv4_NanoV12.json

Naming of one file should be updated:
data_Summer23_mu_BTV_Run3_2023_Comm_MINIAODv4_NanoV12.json ->
data_Summer23_2023_mu_BTV_Run3_2023_Comm_MINIAODv4_NanoV12.json

.github/workflows/BTA_workflow.yml Outdated
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might not work for users who installed only micromamba

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is independent for which environment of the user, this is for the ci

.github/workflows/ctag_DY_workflow.yml Outdated
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as before

.github/workflows/ctag_Wc_workflow.yml Outdated
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as before

.github/workflows/ctag_ttbar_workflow.yml Outdated
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as before

.github/workflows/ttbar_workflow.yml Outdated
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as before

metadata/test_QCD.json Outdated
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this test file?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, removed!

@Ming-Yan
Copy link
Collaborator Author

Ming-Yan commented Mar 12, 2024 •
edited
Loading

The following samples still seem to be missing in the JSON metadata:

SingleElectron: data_Summer22_2022_e_BTV_Run3_2022_Comm_MINIAODv4_NanoV12.json data_Summer22EE_2022_e_BTV_Run3_2022_Comm_MINIAODv4_NanoV12.json data_Summer23_2023_e_BTV_Run3_2023_Comm_MINIAODv4_NanoV12.json data_Summer23BPix_2023_e_BTV_Run3_2023_Comm_MINIAODv4_NanoV12.json

There were no dedicated electron workflows

Naming of one file should be updated: data_Summer23_mu_BTV_Run3_2023_Comm_MINIAODv4_NanoV12.json -> data_Summer23_2023_mu_BTV_Run3_2023_Comm_MINIAODv4_NanoV12.json

Done

Regarding to the mamba environment for the workflows, this is for the git actions :) should independent from user environment :)

I have still open items to fix

  • JEC for 2023 BPix

@Ming-Yan
Copy link
Collaborator Author

change the recommendation to 2023.
clarification of json given
emu-> stands for mueg dataset, so DY should runs on data_Summer23_2023_mu_BTV_Run3_2023_Comm_MINIAODv4_NanoV12.json
and for ctag_ttdilep_sf this is for cSF with dileptonic selection, we can use either electron/muon/mueg dataset. so this is correct as well

@uttiyasarkar
Copy link
Collaborator

Thanks, the PR looks good now. Merging it to the master branch.

@uttiyasarkar uttiyasarkar merged commit 70f87f1 into cms-btv-pog:master Mar 28, 2024
2 of 12 checks passed
@uttiyasarkar uttiyasarkar linked an issue Mar 28, 2024 that may be closed by this pull request
condor submission script #90
Closed
This was referenced Apr 4, 2024
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@uttiyasarkar uttiyasarkar uttiyasarkar left review comments

Assignees
No one assigned
Labels
CI runner
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

condor submission script
4 participants
@Ming-Yan @adeiorio @uttiyasarkar @hsinweihsia