New json files for MiniAODv4, minor updates in runner and README #88
Conversation
…ing into devuttiya
…ing into devuttiya
feat: update installation from miniconda to micromamba
bug: link update of micromamba installation
bug: wrong link update
bug: error in activate command
Coffea2023 is not supported yet, forcing installation to coffea0.7.22
…ing into devuttiya
There was a problem hiding this comment.
Hi @uttiyasarkar thank you for the PR, looks good to me
I have some question regarding to the file names
- why the file name is with v3?
- if you want to modify the new file name instead of replacing the original file list, please also change recommended file name in readme.
metadata/test_data.json
Outdated
There was a problem hiding this comment.
what is this test data for ?
There was a problem hiding this comment.
Duplicate files for test, can be removed
There was a problem hiding this comment.
There are also other test_*.json duplicate files from the past. Do I clean them up as well or are they for some reason?
test_bta_run3.json
test_data.json
test_w_dj_ee.json
test_w_dj_e.json
test_w_dj_emu.json
test_w_dj_mu.json
test_w_dj_mumu.json
There was a problem hiding this comment.
these are use to construct the ci pipeline, please leave it there
|
please do reformatting via black then we can merge :) |
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
|
In the json filesets, we still have the following datasets missing:
|
|
All jsons are prepared now. @Ming-Yan please review and it should be ready to be merged. |
There was a problem hiding this comment.
Hi @uttiyasarkar thanks for the changes!
Only one minor thing on naming was not clear, I think for the file name we would like to format as following.
$CAMPAIGN refers to the --campaign in runner.py
$YEAR refers to --year in runner.py
$SAMPLE are the collections we collected for QCD/QCD-mu, rest of MC and dataset name
$TAG_EXTENSTION refers to the production string in our customize production. i.e. BTV_Run3_2022_Comm_MINIAODv4 for our current production
MC: MC_$CAMPAIGN_$YEAR_$SAMPLE_$TAG_EXTENSION
data: data_$CAMPAIGN_$YEAR_$DATASET_$TAG_EXTENSION
README.md
Outdated
| DoubleMuon (BTA,BTV_Comm_v2)| 1243MB | 848MB |1249MB| | ||
| DoubleMuon (BTA,BTV_Comm_v3)| 1243MB | 848MB |1249MB| | ||
| DoubleMuon (PFCands, BTV_Comm_v1)|1650MB |1274MB |1632MB| | ||
| DoubleMuon (Nano_v11)|1183MB| 630MB |1180MB| | ||
| WJets_inc (BTA,BTV_Comm_v2)| 1243MB |848MB |1249MB| | ||
| WJets_inc (BTA,BTV_Comm_v3)| 1243MB |848MB |1249MB| |
README.md
Outdated
| @@ -208,7 +208,7 @@ python runner.py --workflow valid --json metadata/$json file | |||
|
|
|||
| Based on Congqiao's [development](notebooks/BTA_array_producer.ipynb) to produce BTA ntuples based on PFNano. | |||
|
|
|||
| :exclamation: Only the newest version [BTV_Run3_2022_Comm_v2](https://github.com/cms-jet/PFNano/tree/13_0_7_from124MiniAOD) ntuples work. Example files are given in [this](metadata/test_bta_run3.json) json. Optimize the chunksize(`--chunk`) in terms of the memory usage. This depends on sample, if the sample has huge jet collection/b-c hardons. The more info you store, the more memory you need. I would suggest to test with `iterative` to estimate the size. | |||
| :exclamation: Only the newest version [BTV_Run3_2022_Comm_v3](https://github.com/cms-jet/PFNano/tree/13_0_7_from124MiniAOD) ntuples work. Example files are given in [this](metadata/test_bta_run3.json) json. Optimize the chunksize(`--chunk`) in terms of the memory usage. This depends on sample, if the sample has huge jet collection/b-c hardons. The more info you store, the more memory you need. I would suggest to test with `iterative` to estimate the size. | |||
There was a problem hiding this comment.
The newest version should not refers to PFNano but the btvnano development. cms-sw/cmssw#43485
There was a problem hiding this comment.
Pointed to the recent framework https://github.com/cms-btv-pog/btvnano-prod
There was a problem hiding this comment.
is there any recent to put as BP?
BTW, given Summer23 already refers to run3, I think Run3 is no longer needed in the syntax.
There was a problem hiding this comment.
same comments with all BPix samples
There was a problem hiding this comment.
choices=[
"Rereco17_94X",
"Winter22Run3",
"Summer22Run3",
"Summer22EERun3",
"Summer23",
"Summer23BPix",
"2018_UL",
"2017_UL",
"2016preVFP_UL",
"2016postVFP_UL",
],
There was a problem hiding this comment.
Thanks @uttiyasarkar looks good to me! we can merge this PR
Missing jsons:
mc_summer22EE_MINIAODv4_qcdmu
mc_summer22_MINIAODv4_qcdincl
mc_summer22EE_MINIAODv4_BTV