Calibrations Guide
- Ant calibrations overview
- Preparation
- Calibration Process
- Order of Calibrations
- Example Calibrations (Before and After Iterations)
- How To's, Tips and Tricks
- Troubleshooting
A calibration module, in general, consists of three parts:
- A physics class which produces histograms for each detector channel (usually a 2D histogram)
- A GUI displaying the histograms channel-wise and inferring calibration parameters
- A reconstruction hook applying the calibration parameters during reconstruction
All calibrations for a beam-time (or some chunk of data to be analysed) are configured by the Setup living in ExpConfig, see How to write a Setup
The calibration module is supported by the calibration data manager, which uses a file-based database to load/store calibration parameters. The calibration root-files are stored to and read from the database directory found in the main ant directory. The calibrations are sorted into their corresponding Setup_YEAR_MONTH folders. If you prefer to store the calibration files elsewhere, you can also setup symbolic links in this directory.
Before you start your calibration efforts, certain preparations are needed or recommended:
-
Create a setup file for the beam time you will calibrate, if it doesn't already exist.
See the corresponding sub section in the How To's section below for information on how to do this. -
git-track your calibration
Besides the obvious advantages of git-tracking each step in your calibration, you can also more easily share your calibration with others who want to analyse the same beam time.
As the calibration files produced tend to get large, it is recommended to use the GitLab set up on the flora server in KPH Mainz: https://pbb-flora.kph.uni-mainz.de:8081. In the How To section below you can find instructions on how to set up a personal repository on the Flora GitLab. However, to have a common collection of all created calibrations, it is better if you join the ant_calibration group (as an existing member to add you) and create a project there for the beam time you are calibrating. -
Find good data files to calibrate on
For some of the calibrations, only a smaller subset of runs, representing your beam time, is needed. For other, e.g. when doing time-dependent calibrations, you might need nearly all the runs in your beam time. But you don't want to tune your calibrations on bad runs. Hence, make sure you are using runs with a consistent setup (e.g. no changes in high voltages, no CB time jumps, etc.) and make a list. Don't forget to share this list on the corresponding beam time page on the daq wiki
As outlined in the overview above, there are two major steps when performing a specific calibration:
-
Run the corresponding physics class to produce the histograms from which calibration parameters are to be extracted.
-
Run the GUI that handles the histograms to extract and save the calibration parameters.
A third step, based on whether or not you are git-tracking your calibrations, is:
- Commit the newly created calibration files to your git repository.
These two or three steps are done for each calibration. For some calibrations, you only need to do them once, and for other, many iterations of these steps must be done. Make sure that, when you start a new iteration, i.e. do step 1 again, you are actually using the calibration files you created in step 2. E.g. if you are doing step 1. on blaster and step 2. on your own computer, you should in step 3, also push your local calibration-git repository to GitLab and then pull it to blaster.
Below you will find more information on how to do each step in the sub sections Running the physics classes, Running the calibration GUI and Committing to the git repository
In the section Order of Calibrations below, it is attempted to provide a reasonable order of the different calibrations performed on the different detectors. Of course you must use your own good judgement. Eg. if you find strange behaviours in the spectra, you need to look into what causes these. Perhaps a previous calibration must be re-done, perhaps some runs needs to be excluded etc.
In the section Example Calibrations (Before and After Iterations) you can see some examples of how calibrations should look when they've been iterated enough.
If you are running a physics class locally:
Ant -s <name of current setup> -c <physics class> -i <name of input file> -o <name of output file>
If you are running the physics classes by submitting jobs to the farm on blaster you must first create a file called Antrc and fill this with the arguments you want to pass to the Ant job.
An example of the content of Antrc:
-s Setup_2018_05
-S DisableTimecuts=1
-c Tagger_Time
-c CB_Time
Then create a textfile containing a list of the full paths to all the runs you want to analyse. And, standing in the directory you want the resulting log and root files to end up, you submit to the farm with:
AntSubmit --filelist filelist.txt
-
Move into the directory that holds all the root-files created by the physics class. If you ran the physics class by submitting the jobs to the farm on blaster, you will find them in the directory called root which was created in the directory from where you submitted the jobs to the farm.
-
Run the following code:
Ant-calib -s <name of current setup> -c <calibration class> <all output root files from farm> --default
For example:
- In the GUI that pops up, you’ll want to make sure all the channel graphs are fitted. If a channel is fitted like the following examples:
then hit Next(or the n key) to move to the next graph. If it is not fitted, see the section Fitting Histograms for how to fit histograms before moving on to the next graph. If there is no data or something unexpected/strange, make a note of it and the channel. Once you go through all of the channels, you can hit Finish Slice(or the space bar) to finish the calibration. A bunch of black dots will appear, and if the calibration is well done, they will form a fairly straight line, like so:
Note: The PID Phi Angle calibration will not have a line of black dots, but rather a diagonal line of x's that should look like the following:
Each calibration should firstly be done time independent, using a suiting subset of your beamtime, preferably sampling the entire beamtime. Following the instructions above achieves this by using the --default option.
However, since gains might drift over time, making time dependent calibrations can have a significant impact on the energy resolution. For details on how the time dependent calibration works, see the PhD thesis of Andreas Neiser.
Firstly, make sure you have performed the time-independent calibration to obtain well-calibrated default gains. Then, to do a run-by-run calibration, use Ant-calib -a 10 as Savitzky-Golay averaging (defaults to polyorder 4, which should be ok) and subsequently reduce "fitting noise" by Ant-calib-smooth -a 20 --polyorder 6. The -a option determines how many runs should be used in the averaging. If the spectra you use for calibration have poor statistics you might want to increase this number. Don't forget to commit your Ant database and push it to your cluster. The loop is as follows for the exampel of CB energy calibration:
-
Anton full beamtime Ant-calib -a 10 -c CB_Energy_RelativeGains <input files>Ant-calib-smooth --write -a 20 --polyorder 6 -c CB_Energy_RelativeGains -s <yoursetup>- Go to step 1. if not good enough (less than 0.5% Pi0 peak offset over all channels is ok)
After every calibration, you must commit to the Git repository you created for the setup and push it. See Setting Up GIT Repository with GITLab for how to set one up.
- Navigate to your Git repository in the terminal. It should contain a directory called calibration, and likely a README.md file.
- Use the following lines of commands (one at a time) to add the new calibration, commit it, and push it to your Git repository online:
git add <name of files you've changed> or git add *
git commit -m <message about what you've changed/added>
git push origin master
For example:
Note: Use the command git status to check if you have a clean, working tree; if not, you may have missed something you need to commit. Ant-calib will complain if your repository is dirty.
This section outlines a general order in which calibrations can be done. The physics classes listed in each step can be put in the Antrc file together, unless otherwise specified. The setup can also be specified in the Antrc, although it may not need to be since the setup file should recognize the time the input files were taken.
These should not need many iterations, likely you'll only need to do them once.
When submitting the jobs to the farm, as outlined in Submitting Jobs to the Farm, be sure to include the command -S DisableTimecuts=1 in your Antrc as well. Below are the physics classes which produce the necessary histograms for these calibrations and you can list them in the Antrc file as this:
- CB_Time
- PID_Time
- TAPS_Time
- TAPSVeto_Time
- Tagger_Time
- PID_PhiAngle
The calibration modules, that will display and fit the histograms needed for each of the time calibrations as well as the PID position cailbration, have the same name as the physics classes. Run these, one after another, as specified here. After each completed calibration, commit the newly created calibration files to the git repository.
Note: During the time calibrations, note down bad channels. Before running energy calibrations we'd like to mark these in the setup file as as BadTDC.
Pedestals should only require one iteration.
Pedestal calibrations are needed for the PID, TAPS and TAPSVeto. For PID and PbWO4 there are special pedestal runs collected (usually starting with CBTaggTAPSPed). Also, BaF2 and TAPSVeto can be analysed using these files.
Include -S DisableThresholds=1 and -S DisableTimecuts=1 in your Antrc file. However, this is unfortunately not enough. Before any pedestal-calibrations exists, there is a default value used. You can see it in your Setup file for each Energy-converter. This default must be set to zero for PID, TAPS and TAPSVeto before running the physics classes for a pedestal calibration. Either you do this manually or you can have a look in Setup_2017Plus_NewTagger_Base.cc where the option SetPedestalsToZero was implemented. So, before running the physics classes, modify your Setup file and compile it (if your Setup file is based on the Setup_2017Plus_NewTagger_Base class, the SetPedestalsToZero option is already implemented and you needn't do anyting). If you are using the implemented SetPedestalsToZero option, also add -S SetPedestalsToZero=1 in your Antrc file. Note: this option only works if there are no previous pedestal calibrations available.
The following physics classes produce the needed histograms:
-c PID_Energy
-c TAPS_Energy
-c TAPS_ShortEnergy
-c TAPSVeto_Energy
The calibration modules have different names than the physics classes for pedestals; they are PID_Energy_Pedestals, TAPS_Energy_Pedestals, TAPS_ShortEnergy_Pedestals and TAPSVeto_Energy_Pedestals.
CB time walk calibrations in Ant relies on the how the time of the CB pulse depends on the uncalibrated energy value, which is why it can be done before any energy calibration.
After completing one time-walk calibration, it is best to redo the CB time calibration and perhaps after that do another CB time walk. Make sure they aren't drastically changing between calibrations.
After completing the CB time walk calibration and the CB time calibration you performed earlier, you will likely have found channels which you were unable to calibrate. Maybe they were empty, maybe the spectra looked weird, etc. Make a list of these and mark them as BadTDC in the Setup file.
Include -S DisableTimecuts=1 in your Antrc file. The needed physics class is:
-c CB_TimeWalk
The calibration module has the same name as the physics class.
The energy calibrations will require a few iterations (you'll need to re-submit and re-calibrate them multiple times until they look like the examples in Example Calibrations (Before and After Iterations).)
Additionally, the energy calibrations depend on each other. Which means that if you redo one, you might need to redo the ones depending on this as well. The dependence, and also suggested order, is as follows
- CB does not depend on any other detector (weakly on PID but only to check if the veto energy = 0) and hence should be done first
- PID depends on CB
- TAPS depends on CB since there is usually not enough statistics to produce a pi0 peak from photons in TAPS alone and TAPS hits are therefore combined with CB hits. Similar as for CB, TAPS energy calibration also weakly depends on the TAPSVeto and PID energy.
- TAPSVeto depends on TAPS
For each detector, during the first calibration(s), note which elements you are unable to calibrate. Perhaps their spectrum is completely empty and likely the channel is broken. Then they should be marked as Broken in the Setup file. Perhaps no clear pi0 peak is visible for these channels, which can happen for the the inner and/or outer elements of TAPS and CB due to shower leakage. Then the channels can be marked as NoCalibFill in the Setup file, which means that their gains are determined from an average of their neighbours. Or if you don't trust the neighbours and only want the default gain values to be used, mark the channel(s) as NoCalibUseDefault. Note that removing channels like this during the calibration process requires re-iterations of the calibrations.
-c CB_Energy
-c PID_Energy
-c TAPS_Energy
-c TAPSVeto_Energy
The calibration modules are CB_Energy_RelativeGains, PID_Energy_RelativeGains, TAPS_Energy_RelativeGains and TAPSVeto_Energy_RelativeGains.
This section provides examples of what a calibration looks like after one iteration, and what it should look like by the end. If your histograms appear more like the former than the later, it is likely you made a error in calibrating or need to do more iterations. To view your histograms and compare with those below, start ROOT and then type new TBrowser.
Note: Some of these histograms were created from single run files, rather than the data of the whole beam time. Your calibrations will likely not require nearly as many iterations (especially for time calibrations.)
As seen below, you want the data in the time histograms to line up with x=0. The PID graph should line up in diagonals.
Before:
(1 iteration)
After:
(3 iterations)
Before:
(1 iteration)
After:
(5 iterations)
Before:
(1 iteration)
After:
(7 iterations)
Before:
(1 iteration)
After:
(5 iterations)
Before:
After:
Before:
(1 iteration)
After:
(3 iterations)
Before:
(1 iteration)
(5 iterations)
(10 iterations)
(15 iterations)
After:
(20 iterations)
This section outlines how to create the setup file for the new beam time. For deeper and more code-focused instructions, see the page on How to write a Setup.
- Navigate to ant/src/expconfig/setups. Copy the last setup file and rename the copy to represent the current setup.
- Open up the newly created setup file using a text editor. Replace comments of the last setup with the current setup, comment out flags for Broken, BadTDC, and NoCalibFill elements (to be filled it later), and change the time stamp to that of the current beamtime’s start and end dates.
- In your ant directory, create a build folder and cd into it. Type
cmake ..and thenmaketo compile. Note: This must be done everytime the setup file is changed.
To be able to push and pull from GITLab with your new repository, you'll need to generate an SSH key, as outlined in this section.
-
In your home directory, type
ssh-keygen -t rsa. When asked to enter a file in which to save the key, simply hit the enter key (to save it in your home directory.) At the prompt, enter the password you wish to use. Note: Do NOT forget this password - there is no way of later changing or retrieving it. -
Use the command
cat .ssh/id_rsa.pubto print the key to the terminal. Copy this text, which starts with ssh-rsa. -
Sign into GIT Lab using your KPH account and password. Link: https://pbb-flora.kph.uni-mainz.de:8081/users/sign_in
-
Click Profile settings in the top right corner. In the menu on the left-hand side, select SSH Keys. Then click Add SSH Key in the top right.
-
Give your key a title and paste the key you copied from the terminal into the text box labeled Key. Click Add key.
- Create a directory outside of your ant directory. It can be called something like ant_database.
- Inside that directory you just created, initialize a GIT repository with the name of your current setup. This can be done with the following code:
git init <name of setup>
For example:
Note: Your GIT repository should have the same name as the setup you’ve created for your calibrations.
-
Sign into GIT Lab using your KPH account and password. Link: https://pbb-flora.kph.uni-mainz.de:8081/users/sign_in
-
Click the + sign in the top right-hand side of the page to create a new project. Note: Use the same name that you used to initialize your GIT repository and set it to be public.
- Back in the terminal, type the following two lines of commands for a global setup (one at a time):
git config −−global user.name ”<your first name> <your last name>”
git config −−global user.email ”<your KPH email>”
For example:
- In your GIT repository (in the terminal) create a README file, add it, and commit it, using the following three lines of commands (to be entered one at a time):
touch README.md
git add README.md
git commit -m ”<message>”
For example:
- Still in your GIT repository (in the terminal) type the following two commands (one at a time):
git remote add origin <ssh link given on GITlab page>
git push -u origin master
For example:
Note: Use the ssh link, not the https link.
-
Make a list of all run files, which usually start with CBTaggTAPS. Go through the E-Log (found at https://a2elog.kph.uni-mainz.de/Main+Logbook+2018/) and note down any files that are not good. This might include junk files, or runs that had errors and were less than ~10 minutes long. If the runs had errors but were ran for a longer period of time they may still having sufficient data to be useful.
-
Create a text file called something like RunPaths.txt containing a list of the full paths to the good run files from the beamtime. For example:
- Create a file called Antrc. In this, list all the options you want to pass to Ant. This will be changed for each set of calibrations (see Order of Calibrations for the order in which calibrations should be done.) For example:
Note: -s Setup_2018_05 doesn't need to be added if the time stamp was properly changed in the setup file; however, it is added here as a precaution to ensure that Ant knows which setup to use.
- In the same directory as your list of run files and Antrc file, submit the jobs to the farm using the following commands:
AntSubmit −−tag ”<tag>” --clean −−filelist <name of run path file list>.txt
For example:
Note: You can run AntSubmit without using the tag command; AntSubmit will just tag your files with ’Ant’ instead of whatever string you would’ve passed it.
After submitting jobs to the farm, you can see the status of your job and others by typing qstat into the terminal. If you’d like to pick out yours specifically rather than everyone’s, you can use the following command:
qstat | grep <your username>
Rather than listing every output root file, one can use an asterisk (*) to list multiple files at once. An asterisk before text will include every file ending in the text followed by the asterisk, and an asterisk after text will include every file beginning with that text. For example:
-
*.rootwill list every file that ends with .root -
calib*will list every file beginning with calib
To speed up the process of fitting histograms in the GUI after running Ant-calib (rather than having to click through every single one), try the following:
- Check the box near the top that says AutoContinue
- Uncheck the box in the upper right corner the says Show each fit
- Increase the Chi2/NDF limit to something more appropriate for the run (closer to the value output in the terminal)
- Check the box that says Ignore prev fit params
If you find that you've accidentally submitted the wrong job to the farm (i.e. you forgot to change your Antrc, or forgot to do a certain type of calibration before another) you can kill the jobs rather than waiting for them to finish by using the following command:
AntSubmit --tag "<tag used when submitting the jobs>" --killrunning
If your some jobs seem to be stuck in the farm and not processing (i.e. the time stamp shown with qstat isn't changing), use the following command to try them again without having to re-do the entire set of run files:
AntSubmit --filelist <name of run path file list>.txt --resubmit
If your data won't fit itself, try the following:
- Hit SetDefaults(or the d key) and then Fit(or the f key). Note: you may have to try this a couple times for it to work.
- Align the blue x0 line with the peak of the data. Then hit Fit(or the f key).
- Slide the max and min lines to be closer to the peak of the data. Then hit Fit(or the f key).
- Scale the graph so that you can only see the peak in the data. Then hit SetDefaults(or the d key) and then Fit(or the f key).