Remove ACC from Experimenter timesheets #28
Description
We've been given green light to remove ACC from experimenter timesheets from Physics (Doug) and Accelerator (Eduard) leadership. This will simplify time accounting, especially the end-of-shift cross checks.
It is probably the biggest single problem with machine time accounting as ACC is an Accelerator Division activity in which Halls are officially OFF. The Crew Chief's Accelerator timesheet records ACC, so we don't need the experimenters to do so as well. Since Experimenter timesheets have both ACC and OFF, and ACC is a subset of OFF, special care must be taken to properly record it. Further, the Experiment Beam Time (ER, PCC, UED, OFF) must also be set correctly during ACC, probably to ER (this isn't currently happening). Plus, if a hall is not scheduled to be on, such as most recently Hall C, the measured values still read ACC. Finally, the Crew Chief records time for halls as well, but they only record OFF per hall, not both OFF and ACC. This makes reconciling Crew Chief vs Experimenter seemingly more complicated than necessary.
We recently deployed a change to the test environment that replaces the Crew Chief measure of hall OFF on Experimenter timesheets with a new measure of hall Off that subtracts ACC (#26). This fixed the Accelerator Beam Time (left side of Experimenter timesheet), but then broke the Experiment Beam Time (right side of Experimenter timesheet) because OFF is shared between left and right sides and the time removed from OFF to accommodate ACC must be accounted for on the right as well by adding to either ER, PCC, or UED, which isn't currently configured and is a seemingly unnecessary complication. This change has not been deployed to production and probably needs to be rolled back unless we are going to try to capture bonus ABU automatically (see below).
This change will cascade to many other places besides the Experimenter timesheet that also need to be updated including:
- IOC code that measures BTA (needs to remove ACC, and replace with OFF and maybe even bonus ABU)
- Reports (PD Daily 8:00 AM and Wed. 1:30 BTA, especially the Hall Availability Report)
- The user guide and guidance docs will need to be updated
One obvious question is why was ACC included in the first place? The original requirements were written down decades ago and a scanned copy is attached to BTM here:
https://github.com/user-attachments/files/21988943/definitions.pdf
So the simplest answer is because those were the requirements and there is a ton of infrastructure and existing tools that assume those metrics. A better answer is probably because Experimenters explicitly request and have a scheduled allowance for ACC so it's on their minds. However, they're conspicously ignoring the fact there are other accelerator activities as well, such as beam studies, restore, and accelerator internal down that also affect halls similarly to ACC. During Accelerator activities, halls recording ACC, but not the others, and recording anything other than OFF is a complication and fragile as it leads to many errors, meaning we don't really trust the Experimenter numbers (The 4 experimenter hall measures of ACC are error prone and often disagree with each other, and also the Crew Chief). For example, since Experimenters are partially recording accelerator activities, and notably ignoring the distinction between OFF vs Failure (Internal Accelerator Down), there is no way for Experimenters to record that the accelerator was in ACC and something broke. They can either ignore accelerator failures and continue to report ACC (which is wrong), or report OFF (which is correct), but what usually happens is experimenters record this as BNA (which is wrong). It's an unnecessary complication and an unreasonable expectation placed on Experimenter timekeepers to keep this straight.
More importantly than why was it done originally; the real question is what are the ramifications of removal? One real issue to be aware of is Bonus ABU. Both accelerator division and physics division appreciate seeing opportunistic beam (or Bonus ABU), that is beam that can be used by a hall during accelerator activities, and there is concern that measuring OFF during ACC will stifle that. It's not clear at the moment that this change will negatively affect this as measuring ACC isn't equal to measuring bonus ABU (but a forced OFF is different than smart IOC code and it's not clear how that's handled at the moment), and it's useful to remember measured values are just suggestions and Experimenter timekeepers are to report what actually happened, not what the measured suggestions say. It's reasonable to suggest during accelerator activities that the hall is OFF, but if beam happens to be usable it should be reported as ABU. It is possible the IOC code that automates measurements could try to capture this. This code would need to update the right side of the experimenter timesheet as well however (Experiment Beam time) with ER during this bonus time. Plus the OFF would also need to be a special Experimenter one, not the Crew Chief one. If this is desired we should probably keep the change made in #26.