Behaviour at reduced source rate (1 Hz / 7 Hz commissioning) depends on the event_time_zero convention #1352

During commissioning the ESS source may run at 1 Hz or 7 Hz instead of 14 Hz. This issue records how our workflows and services behave at those rates, and what still needs deciding.

The answer depends on how event_time_zero (ETZ) and event_time_offset (eto) are produced at reduced rate. We do not know which convention ESS uses:

• Scenario A: ETZ marks only ticks that have beam, so eto runs up to 143 ms (7 Hz) or 1 s (1 Hz).
• Scenario B: ETZ stays on every 14 Hz tick and eto wraps at 71.4 ms. This looks like pulse-skipping, except the source itself skips pulses instead of a chopper.

Summary: what to configure or fix

✅ works as is · ❓ unknown · ⚙️ configure in the dashboard · 🔧 code fix · 📡 needs a change outside livedata (EFU or device) · ⚠️ works with a limitation · ❌ not possible

f is the real source rate. Details and reasons are in the sections below.

In scenario A, every ⚙️ row also assumes the EFU passes eto beyond 71.43 ms. The TBL EFUs do not do that today; see the TBL section.

A setting that works for event data under every convention: LUT Pulse.frequency = 14 Hz with stride 14/f (auto-stride off).¹ It does not cover histogram-mode monitors in scenario A.

f must be 14/n Hz wherever the LUT is built for it; the analytical LUT fails to build for other rates.

¹ Relies on a per-batch guess of which 14 Hz tick is the real pulse; untested at commissioning count rates.

TBL

Questions for ECDC / timing

• At reduced rate, is there an ETZ (reference_time) on every 14 Hz tick, or only on ticks with beam?
• The TBL EFUs cap eto at 71.43 ms (see the TBL section). In scenario A, what happens to events that arrive later: are they dropped, or attributed to the next ETZ?
• The TBL He3 and Multiblade ETZ values are not on a 1/14 s grid (spacing 71.4–83 ms). Is that expected while there is no beam, or are these EFUs not yet locked to the timing system?
• Can we rely on the timing PVs on tn_data_general to tell the beam rate: TD-M:Ctrl-EVR-1:EvtF14HzCnt-I (14 Hz cycle counter), TD-M:Ctrl-EVR-1:EvtBPulseStCnt-I (beam-pulse-start counter), TD-M:Ctrl-EVR-1:DbufBPresent-II (beam present)? Their names suggest the 14 Hz cycle continues whether or not a cycle has beam, which points towards scenario B. They are in the TBL CODA file under /entry/instrument/source, but our TBL stream list does not include them, so livedata does not read them. Today the lookup-table (LUT) workflow's Pulse.frequency is a manual parameter that nothing checks.

TBL

TBL does not offer wavelength mode yet. All its detector views are logical views, which are TOA-only, and monitor_1 uses TOAOnlyMonitorDataParams. Adding it is tracked in #1354, and the wavelength rows of the TBL table show the expected behaviour once that is done. Until then, only the TOA range and what the EFUs and devices deliver matter at TBL. Each detector view feeds NICOS *_counts_total devices, and counts_total is the sum of the TOA histogram (#1242), so a TOA range that is too short also undercounts in NICOS.

Findings from the two most recent TBL CODA files (coda_tbl_999999_00024013.hdf, 2026-10-06, 12 min; coda_tbl_999999_00022026.hdf, 2026-09-22, 5 min). In both, the accelerator PVs under /entry/instrument/source are empty or zero (pulse_charge 0, no beam_present updates), so these runs had no beam. They cannot tell us which ETZ convention applies, but they do show what the readout does:

The bandwidth choppers bwc_1 and bwc_2 were parked (0 Hz) in the 2026-10-06 run.

Wavelength conversion

The LUT workflow has two settings that matter here: Pulse.frequency (default 14 Hz, wavelength_lut_workflow_specs.py:38) and the pulse stride. Which values are correct depends on the ETZ convention, not only on the source rate:

• Entering the real source rate is only correct in scenario A. In scenario B, eto never exceeds 71.4 ms. A LUT for a 143 ms or 1 s frame then gives NaN wavelengths to every neutron whose true arrival time falls after 71.4 ms, so those events are dropped silently. "The source runs at 7 Hz, so enter 7 Hz" is therefore the wrong fix if ETZ stays at 14 Hz.
• Leaving the default (14 Hz, stride 1) is correct in scenario B when the choppers are set up for 14 Hz. Each eto (mod 71.4 ms) still maps to a single pulse and chopper opening, so the missing pulses only cost counts. In scenario A the same setting drops every event with eto > 71.4 ms.
• 14 Hz with stride 14/f was correct under both conventions. essreduce's pulse_index unwrap places each event within the longer frame using ETZ. This could make the ECDC answer unnecessary for event data, with two caveats:
  • It needs ETZ per event, so it does not help histogram-mode monitors (see below).
  • It relies on a per-batch guess of which 14 Hz tick is the real pulse (PulseStrideOffset=None, a "fewest NaNs" heuristic). I have not tested how stable that guess is at commissioning count rates.

Consumers read the period and stride from coords that travel with the published LUT, so whatever the LUT workflow is set to applies everywhere. Auto-stride also picks up slow or pulse-skipping choppers. ETZ survives our chain (ToNXevent_data → group_event_data → projector), and wavelength conversion runs before projection, so the unwrap has what it needs. The analytical LUT only builds for rates of 14/n Hz; for example, 5 Hz raises "chopper is out of phase with the source".

These results come from the script below. tof ray-traces events with known true wavelengths from a 7 Hz or 1 Hz source. The events are labelled according to scenario A or B, then converted with LUTs from essreduce's analytical method, the same method our LUT workflow uses. The geometry is a toy one (choppers from ess.reduce.unwrap.fakes, detector at 60 m), so the size of the losses does not carry over to real instruments; only the pattern of which setting works where does.

Time-of-arrival (TOA) histograms and monitors

• The TOA edges for the detector view and monitors default to 0–71.43 ms (parameter_models.py:26). Events beyond that are excluded, as the parameter description states. The field has no upper limit.
• Scenario A, event-mode monitors and detectors: setting the upper edge to 142.86 ms or 1000 ms is enough. DREAM's _total_counts output hardcodes 0–71 ms (dream/factories.py:124).
• Scenario B: widening the edges changes nothing, because eto never exceeds 71.4 ms. With a pulse-skipping chopper setup, the TOA view therefore shows both sub-frames folded on top of each other. Only wavelength mode unfolds them.
• Histogram-mode monitors (da00): the device that produces the histogram defines the time axis. We only rebin it (monitor_workflow.py:103-109), so edges beyond the device's own range just add empty bins. The device has to be configured for the longer frame. Wavelength mode on these histograms has no ETZ, so essreduce cannot unwrap stride > 1. It works only in scenario A, with the LUT frequency set to the real rate.

Service level (no change needed)

• The rate-aware batcher estimates each stream's rate and snaps it to whole Hz, so 7 Hz and 1 Hz work, including one pulse per batch. Streams below 1 Hz are still delivered, just not gated per pulse.
• Adaptive batch lengths are rounded to multiples of 1/14 s (message_batcher.py:346). At 1 Hz, the escalated windows (1.43 s, 2.86 s, …) hold 1 or 2 pulses alternately, so "current" outputs can flicker by up to a factor of 2. This only happens under overload, which is unlikely at reduced rate.
• Nothing normalises per pulse. Window outputs are counts per batch window.
• eto is stored as int32 ns in ToNXevent_data, which holds up to 2.1 s, so rates down to about 0.5 Hz fit.
• The number of LUT time bins grows as 1/f (4,000 at 1 Hz with 250 µs bins). That is fine for the tables built here, but worth checking against the Kafka message-size limit for instruments whose flight-path range is wide.

To decide

• Do we recommend 14 Hz with stride 14/f for commissioning, which avoids depending on the ECDC answer for event data? Or do we wait for the answer and set the LUT to match? Histogram-mode monitors need the real-rate setting either way, and that only works in scenario A.
• Should we ingest the timing PVs listed above, so the dashboard can show the beam rate and catch a LUT Pulse.frequency that does not match it?
• Should we warn when a large fraction of events falls outside the LUT's time range? That is how scenario A fails silently when the frequency is left at 14 Hz.

"""Ray-trace a reduced-rate ESS source with `tof` to get events with known true
wavelengths, then convert them with lookup tables (LUTs) built by essreduce's
analytical method, as used by the livedata wavelength-LUT workflow.

Scenario A: event_time_zero (ETZ) only on ticks with beam; eto relative to that.
Scenario B: ETZ on every 14 Hz tick; eto wraps at 71.4 ms.
Truth is the simulated wavelength of each neutron.
"""
import numpy as np
import scipp as sc
import tof
from ess.reduce.unwrap import fakes
from ess.reduce.unwrap.lut import (
    SourceBounds, default_parameters, make_wavelength_lut_from_polygons,
)
from ess.reduce.unwrap.to_wavelength import _compute_wavelength_events
from scippneutron.tof import chopper_cascade

L = 60.0  # detector distance in m
SRC = fakes.source_position()
BOUNDS = default_parameters()[SourceBounds]
TICK = 1e9 / 14  # ns


def lut(choppers, hz, stride, npulses):
    """Like ess.reduce.unwrap.lut.compute_frame_sequence, but npulses is free."""
    period = 1.0 / sc.scalar(float(hz), unit='Hz')
    rot = (1.0 / period.to(unit='s')) / (stride * 2)
    chops = [
        chopper_cascade.Chopper(
            distance=sc.norm(ch.axle_position - SRC.to(unit=ch.axle_position.unit)),
            time_open=ch.time_offset_open(pulse_frequency=rot),
            time_close=ch.time_offset_close(pulse_frequency=rot),
        )
        for ch in choppers.values()
    ]
    frames = chopper_cascade.FrameSequence.from_source_pulse(
        time_min=BOUNDS.time[0], time_max=BOUNDS.time[1],
        wavelength_min=BOUNDS.wavelength[0], wavelength_max=BOUNDS.wavelength[1],
        pulse_period=period, npulses=npulses,
    ).chop(chops)
    return make_wavelength_lut_from_polygons(
        (sc.scalar(L - 0.5, unit='m'), sc.scalar(L + 0.5, unit='m')),
        sc.scalar(0.1, unit='m'), sc.scalar(250.0, unit='us'), period, stride, frames,
    )


def simulate(choppers, hz, npulses):
    tof_choppers = []
    for name, ch in choppers.items():
        c = tof.Chopper.from_diskchopper(ch, name=name)
        c.distance = sc.norm(ch.axle_position)
        tof_choppers.append(c)
    model = tof.Model(
        source=tof.Source(
            facility='ess', neutrons=50000, pulses=npulses,
            frequency=sc.scalar(float(hz), unit='Hz'), seed=1,
        ),
        choppers=tof_choppers,
        detectors=[tof.Detector(distance=sc.scalar(L, unit='m'), name='d')],
    )
    return model.run().detectors['d'].data


def events(sim, every, scenario):
    """Label events with ETZ/eto; the source fires on every `every`-th 14 Hz tick."""
    sel = ~sim.masks['blocked_by_others'].values
    toa = sim.coords['toa'].to(unit='ns').values[sel]
    if scenario == 'A':
        tick = np.floor(toa / (TICK * every)).astype(int) * every
    else:
        tick = np.floor(toa / TICK).astype(int)
    etz = 1_790_000_000 * 10**9 + np.round(tick * TICK).astype(np.int64)
    eto = (toa - np.round(tick * TICK)).astype(np.int64)
    ev = sc.DataArray(
        sc.ones(dims=['event'], shape=[toa.size], unit='counts'),
        coords={
            'event_time_offset': sc.array(dims=['event'], values=eto, unit='ns'),
            'event_time_zero': sc.datetimes(dims=['event'], values=etz, unit='ns'),
        },
    )
    begin = sc.zeros(dims=['detector'], shape=[1], dtype='int64', unit=None)
    da = sc.DataArray(sc.bins(begin=begin, dim='event', data=ev))
    return da, sim.coords['wavelength'].values[sel], eto.max() / 1e6


def report(title, choppers, every, npulses):
    real_hz = 14 / every
    sim = simulate(choppers, real_hz, npulses)
    tables = {
        '14 Hz, stride 1': lut(choppers, 14, 1, 1),
        f'{real_hz:g} Hz, stride 1': lut(choppers, real_hz, 1, 1),
        f'14 Hz, stride {every}': lut(choppers, 14, every, every),
    }
    ltotal = sc.full(dims=['detector'], shape=[1], value=L, unit='m')
    for scenario in 'AB':
        da, truth, eto_max = events(sim, every, scenario)
        print(f'{title}, scenario {scenario} ({truth.size} events, max eto {eto_max:.0f} ms)')
        for name, table in tables.items():
            out = _compute_wavelength_events(
                da=da, lookup=table, ltotal=ltotal, pulse_stride_offset=None
            )
            wav = out.bins.coords['wavelength'].bins.constituents['data'].values
            nan = np.isnan(wav)
            gross = (np.abs(wav - truth)[~nan] > 1.0).sum() / wav.size
            print(f'  LUT {name:16s} NaN (dropped) {nan.mean():6.1%}   off by >1 Å {gross:5.1%}')


report('7 Hz source, 14 Hz chopper', fakes.psc_choppers(), 2, 8)
report('7 Hz source, pulse-skipping choppers', fakes.pulse_skipping_choppers(), 2, 8)
report('1 Hz source, 14 Hz chopper', fakes.psc_choppers(), 14, 4)