[Requirement] Masking tools

[celinedurniak]

Executive summary

Provide masks for different physical quantities: wavelength, 2 theta, intensity and detectors

Context and background knowledge

These masks are first to be used in the Powder Diffraction workflow for DREAM.
Different physical parameters could be used in order to mask:

• Faulty detectors
• Contribution from SE or instrument component in the detected signal
• Debye-Scherrer rings
• Too-high, too low or uneven distribution of intensity

Follow-up:

• adapt implementation with DREAM NeXus file
• use widget to determine masks (optional and for some physical quantities only)

Inputs

• For masks in 2 theta, wavelength: DataArray containing the quantity to be masked as coordinate

• For masks of detectors: metadata about detectors should be stored (positions, other characteristics like sumo, segment, sector…)

Methodology

• For the detectors, the masks could be specified either by the spatial coordinates of the voxels to mask or by sub-elements (i.e., sumo, sector, wire, strip). It could be a set like bank (hr, sans, endcap fwd, endcap bwd, mantle), sector (for SANS and HR) or sumo (for endcaps), module, segment, wire(s) and or strip(s), counter

Outputs

Data array with the same structure as input and with added mask.

Which interfaces are required?

Integrated into reduction workflow, Python module / function

Test cases

• Mask in two theta, intensity: data from D16 @ ILL link1 or DREAM csv files below
• Detector mask, wavelength: DREAM csv data (any csv files in link2)

Examples:

• mask in 2 theta

• mask of pixels (2d detectors for D16 @ ILL)

Comments

Note: The file from D16 can be used in the online documentation.

[nvaytet]

So masking should be part of a workflow.
Would you expect the masks to be different for each run/sample during an experiment or can you use a mask that was saved to a file?
I'm trying to figure out how this would fit in a pipeline?

Would you have a set of parameters at the top of your notebook like a list of wavelength or two_theta ranges that would defined what to mask, or do you need to have an interactive tool where you can draw masks?
I guess maybe a mixture of the 2?

The former could be something like (API to be discussed):

params[TwothetaMasks] = [sc.array(dims=['two_theta'], values=[53, 55], units='deg),
                         sc.array(dims=['two_theta'], values=[80, 84], units='deg)]

params[WavelengthMasks] = [sc.array(dims=['wavelength'], values=[3.2, 3.7], units='angstrom)]

An interactive tool could be used to save masks to file, and then we could load them like we do with the SANS workflows (we don't have to use xml format, we can use something else).

[nvaytet]

Another question: the lines in the two_theta plots are curved. Will there be some curved lines that we also want to mask or will the unwanted signal not be curved?
If so, is it possible to convert the data into a coordinate (maybe d-spacing?) where the lines are straight? If we can delay the masking until then, it will be much easier to accurately mask features either drawing rectangles or using a range of values as in the example code snippet above.

[celinedurniak]

About the 2nd question: In principle, the ranges of two theta to mask could be linear or curved. For this first implementation, we could focus on linear masking only and split this requirement into 2 parts with the mask in 2theta to be implemented later. Let me know what you think.

About the 1st question, "Would you expect the masks to be different for each run/sample during an experiment or can you use a mask that was saved to a file?"
It is a mixture of both (as you guessed):

• for example with the polariser in place, masking in wavelength will have to be applied to all runs corresponding to this setting (cycle, several beamtimes...)
• masking in 2theta could be different from sample to sample (for example, different textures)
• for the detectors, some masking should be in the NeXus files if issues with some detectors are picked up by NICOS. But additional masking should be implemented during reduction.
• masking in intensity could be detector- or sample-dependent

[nvaytet]

Looking at the workflow in https://scipp.github.io/essdiffraction/examples/POWGEN_data_reduction.html#Compute-final-result, we are converting the raw data directly to d-spacing.

Since you would like to have masks for detector pixels, wavelength ranges, and 2theta ranges, is it correct that we should add intermediate steps in the workflow that converts the data to the relevant coordinates?

What I mean is we could:

1. Mask pixels after the data has been normalized to proton charge: have a function that takes in NormalizedByProtonCharge[SampleRun], adds pixel masks, and then returns e.g. PixelMaskedData[SampleRun].

2. Mask one or more wavelength ranges: convert PixelMaskedData[SampleRun] to WavelengthData[SampleRun]. Then have a function that takes in WavelengthData[SampleRun] and returns WavelengthMaskedData[SampleRun].

3. Mask in 2theta: convert WavelengthMaskedData[SampleRun] to TwoThetaData[SampleRun]. Then have a function that takes in TwoThetaData[SampleRun] and returns MaskedTwoThetaData[SampleRun].

4. Finally, change to_dspacing_with_calibration to accept MaskedTwoThetaData[SampleRun] as input instead of NormalizedByProtonCharge[SampleRun].

Does that sounds like a good plan?

Question: would you also need to mask in d-spacing? i.e. add a MaskedDspacingData[SampleRun]? If not immediately required, I will leave it out for now. We can always add it later.

[celinedurniak]

@nvaytet Your plan is good. Here are a few comments

The conversion to $\lambda$ and $2\theta$ should be added to a more complete version of the workflow anyway. Their presence of a specific mask also depends on the type of experiments: for example, powder diffraction with known texture, parasitic scattering by some SEs….
These masks will also be essential tools for cold and hot commissioning of the instrument.

For the workflow of POWGEN

1. Masking pixels could be also done at an earlier stapge in the reduction. For example, after loading if some counts are too high.
2. OK
3. OK
4. OK for the POWGEN workflow. But for a more general approach, can’t we have both inputs?

A mask in d-spacing would also be required. But it can be left out of this first implementation after having tested the other types of masks.

[nvaytet]

Thanks for the reply.
Another question: in one of your images, the mask in 2theta is a vertical band that spans all pixels. Could we apply masks on a 1d plot of intensity vs 2theta? (i.e. making a 1d mask)

[jl-wynen]

Alternatively, is the 'spectrum' dim in those pictures a stand-in for another dim? E.g., phi (azimuthal angle) or y?