Towards fully automated initial calibration — and sharing our downstream experience

[TianyiMa96]

Hi pyFAI team,

I am Tianyi Ma, a PhD student at the Institute of Chemistry, Chinese Academy of Sciences (ICCAS). I'd like to raise a topic that we think could significantly broaden pyFAI's user base, and share some context from our downstream projects.

Fully automated initial calibration: a missing piece

pyFAI already provides excellent re-calibration tools (e.g. pyFAI-recalib, SingleGeometry.extract_cp, RingExtraction) that can run without human intervention given a prior approximate geometry. However, the initial calibration — going from an uncharacterized diffraction image to a calibrated geometry — still requires manual ring selection through pyFAI-calib or pyFAI-calib2.

For experienced synchrotron users who routinely develop their own pipelines this is manageable, but for newcomers and researchers whose primary focus is not X-ray diffraction, a standardized, high-accuracy calibration method that requires only the diffraction image as input (no manual intervention) would be a substantial step forward.

Our experience with AI-assisted workflows (github.com/TianyiMa96/diffraction-scatter-skills) showed us that small models can handle batch, unattended integration tasks effectively. But calibration remains harder to automate — even capable vision models (e.g. Gemini, GPT) struggle with identifying and assigning the correct calibrant rings. We have experimented with genetic algorithms combined with traditional global center-search methods, but so far the success rate has been low and the computation time prohibitive. We would be very interested to learn whether the pyFAI team has thoughts or ongoing efforts in this direction, or if this is something the community could work on together.

Our downstream projects

For context, we maintain two open-source projects that rely on pyFAI as the computational backbone:

XFAIS (github.com/ICCAS-EPlab-PMP/XFAIS) is a graphical application for X-ray scattering and diffraction data analysis, built on Electron + Vue 3 with a Python backend. The "FAI" in the name reflects its dependence on pyFAI, which together with fabio serves as the core integration engine. The application was developed through AI-agent full-stack development and targets Windows users in laboratory settings. It also includes practical utilities such as .D calibrant file generation and quick pixel-to-PONI conversion. We will continue wrapping pyFAI's core capabilities into GUI workflows to lower the barrier for newcomers, and we hope to promote pyFAI combined with Jupyter-based workflows to a broader community of synchrotron and X-ray scattering/diffraction users in China.

PolymCrystIndex (github.com/ICCAS-EPlab-PMP/PolymCrystIndex) is a fiber diffraction pattern indexing platform, published in J. Appl. Cryst. (2025, Ma et al.). pyFAI powers the peak extraction pipeline — its high-speed integration reduced our processing time by roughly 4 hours compared to our previous approach.

Through routine use we regularly encounter edge cases and usability issues — our recent calibrant error handling fix (#2887) is one example, and we are happy to keep reporting these as tested PRs.

Acknowledgements

We sincerely thank ESRF and the pyFAI/silx team for maintaining this foundational work. Our research would not be feasible without it, and we would welcome any opportunity for closer collaboration — whether as downstream test cases, feedback from laboratory practice, or otherwise.

Tianyi Ma

[kif]

Hi TianyiMa,
I have been investigating very much this direction when I started pyFAI, around 2010 but got less interested since Valentin (@vallsv ) did a superbe job in calib2. That said, there are several issues which could be separated:

• image recognition algorithm are targeted at "visible images", transforming high dynamic range images from diffraction into 8bit-linear data in a reliable way would help traditional computer vision tools to work efficiently.
• CNN are today very relevant for this kind of image analysis, especially that a complete framework exists to generate calibration images.
  But I am currently not working on this kind of tools. It remains a good subject. I believe the low-hanging fruits have already been picked in the field and this will require harder work but maybe I am wrong ...

Cheers,
Jerome