Tails: chasing comets with the Zwicky Transient Facility and Deep Learning

Tails is a deep-learning framework for the identification and localization of comets in the image data of the Zwicky Transient Facility (ZTF), a robotic optical sky survey at the Palomar Observatory in California, USA.

Tails uses a custom EfficientDet-based architecture and is thus capable of finding comets in single images in near real time, rather than requiring multiple epochs as with traditional methods. In production, we have observed 99% recall, <0.01% false positive rate, and 1-2 pixel root mean square error in the predicted position.

Tails enabled the first AI-assisted discovery of a comet - C/2020 T2.

Comet discoveries by Tails

• C/2020 T2
• C/2022 P2
• C/2022 P3

Install and run Tails

Clone the repository:

git clone https://github.com/dmitryduev/tails.git && cd tails

Before proceeding, you may want to create/activate a virtual environment, for example:

python -m venv tails-env
source tails-env/bin/activate

Install swarp. For example, with conda:

conda install -c conda-forge astromatic-swarp

Install Tails:

python setup.py install

Fetch pre-trained models:

mkdir models
gsutil -m cp -r -n gs://tails-models/* models/

Run Tails on a (publicly accessible) ZTF observation of 2I/Borisov:

cd scripts
python run_tails.py \
  --config=../config.defaults.yaml \
  --checkpoint=../models/tails-20210107/tails \
  --score_threshold=0.5 \
  --cleanup=none \
  --single_image=ztf_20191014495961_000570_zr_c05_o_q3

Check out the runs/20191014 directory for the output:

• A csv file with the detection metadata
• A 256x256 pix cutout image triplet (epochal, reference, and difference) containing the detection: an .npy file and a .png visualization:

ZTF sentinel service

A containerized service that monitors Kowalski/IRSA for new ZTF data, executes Tails on them, and optionally posts the identified candidates to Fritz.

Requires Kowalski and IRSA accounts, see the config file. Additionally, a Fritz account is required if sentinel.app.post_to_fritz is set to true.

Fetch models from GCP:

./sentinel.py fetch-models

Spin up:

./sentinel.py up

Shut down:

./sentinel.py down

---

The Sentinel:

• Managed with the sentinel.py utility.
• Implemented as a containerized service, orchestrated with docker-compose.
• Monitors the ZTF_ops collection on Kowalski for new ZTF data (Twilight only by default).
• Uses dask.distributed to process individual ZTF image frames (ccd-quads). Each worker is initialized with a TailsWorker instance that maintains a Fritz connection and preloads Tails. The candidate comet detections, if any, are posted to Fritz together with auto-annotations (cross-matches from the MPC and SkyBot) and auxiliary data.
  • By default, spins up a dask.distributed.LocalCluster, whose parameters are defined in config.yaml, and a sentinel process, both managed with supervisor inside the tails_sentinel_1 Docker container. To use an external dask.distributed cluster, the user needs to simply provide its host and the scheduler port and remove the dask-cluster supervisor program in the config.

On Fritz, the posted results look like the following:

Candidates page	Source page