Transformers for scientific data: a pedagogical review for astronomers

The deep learning architecture associated with ChatGPT and related generative AI products is known as transformers. Initially applied to Natural Language Processing, transformers and the self-attention mechanism they exploit have gained widespread interest across the natural sciences. The goal of this pedagogical and informal review is to introduce transformers to scientists. Our pedagogical and informal review includes the mathematics underlying the attention mechanism, a description of the original transformer architecture, and a section on applications to time series and imaging data in astronomy. We include with a Frequently Asked Questions section for readers who are curious about generative AI and interested in getting started with transformers for their research problem.