FITS Batch Cutout Generator: Automating Data Workflows for the Era of Next-Generation Telescopes
| Speaker | Mfundo Mdwadube |
|---|---|
| Track | Data Science and Engineering |
| Type | Normal Talk |
Abstract
FITS Batch Cutout Generator is a high-performance, multi-threaded Python pipeline designed for automated, WCS-aligned image segmentation and source extraction in large-scale radio astronomy surveys. As next-generation telescopes like MeerKAT, an SKA precursor, and other high-resolution arrays reveal tens of thousands of sources in a single patch of sky, traditional analysis methods quickly become impractical due to memory exhaustion and manual bottlenecks. In my own research, the first MeerKAT observation of the AKARI Deep Field South (ADFS) region resulted in a catalogue of over 19,000 faint radio sources detected at 1.28 GHz, covering approximately 10 square degrees near the South Ecliptic Pole. Using dedicated cross-matching workflows, about 66% of these sources were successfully associated with optical and infrared counterparts, with redshift information obtained for nearly half, demonstrating the scale and complexity of modern astronomical data analysis.
This talk will demonstrate how the FITS Batch Cutout Generator meets these challenges by automating and scaling the extraction of thousands of precise, WCS-aligned image cutouts from massive FITS files. I’ll show how Python’s scientific ecosystem, leveraging parallel processing, efficient memory management, and reproducible analysis, enabled the transformation of hours of manual work into a robust, minutes-long workflow. Along the way, I’ll share lessons learned in designing modular, scalable scientific software and discuss how these principles and open-source tools can benefit anyone facing big data challenges, whether in astronomy or other data-intensive fields.








