Python and pipelines: Python interfaces for ESO’s CPL & HDRL data reduction libraries

Anthony Horton, Anthony Heng, Aidan Farrell, Mrunmayi Deshpande, Brent Miszalski, Nuwanthika Fernando, Nuria Lorente, Ralf Palsa, Lars Lundin, Armin Gabasch

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

Abstract

The ESO Common Pipeline Library (CPL) and High Level Data Reduction Library (HDRL) together form a comprehensive, efficient and robust software toolkit for data reduction pipelines. They were developed in C for reasons of efficiency and speed, however, with the community’s preference towards Python for algorithm prototyping and data reduction, there is a need for access from Python. PyCPL and PyHDRL provide this, making it possible to run existing CPL data reduction recipes from Python as well as developing new recipes in Python. These new recipes are built using the PyCPL and PyHDRL libraries, which provide idiomatic Python interfaces to CPL and HDRL while allowing users to take advantage of the scientific Python ecosystem. PyCPL and PyHDRL are already being used to prototype recipes for the MAVIS instrument pipeline, and have been used to develop an extensible pipeline development framework. Here we describe their design, implementation and usage.

Original languageEnglish
Title of host publicationSoftware and Cyberinfrastructure for Astronomy VIII
EditorsJorge Ibsen, Gianluca Chiozzi
Place of PublicationBellingham, Washington
PublisherSPIE
Pages131013Q-1-131013Q-4
Number of pages4
ISBN (Electronic)9781510675261
ISBN (Print)9781510675254
DOIs
Publication statusPublished - 2024
EventSoftware and Cyberinfrastructure for Astronomy VIII 2024 - Yokohama, Japan
Duration: 16 Jun 202421 Jun 2024

Publication series

NameProceedings of SPIE
Volume13101
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X

Conference

ConferenceSoftware and Cyberinfrastructure for Astronomy VIII 2024
Country/TerritoryJapan
CityYokohama
Period16/06/2421/06/24

Keywords

  • CPL
  • data reduction pipeline
  • HDRL
  • pybind11
  • PyCPL
  • PyHDRL
  • Python bindings
  • scientific Python

Cite this