fitsmgmt.table.fits_summary

fitsmgmt.table.fits_summary(
    inputs=None,
    extension=None,
    verify_fix=False,
    fname_option='relative',
    output=None,
    keywords=None,
    example_header=None,
    sort_by='file',
    sort_map=None,
    fullmatch=None,
    flags=0,
    querystr=None,
    negate_fullmatch=False,
    nonunique_keys=False,
    header_backend='auto',
    *,
    output_format=None,
    **kwargs,
)

Extract summary rows from FITS headers.

Parameters

inputs : glob pattern, list-like of path-like, list-like of ~astropy.nddata.CCDData, ~pandas.DataFrame convertible, optional. = None

The ~glob pattern for files (e.g., "2020*[012].fits") or list of files (each element must be path-like or ~astropy.nddata.CCDData). Although it is not a good idea, a mixed list of ~astropy.nddata.CCDData and paths to the files is also acceptable. If a ~pandas.DataFrame or convertible (especially ~astropy.table.Table) is given, it finds the "file" column and use it as the input files, make a summary table from the headers of those files. If inputs is None, any output is ignored and None is returned. Default: None.

extension : int, str, (str, int), optional. = None

The extension of FITS to be used. It can be given as integer (0-indexing) of the extension, EXTNAME (single str), or a tuple of str and int: (EXTNAME, EXTVER). If None (default), the first extension with data will be used. Default: None.

verify_fix : bool, optional. = False

Whether to do .verify('fix') to all FITS files to avoid VerifyError. It may take some time if turned on. Default is False.

fname_option : str {\'absolute\', \'relative\', \'name\'} = 'relative'

Whether to save full absolute/relative path or only the filename. Default: 'relative'.

output : str or path-like = None

Output summary file; an existing file is replaced. With output_format=None, .parq and .parquet use Parquet; all other suffixes (including .fits) use CSV. Default: None.

output_format : (csv, parquet) = "csv"

Explicit pandas output format, overriding the suffix. None keeps suffix-based selection.

keywords : list or str(\"*\") = None

The list of the keywords to extract (keywords should be in str). Default: None.

example_header : None or path-like = None

The path including the filename of the output summary text file. If specified, the header of the 0-th element of inputs will be extracted (if glob-pattern is given, the 0-th element is random, so be careful) and saved to example_header. Use None (default) to skip this. Default: None.

sort_by : str = 'file'

The column name to sort the results. It can be any element of keywords or 'file', which sorts the table by the file name. Default: 'file'.

sort_map : dict | None = None

A subset of key parameter in pandas.DataFrame.sort_values(). If a dict is given, then key = lambda x: x.map(sort_map) is passed into .sort_values(). Default: None.

fullmatch : dict = None

The {column: regex} style dict to be used for selecting rows by summarytab[column].str.fullmatch(regex, case=True). Default: None

negate_fullmatch : bool = False

Whether to negate the mask by fullmatch, in case the user does not want to think much about regex to negate it. Default: False.

flags : int = 0

Regex module flags, e.g. re.IGNORECASE. Default: 0

querystr : str = None

The query string used for summarytab.query(querystr). See ~pandas.DataFrame.query. Default: None.

nonunique_keys : bool = False

Whether to remove the keys that have only one unique value throughout ALL input objects. Even if they are unique, keys specified in keywords will not be removed. Default is False.

header_backend : (auto, astropy, fitsio) = "auto"

Header reader used for path-like FITS inputs when it is safe to avoid opening full HDULists. "auto" uses fitsio when installed and falls back to Astropy otherwise. "fitsio" raises if fitsio is not installed. Default: "auto".

****kwargs** : object = {}

The keyword arguments to be passed to ~astropy.io.fits.open.

Returns

summarytab : ~pandas.DataFrame

Summary table with one row per input FITS file.

Notes

I want to use ccdproc.ImageFileCollection instead of this, but it is about 4 times slower than my ~astroimred.fitsmgmt.table.fits_summary, so I cannot use it yet.

Examples

>>> from pathlib import Path
>>> import astroimred as air
>>> keys = ["OBS-TIME", "FILTER", "OBJECT"]
>>> # actually it is case-insensitive
>>> # The keywords you want to extract
>>> # (from the headers of FITS files)
>>> TOPPATH = Path(".", "observation_2018-01-01")
>>> # The toppath
>>> savepath = TOPPATH / "summary_20180101.csv"
>>> # list of all the fits files in TOPPATH/rawdata:
>>> summary = air.fits_summary(
>>>     TOPPATH/"rawdata/*.fits",
>>>     keywords=keys,
>>>     fname_option='name',
>>>     sort_by="DATE-OBS",
>>>     output=savepath
>>> )

Select all rows with OBJECT starts with “DA”:

>>> # fullmatch = {"OBJECT": "DA.*"}
Select all rows with ``OBJECT`` starts with "Ves", ``FILTER`` is "J", and
``EXPTIME`` is 2 or 3:

>>> # fullmatch = {"OBJECT": "Ves.*", "FILTER": "J"},
>>> # querystr="EXPTIME in [2, 3]"

Write a Parquet table explicitly:

>>> summary = air.fits_summary(
...     "rawdata/*.fits", keywords=["OBJECT", "EXPTIME"],
...     output="summary.parquet", output_format="parquet"
... )