# Building a product

This is the **archive-creation stream**, end to end: science arrays plus metadata and backplanes go in, one PDS4-shaped FITS product comes out.

The science arrays here are synthetic. The real stream reads them from a Cassini UVIS PDS product, and the metadata and backplanes arrive already computed from Showalter's pipeline. Nothing below changes when that happens: only the [Source](../reference/Source.md#cubegenpy.Source) is swapped. That is the whole point of the seam.


``` python
from pathlib import Path

import numpy as np
from astropy.io import fits

from cubegenpy import BuildConfig, CubeBuilder, SyntheticSource, read_product

OUT = Path("products")
```


# 1. The source

A [Source](../reference/Source.md#cubegenpy.Source) produces one [ProductInputs](../reference/ProductInputs.md#cubegenpy.ProductInputs), the single payload the rest of the pipeline moves around. Its field names match the writer's arguments one for one.


``` python
source = SyntheticSource()
inputs = source.load()

for name in ("raw_counts", "cal_factor", "wavelength", "cube"):
    value = getattr(inputs, name)
    shape = "None (the calibrator fills it)" if value is None else f"{value.shape} {value.dtype}"
    print(f"{name:12} {shape}")

print()
print("dims        ", inputs.dims)
print("geometry    ", {k: len(v) for k, v in inputs.geometry.items()})
```


    raw_counts   (16, 8, 4) uint16
    cal_factor   (16, 8) float32
    wavelength   (16,) float32
    cube         None (the calibrator fills it)

    dims         Dims(NX=16, NY=8, NZ=4, NT=3)
    geometry     {'SC_GEOM': 20, 'BODY_GEOM': 10, 'GENERAL_GEOM': 3, 'RING_GEOM': 12}


`cube` is the only field a source may leave empty. Everything else is input.

Note the conventions: `raw_counts` is **uint16** in the PDS3 sense, where 65535 is the null rather than a count, and `wavelength` is in **Angstrom** even though pyuvis hands back nanometres.


# 2. Configure the build

[BuildConfig](../reference/BuildConfig.md#cubegenpy.BuildConfig) carries the two knobs that matter: where the calibration factor comes from, and how much background to subtract first.

Here the synthetic source already carries a `CAL_FACTOR`, so `"stored"` reuses it verbatim. The archive stream will use `"pyuvis"` once the real reader lands.


``` python
config = BuildConfig(calibration="stored")
print(config.summary())
print("background mode:", config.background.mode)
```


    cubegenpy[cal=stored, bg=none, v1.0]
    background mode: none


# 3. Build


``` python
product = CubeBuilder(source, config=config).build(OUT)
print("fits :", product.fits_path)
print("label:", product.label_path)
```


    fits : products/EUV2013_047_09_33_59.fits
    label: products/EUV2013_047_09_33_59.xml


    WARNING: VerifyWarning: Card is too long, comment will be truncated. [astropy.io.fits.card]


# 4. What landed


``` python
with fits.open(product.fits_path) as hdul:
    hdul.info()
```


    Filename: products/EUV2013_047_09_33_59.fits
    No.    Name      Ver    Type      Cards   Dimensions   Format
      0  PRIMARY       1 PrimaryHDU      33   (16, 8, 4)   float32   
      1  RAW_COUNTS    1 ImageHDU        11   (16, 8, 4)   int16   
      2  CAL_FACTOR    1 ImageHDU         9   (16, 8)   float32   
      3  BACKGROUND    1 ImageHDU         9   (16, 8)   float32   
      4  SC_GEOM       1 BinTableHDU    102   2R x 25C   [12A, 12E, 12E, 12E, 12E, 12E, 12E, 12E, 12E, 12E, 12E, 12E, 12E, 12E, 12E, 12E, 12E, 12E, 12E, 12E, 12E, E, E, E, L]   
      5  BODY_GEOM     1 BinTableHDU     57   1R x 13C   [12A, 480E, 480E, 480E, 480E, 480E, 480E, 480E, 480E, 480E, 480E, 480L, 480L]   
      6  GENERAL_GEOM    1 BinTableHDU     21   1R x 3C   [12D, 480E, 480E]   
      7  RING_GEOM     1 BinTableHDU     63   1R x 14C   [480E, 480E, 480E, 480E, 480E, 480E, 480E, 480E, 480E, 480E, 480L, 480L, 480E, 480E]   
      8  KERNELS       1 TableHDU        15   3R x 2C   [A32, A4]   
      9  WAVELENGTH    1 ImageHDU         8   (16,)   float32   


``` python
with fits.open(product.fits_path) as hdul:
    header = hdul[0].header
    for keyword in ("PROD_ID", "MISSION", "INSTRUME", "CHANNEL", "TARGET", "OBS_UTC", "INT_TIME"):
        print(f"{keyword:9} = {header[keyword]!r}")
```


    PROD_ID   = 'EUV2013_047_09_33_59'
    MISSION   = 'Cassini'
    INSTRUME  = 'UVIS'
    CHANNEL   = 'EUV'
    TARGET    = 'TITAN'
    OBS_UTC   = '2013-02-16T09:34:00.519'
    INT_TIME  = 240.0


# 5. The calibration, in the open

The whole formula is one line, and it runs entirely in counts per second:

    calibrated = (raw_counts / int_time - background_offset) * cal_factor

Worth reproducing by hand once, so nothing about the written product is mysterious. Undefined calibration entries arrive as NaN and are zeroed on the way out, which is what `nan_to_num` below mirrors.


``` python
written = read_product(product.fits_path)

int_time = float(inputs.header["INT_TIME"])
counts_per_sec = inputs.raw_counts.astype(np.float64) / int_time
expected = np.nan_to_num(counts_per_sec * inputs.cal_factor[:, :, np.newaxis]).astype(np.float32)

print("shapes match :", written.cube.shape == expected.shape)
print("values match :", np.array_equal(written.cube, expected))
```


    shapes match : True
    values match : True


# What changes with real data

Only the first cell of section 1. [SyntheticSource()](../reference/SyntheticSource.md#cubegenpy.SyntheticSource) becomes a source that reads the science arrays from a PDS product and merges in the externally computed metadata and backplanes, and `calibration="stored"` becomes `"pyuvis"`. Sections 3 through 5 stay exactly as they are.