.. _calwebb_spec3: calwebb_spec3: Stage 3 Spectroscopic Processing =============================================== :Class: `jwst.pipeline.Spec3Pipeline` :Alias: calwebb_spec3 Stage 3 processing for spectroscopic observations is intended for combining the calibrated data from multiple exposures (e.g. a dither/nod pattern) into a single combined 2D or 3D spectral product and a combined 1D spectrum. Before being combined, the exposures may receive additional corrections for the purpose of background matching and subtraction, as well as outlier rejection. The steps applied by the ``calwebb_spec3`` pipeline are shown below. This pipeline is intended for non-TSO spectra only. TSO spectral data should be processed using the :ref:`calwebb_tso3 ` pipeline. .. |c| unicode:: U+2713 .. checkmark +-------------------------------------------------------------+-----+-----+-----+-----+-----+------+------+--------+ | Instrument/Mode | NIRSpec | MIRI | NIRISS | NIRCam | +-------------------------------------------------------------+-----+-----+-----+-----+-----+------+------+--------+ | Step | FS | MOS | IFU | FS | MRS | SOSS | WFSS | WFSS | +=============================================================+=====+=====+=====+=====+=====+======+======+========+ | :ref:`assign_mtwcs `\ :sup:`1` | |c| | |c| | |c| | |c| | |c| | |c| | |c| | |c| | +-------------------------------------------------------------+-----+-----+-----+-----+-----+------+------+--------+ | :ref:`master_background `\ :sup:`2` | |c| | | |c| | |c| | |c| | | | | +-------------------------------------------------------------+-----+-----+-----+-----+-----+------+------+--------+ | :ref:`exp_to_source ` | |c| | |c| | | | | | |c| | |c| | +-------------------------------------------------------------+-----+-----+-----+-----+-----+------+------+--------+ | :ref:`outlier_detection ` | |c| | |c| | |c| | |c| | |c| | | | | +-------------------------------------------------------------+-----+-----+-----+-----+-----+------+------+--------+ | :ref:`pixel_replace ` | |c| | |c| | |c| | |c| | |c| | | |c| | |c| | +-------------------------------------------------------------+-----+-----+-----+-----+-----+------+------+--------+ | :ref:`resample_spec ` | |c| | |c| | | |c| | | | | | +-------------------------------------------------------------+-----+-----+-----+-----+-----+------+------+--------+ | :ref:`cube_build ` | | | |c| | | |c| | | | | +-------------------------------------------------------------+-----+-----+-----+-----+-----+------+------+--------+ | :ref:`extract_1d ` | |c| | |c| | |c| | |c| | |c| | |c| | |c| | |c| | +-------------------------------------------------------------+-----+-----+-----+-----+-----+------+------+--------+ | :ref:`spectral_leak ` | | | | | |c| | | | | +-------------------------------------------------------------+-----+-----+-----+-----+-----+------+------+--------+ | :ref:`combine_1d ` | | | | | | |c| | |c| | |c| | +-------------------------------------------------------------+-----+-----+-----+-----+-----+------+------+--------+ :sup:`1`\ The :ref:`assign_mtwcs ` step is only applied to observations of a moving target (TARGTYPE='moving'). :sup:`2`\ The master background subtraction step is applied to NIRSpec MOS exposures in the :ref:`calwebb_spec2 ` pipeline. WFSS and SOSS Processing ------------------------ Notice that NIRCam and NIRISS WFSS, as well as NIRISS SOSS data, receive only minimal processing by ``calwebb_spec3``. WFSS 2D input data are reorganized into source-based products by the :ref:`exp_to_source ` step (see below), have 1D extracted spectra produced for each source, and then the 1D spectra for each source are combined into a final 1D spectrum. NIRISS SOSS inputs do not go through the :ref:`exp_to_source ` step, because they contain data for a single source. Hence the only processing that they receive is to extract a 1D spectrum from each input and then combine those spectra into a final 1D spectrum. This type of processing is intended only for NIRISS SOSS exposures that are not obtained in TSO mode. TSO mode NIRISS SOSS exposures should be processed with the :ref:`calwebb_tso3 ` pipeline. Combined NIRSpec MOS and FS Exposures ------------------------------------- It is possible to observe NIRSpec fixed slit targets alongside MSA slitlets in NIRSpec MOS exposures. In this case, the input files produced by the :ref:`calwebb_spec2 ` pipeline contain both MOS and FS data. Any :ref:`master_background ` corrections applied in this pipeline will operate on the fixed slits only, since background corrections for MOS slitlets should be applied in the :ref:`calwebb_spec2 ` pipeline. After the sources are separated in the :ref:`exp_to_source ` step, each data product will contain only a fixed slit or a MOS target, so all further processing follows the standard flow for each exposure type. Arguments --------- The ``calwebb_spec3`` pipeline does not have any optional arguments. Inputs ------ 2D calibrated data ^^^^^^^^^^^^^^^^^^ :Data model: `~jwst.datamodels.ImageModel`, `~jwst.datamodels.IFUImageModel`, `~jwst.datamodels.SlitModel`, or `~jwst.datamodels.MultiSlitModel` :File suffix: _cal The inputs to ``calwebb_spec3`` should be in the form of an ASN file that lists the multiple exposures to be processed into combined output products. The individual exposures should be calibrated the ("_cal") products from :ref:`calwebb_spec2 ` processing. The member list for each product in the ASN file can also contain exposures of dedicated background targets, which are intended for use in the :ref:`master_background ` step. These input exposures must be the "x1d" products (extracted 1-D spectra) of the background target(s) and are usually the "x1d" files produced by the :ref:`calwebb_spec2 ` pipeline. They must be listed in the ASN file with "exptype" values of "background" in order to be correctly identified as background exposures. See the :ref:`master_background ` for more details. Outputs ------- Source-based calibrated data ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ :Data model: `~jwst.datamodels.MultiExposureModel` :File suffix: _cal For NIRSpec fixed-slit and NIRSpec MOS, which have a defined set of slits or sources, the data from the input calibrated exposures is reorganized by the :ref:`exp_to_source ` step so that all of the instances of data for a particular source/slit are contained in a single product. These are referred to as "source-based" products, as opposed to the input exposure-based products. The source-based collections of data are saved in intermediate files, one per source/slit. The root names of the source-based files contain the source ID as an identifier and use the same "_cal" suffix as the input calibrated exposure files. An example source-based file name is "jw00042-o001_s00000002_niriss_gr150r_f150w_cal.fits", where "s00000002" is the source id. NIRCam and NIRISS WFSS observations also have a defined set of sources, and those data still go through `exp_to_source`, but the reorganized data products are not saved as intermediate files because there may be hundreds or thousands of sources in a single exposure. The reorganized sets of data are sent to subsequent steps to process and combine all the data for one source at a time. CR-flagged exposures ^^^^^^^^^^^^^^^^^^^^ :Data model: `~jwst.datamodels.ImageModel` :File suffix: _crf If the :ref:`outlier_detection ` step is applied, a new version of each input calibrated exposure is created, in which the DQ array has been updated to flag pixels detected as outliers. These files use the "_crf" (CR-Flagged) product type suffix and also includes the association candidate ID as a new field in the original product root name, e.g. "jw96090001001_03101_00001_nrs2_o001_crf.fits." 2D resampled and combined spectral data ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ :Data model: `~jwst.datamodels.SlitModel` :File suffix: _s2d When processing non-IFU modes, a resampled/rectified 2D product of type "_s2d" is created containing the rectified and combined data for a given slit/source, which is the output of the :ref:`resample_spec ` step. 3D resampled and combined spectral data ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ :Data model: `~jwst.datamodels.IFUCubeModel` :File suffix: _s3d When processing IFU exposures, a resampled and combined 3D IFU cube product created by the :ref:`cube_build ` step is saved as an "_s3d" file. 1D extracted spectral data ^^^^^^^^^^^^^^^^^^^^^^^^^^ :Data model: `~jwst.datamodels.MultiSpecModel` :File suffix: _x1d All types of inputs result in a 1D extracted spectral data product, which is saved as a "_x1d" file, and is normally the result of performing the :ref:`extract_1d ` step on the combined "_s2d" or "_s3d" product. For NIRCam and NIRISS WFSS, as well as NIRISS SOSS data, the :ref:`extract_1d ` is performed on the individual unresampled 2D cutout images, resulting in multiple 1-D spectra per source in a "_x1d" product. Those spectra are combined using the subsequent :ref:`combine_1d ` step (see below). For NIRCam and NIRISS WFSS, the output ``_x1d`` product holds the spectra from all the sources in a single product. The data model is `~jwst.datamodels.WFSSMultiSpecModel`, and has one extension per exposure per spectral order, with each extension containing a binary table of all the spectra (and associated metadata) for all sources extracted from that exposure and spectral order. See :ref:`extract_1d ` for more details. 1D combined spectral data ^^^^^^^^^^^^^^^^^^^^^^^^^ :Data model: `~jwst.datamodels.CombinedSpecModel` :File suffix: _c1d For NIRCam and NIRISS WFSS, as well as NIRISS SOSS data, the :ref:`combine_1d ` combines the multiple 1-D spectra for a given source into a final spectrum, which is saved in a "_c1d" product. For NIRCam and NIRISS WFSS, the output ``_c1d`` product holds the spectra from all the sources in a single product. The data model is `~jwst.datamodels.WFSSMultiCombinedSpecModel`, and has a single binary table per spectral order containing the exposure-combined spectra for all sources extracted from that exposure. The data type is similar to that of the ``_x1d`` product, but with just one data extension per extracted spectral order (because the exposures have been combined) and with a more limited set of data and metadata columns.