astro.fits
Class IcatPrepare

java.lang.Object
  util.PropertyContainer
      util.PropertyResources
          util.PropertyBundles
              astro.fits.Icat
                  astro.fits.IcatPrepare

All Implemented Interfaces:

Cloneable, ExitCleaning, Initializable, LocalizedSupplying, PropertySupplying, ResourceSupplying

public class IcatPrepare
extends Icat
implements ExitCleaning

Tool to prepare Wifsip fits for reduction with the Icat pipeline writtn by P. Colome. It takes a list of science fits files, a list of bias flats and, possibly a list of dark files that should be used together in the reductional process. The appropriate list of calibration and science files has to be provided by an other application, most easiliy with a command-line interface. This class then does the following:

• The WiFSIP data has four extension, which are split into four differnt fits objects in the method split(nom.tam.fits.Fits, int, boolean)
• According to KEY_KEEPPRIMARY, KEY_KEEPSECONDARY, the keys from the different sections are merged. Keys not found in any of these lists are copied to the resulting header, if in the primary extension, or silently ignored, if in the amplifier extensions.
• Keys not present are copied from #KEY_COPY. There, fixed values are specified at run-time.
• Keys, whose values have to be calculated, are fouble in #KEY_GENERATE.
• In the input directory of ICAT, Icat.KEY_ICATIN, four different subdirectories are generated, named after the four quadrants, i.e. q0 to q3.
• In each of these q# directories, and 'out' directory is created.
• In each input directory, one of the quadrants is stored as an input file name according to the ICAT convention. Science images:
  yyyyMMdd'_pppss'TTTTT###F'R_imr.fits'
  Calibration images:
  yyyyMMdd'_CAL'TTTTT##'_c'EXT
  where yyyyMMdd is the observing date, TTTTT is five digits of the observing time times ten, i.e. an observing time of 1.0 would show up as 00010, an observing time of 1.125 00011. ## or ### is the running number, for science images it has three digits, for calibrations only two!. All calibration types have their own running number, i.e. the first flat always has 01, regardless of the number of bias/darks. F is the filter name, like 'V'. Care should be taken with strange filter names, Sloan is therefor up, gp, etc. The EXT extension on the calibration image is determined by its type, 'bs.fits' for bias, 'th.fits' for darks, and 't'F'.fits' for flats, where F again is the filter.
• Depending on the availability of dark frames, the calib.par file located at Icat.KEY_CALIBPAR is either copied from Icat.KEY_CALIBPARDARK or Icat.KEY_CALIBPARNODARK.
• The analysis parameter file Icat.KEY_ANALYSISPAR is set such that no astronomical reduction beside calibration is done ($ana=0;). This is done by copying Icat.KEY_ANALYSISPARNO to Icat.KEY_ANALYSISPAR.
• Because all images have the same names in the different quadrant folders, the img.dat and img_calib.dat list files, pointed to by KEY_IMGLIST and KEY_CALIBLIST are writtn only once to reflect the list of input images.
• config.par and inst.par are left unchanged.
• After this, icat can be started, if attention is paid to a proper renaming of the mode.par file.

This does image calibration in all four quadrants and stops there. If the individual images should be combined and astrometrical analysis should be performed on them, see IcatCombine.

In combination with an IcatSource, either provided by a class with a default constructor (KEY_SOURCECLASS) or with a properties constructor (#KEY_SOURCEPROP), it suffices to pass the list of reducable files to the autoPrepare(java.util.List) method.

Nested Class Summary

static class	IcatPrepare.Dir This class reads all fits images in the specified directory and passes them to the split method, or, if a second command line argument is present, reads the specified file for the list input.
static class	IcatPrepare.Scan We list all science fits in a directory and pass it to the register source.
static class	IcatPrepare.Single This class splits the input file name on the first command line argument.

Nested classes/interfaces inherited from class astro.fits.Icat

Icat.Compress, Icat.Snoop, Icat.Uncompress

Nested classes/interfaces inherited from class util.PropertyResources

PropertyResources.URLResource

Field Summary

private Map<String,List<HeaderCardDescriptor>>	addcards The header card descriptors that are added for each quadrant.
private AmplifierCrosstalk	amplifiers The object for crosstalk correction or null.
private static String	DEFCALIBLIST The location of the calib-list file.
private static boolean	DEFCROSSTALK Default action: do crosstalk correction.
private static String	DEFIMGLIST The location of the image-list file.
private Map<String,String>	fits2set These header keys whose values should replace the parameter per name.
static String	KEY_AMPLIFIERS The properties of the amplifier crosstalk correction, if needed.
static String	KEY_CALIBLIST The location of the calib-list file.
static String	KEY_COPYCARDS This points to a list of fixed header cards.
static String	KEY_CROSSTALK If true, use crosstalk for flat and science, otherwise for none.
static String	KEY_FITSSETBRIDGE The mapping of fits keys to parameter names.
static String	KEY_GENERATECARDS A list of header cards generated from a parameter set.
static String	KEY_IMGLIST The location of the image-list file.
static String	KEY_KEEPPRIMARY A list of keys that should be kept in the split fits versions.
static String	KEY_KEEPSECONDARY A list of keys that should be kept in the extensions fits versions.
static String	KEY_OVERSCAN The properties of the overscan correction, if needed.
static String	KEY_PURGEPRIMARY A list of keys that should be kept in the split fits versions.
static String	KEY_PURGESECONDARY A list of keys that should be kept in the extensions fits versions.
static String	KEY_SET The properties of the parameter set used for the header cards.
static String	KEY_SOURCECLASS The class that should be used as the ICAT source, empty construct.
static String	KEY_SOURCEPROPS The class that should be used as the ICAT source, properties.
private Overscan	overscan The object for overscan correction or null.
static String	PREFIX_CARDS Prefix for additional fixed header keys for the four quadrants.
private ParameterClustering	set The set used for calculating the non-fixed header cards.
private IcatSource	src In automatic mode, this is the source provider.

Fields inherited from class astro.fits.Icat

BIASCOMBINED, DATAEXTENSION, DEFFROM, EXPTIME, FE, FLATHEADBIAS, FLATHEADDARK, FLATMASTERBIAS, FLATMASTERDARK, FLATMASTERSIGMA, INDEXCAL, INDEXSCI, INPUTBIAS, INPUTDARK, INPUTFLATPART, KEY_ANALYSISPAR, KEY_ANALYSISPARNO, KEY_ANALYSISPARYES, KEY_ARCHIVE, KEY_CALIBPAR, KEY_CALIBPARDARK, KEY_CALIBPARNODARK, KEY_CALIBPARSKIP, KEY_DATEKEY, KEY_EXPOSUREKEY, KEY_FILTERKEY, KEY_FROM, KEY_ICATDIR, KEY_ICATIN, KEY_IMAGETYPEKEY, KEY_IMCOPY, KEY_IMCOPYFLAG, KEY_LOCALFITS, KEY_LOCALTMP, KEY_MODEANALYSE, KEY_MODEPAR, KEY_REDUCED, KEY_REDUCEDONE, KEY_REPLACEFROM, KEY_REPLACETO, KEY_SERIALIZE, KEY_TO, lookbias, lookdark, lookflat, lookup, OUT, OUTBDSFLAT, OUTBIAS, OUTBSFLAT, PHOTOMETRYASC, QUADRANTS, SCIENCEBDFLAT, SCIENCEBFLAT, SCIENCEBIAS, SCIENCEDARK, SCIENCEFINAL, SCIENCERAW, SCIENCETEMP, YYYYMMDD

Fields inherited from class util.PropertyBundles

KEY_LOCALECOUNTRY, KEY_LOCALELANGUAGE, KEY_RESOURCEBUNDLES

Fields inherited from class util.PropertyResources

KEY_NOINITONCREATE, localurl, locate, POSTFIX_DIR, POSTFIX_EXT, POSTFIX_FILE, POSTFIX_LIST, POSTFIX_URL, urlset

Fields inherited from class util.PropertyContainer

KEY_LISTSEPARATOR, KEY_MAPKEYVALUECHAR, KEY_MAPSEPARATOR

Fields inherited from interface util.ResourceSupplying

KEY_URLRESOURCES, KEY_URLUSECONFIG, KEY_URLUSECURRENT, KEY_URLUSEHOME

Fields inherited from interface util.PropertySupplying

CONFIG, KEY_CLASS

Constructor Summary

IcatPrepare(Map<String,String> prop)
Creates a new IcatPrepare object and defaults keys not present in the properties handed over.

Method Summary

boolean	addBias(List<File> bias) Takes a list of bias images.
boolean	addDark(List<File> dark) Takes a list of dark images.
boolean	addFlat(List<File> flat) Takes a list of flat images.
boolean	addMasterFlat(List<File> flat) From a previous run, we copy a master flat.
boolean	addScience(List<File> science) Takes a list of science images.
boolean	autoPrepare(List<File> science) Auto prepare depends on a valid source object to be registered.
private Map<String,nom.tam.fits.HeaderCard>	createJoint(nom.tam.fits.Header h0, nom.tam.fits.Header h1) Creates postive and negativ lists out of our properties and uses the fits-tool merge to create a joint list.
void	exit() On exit, we prepare all config files in icat to be ready to run.
void	init() The parameter set is initialized and the list of HeaderCardDescriptors is generated.
String	split(nom.tam.fits.Fits in, int nr, boolean osexp) This method reads a four-extension fits file and splits it into different four single-extension fits files, residing in the four quadrant directories q0, q1, q2, and q3 below the Icat.KEY_ICATIN directory.
private Map<File,String>	splitList(List<File> in, int i0, boolean crosstalk, boolean expfit) Takes a list of input files, splits them and records the original and splitted file names.

Methods inherited from class astro.fits.Icat

addExtension, compress, getIcatInputExtension, getIcatName, getIcatOutputExtension, getInput, getMain, getOutput, getQuadrantInput, getQuadrantOutput, getSerializeFile, reducedDir, restoreLookup, saveLookup, sendMail, uncompress

Methods inherited from class util.PropertyBundles

clone, getLocalized, getLocalized, getLocalizedString, getLocalizedString, loadResource

Methods inherited from class util.PropertyResources

createFrom, createFrom, createFrom, getApplet, getAsResources, getLocalClassLoader, getPropertiesToKey, getPropertiesToKey, getResource, getResourceAsStream, getResourceFromKey, getResources, keyCreate, keyCreate, reload, setApplet

Methods inherited from class util.PropertyContainer

augment, augment, augment, defaultBoolean, defaultChar, defaultDouble, defaultFloat, defaultInt, defaultLong, defaultObject, defaultObject, defaultProperties, defaultProperty, getAsBoolean, getAsChar, getAsDouble, getAsEnums, getAsFloat, getAsInt, getAsList, getAsLong, getAsMap, getAsMap, getAsObject, getAsObject, getProperties, getProperty, has, isNew, parseObject, reload, removeProperty, rescanned, setObject, setProperties, setProperty, stringProperties, toString

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface util.ResourceSupplying

getResource, getResourceAsStream, getResources

Methods inherited from interface util.PropertySupplying

defaultBoolean, defaultChar, defaultDouble, defaultFloat, defaultInt, defaultLong, defaultObject, defaultObject, defaultProperties, defaultProperty, getAsBoolean, getAsChar, getAsDouble, getAsFloat, getAsInt, getAsList, getAsLong, getAsMap, getAsObject, getAsObject, getProperties, getProperty, has, parseObject, removeProperty, setObject, setProperty, stringProperties

Field Detail

PREFIX_CARDS

public static final String PREFIX_CARDS

Prefix for additional fixed header keys for the four quadrants.

See Also:

Constant Field Values

KEY_SOURCECLASS

public static final String KEY_SOURCECLASS

The class that should be used as the ICAT source, empty construct.

See Also:

Constant Field Values

KEY_SOURCEPROPS

public static final String KEY_SOURCEPROPS

The class that should be used as the ICAT source, properties.

See Also:

Constant Field Values

KEY_AMPLIFIERS

public static final String KEY_AMPLIFIERS

The properties of the amplifier crosstalk correction, if needed.

See Also:

Constant Field Values

KEY_OVERSCAN

public static final String KEY_OVERSCAN

The properties of the overscan correction, if needed.

See Also:

Constant Field Values

KEY_CROSSTALK

public static final String KEY_CROSSTALK

If true, use crosstalk for flat and science, otherwise for none.

See Also:

Constant Field Values

KEY_KEEPPRIMARY

public static final String KEY_KEEPPRIMARY

A list of keys that should be kept in the split fits versions.

See Also:

Constant Field Values

KEY_PURGEPRIMARY

public static final String KEY_PURGEPRIMARY

A list of keys that should be kept in the split fits versions.

See Also:

Constant Field Values

KEY_KEEPSECONDARY

public static final String KEY_KEEPSECONDARY

A list of keys that should be kept in the extensions fits versions.

See Also:

Constant Field Values

KEY_PURGESECONDARY

public static final String KEY_PURGESECONDARY

A list of keys that should be kept in the extensions fits versions.

See Also:

Constant Field Values

KEY_COPYCARDS

public static final String KEY_COPYCARDS

This points to a list of fixed header cards.

See Also:

Constant Field Values

KEY_GENERATECARDS

public static final String KEY_GENERATECARDS

A list of header cards generated from a parameter set.

See Also:

Constant Field Values

KEY_SET

public static final String KEY_SET

The properties of the parameter set used for the header cards.

See Also:

Constant Field Values

KEY_FITSSETBRIDGE

public static final String KEY_FITSSETBRIDGE

The mapping of fits keys to parameter names.

See Also:

Constant Field Values

KEY_IMGLIST

public static final String KEY_IMGLIST

The location of the image-list file.

See Also:

Constant Field Values

KEY_CALIBLIST

public static final String KEY_CALIBLIST

The location of the calib-list file.

See Also:

Constant Field Values

DEFIMGLIST

private static final String DEFIMGLIST

The location of the image-list file.

See Also:

Constant Field Values

DEFCALIBLIST

private static final String DEFCALIBLIST

The location of the calib-list file.

See Also:

Constant Field Values

DEFCROSSTALK

private static final boolean DEFCROSSTALK

Default action: do crosstalk correction.

See Also:

Constant Field Values

amplifiers

private AmplifierCrosstalk amplifiers

The object for crosstalk correction or null.

overscan

private Overscan overscan

The object for overscan correction or null.

set

private ParameterClustering set

The set used for calculating the non-fixed header cards.

fits2set

private Map<String,String> fits2set

These header keys whose values should replace the parameter per name.

addcards

private Map<String,List<HeaderCardDescriptor>> addcards

The header card descriptors that are added for each quadrant.

src

private IcatSource src

In automatic mode, this is the source provider.

Constructor Detail

IcatPrepare

public IcatPrepare(Map<String,String> prop)

Creates a new IcatPrepare object and defaults keys not present in the properties handed over.

Method Detail

init

public void init()

The parameter set is initialized and the list of HeaderCardDescriptors is generated. Preparation work is done including generating/clean-up of the quadrant input directories and their appropriate temp/uparm/out subdirectories.

Specified by:

init in interface Initializable

Overrides:

init in class Icat

exit

public void exit()

On exit, we prepare all config files in icat to be ready to run.

Specified by:

exit in interface ExitCleaning

autoPrepare

public boolean autoPrepare(List<File> science)

Auto prepare depends on a valid source object to be registered. The science input list is first registered at the source, then the bias, flat, and dark images are added to this icat prepare instance.

split

public String split(nom.tam.fits.Fits in,
                    int nr,
                    boolean osexp)
             throws nom.tam.fits.FitsException,
                    IOException

This method reads a four-extension fits file and splits it into different four single-extension fits files, residing in the four quadrant directories q0, q1, q2, and q3 below the Icat.KEY_ICATIN directory. The names of the file is according to ICAT standards: Science images:
yyyyMMdd'_pppss'TTTTT###F'R_imr.fits'
Calibration images:
yyyyMMdd'_CAL'TTTTT##'_c'EXT
where yyyyMMdd is the observing date, TTTTT is five digits of the observing time times ten, i.e. an observing time of 1.0 would show up as 00010, an observing time of 1.125 00011. ## or ### is the running number, for science images it has three digits, for calibrations only two!. All calibration types have their own running number, i.e. the first flat always has 01, regardless of the number of bias/darks. F is the filter name, like 'V'. Care should be taken with strange filter names, Sloan is therefor up, gp, etc. The EXT extension on the calibration image is determined by its type, 'bs.fits' for bias, 'th.fits' for darks, and 't'F'.fits' for flats, where F again is the filter.

Parameters:

nr - The running index of the file.

osexp - If true, try to fit the glow region.

Returns:

The constructed file name body, without extension

Throws:

nom.tam.fits.FitsException

IOException

splitList

private Map<File,String> splitList(List<File> in,
                                   int i0,
                                   boolean crosstalk,
                                   boolean expfit)
                            throws nom.tam.fits.FitsException,
                                   IOException

Takes a list of input files, splits them and records the original and splitted file names.

Throws:

nom.tam.fits.FitsException

IOException

addScience

public boolean addScience(List<File> science)
                   throws nom.tam.fits.FitsException,
                          IOException

Takes a list of science images.

Throws:

nom.tam.fits.FitsException

IOException

addBias

public boolean addBias(List<File> bias)
                throws nom.tam.fits.FitsException,
                       IOException

Takes a list of bias images.

Throws:

nom.tam.fits.FitsException

IOException

addDark

public boolean addDark(List<File> dark)
                throws nom.tam.fits.FitsException,
                       IOException

Takes a list of dark images.

Throws:

nom.tam.fits.FitsException

IOException

addFlat

public boolean addFlat(List<File> flat)
                throws nom.tam.fits.FitsException,
                       IOException

Takes a list of flat images.

Throws:

nom.tam.fits.FitsException

IOException

addMasterFlat

public boolean addMasterFlat(List<File> flat)
                      throws nom.tam.fits.FitsException,
                             IOException

From a previous run, we copy a master flat. We simply split the fits into four individual source files, named either flatbds or flatbs, depending on the availablity of dark frames. Note that this does not mean that the flat have been produced with darks, it merely lures the ICAT into finding the correct flats.

Throws:

nom.tam.fits.FitsException

IOException

createJoint

private Map<String,nom.tam.fits.HeaderCard> createJoint(nom.tam.fits.Header h0,
                                                        nom.tam.fits.Header h1)

Creates postive and negativ lists out of our properties and uses the fits-tool merge to create a joint list.