ReductionMaster

Overview

Package

Class

Use

Tree

Deprecated

Index

Help

PREV CLASS NEXT CLASS

FRAMES NO FRAMES

SUMMARY: NESTED | FIELD | CONSTR | METHOD

DETAIL: FIELD | CONSTR | METHOD

astro.fits
Class ReductionMaster

java.lang.Object
  util.PropertyContainer
      util.PropertyResources
          util.PropertyBundles
              astro.fits.ReductionMaster

All Implemented Interfaces:: Cloneable, Initializable, LocalizedSupplying, PropertySupplying, ResourceSupplying

Direct Known Subclasses:: BiasMaster, DarkMaster, FlatMaster

public class ReductionMaster
extends PropertyBundles
extends PropertyBundles

Base class that allows reduction of Wifsip images.

Nested Class Summary
`static class`	`ReductionMaster.ImageBlock` This class represents a block of identical type taken consecuetively during a single night on the same telescope.
`static class`	`ReductionMaster.ImageInfo` This class contains all the relevant information gathered from the db that might be relevant in the further reduction process.
`static class`	`ReductionMaster.Interpolate` This are the possible types of interpolation for calibration files.
`static class`	`ReductionMaster.Listing` We create the list of all object id's for the given night.
`static class`	`ReductionMaster.MasterFits` This class comprises a reduced version of an calibration image block.

Nested classes/interfaces inherited from class util.PropertyResources
`PropertyResources.URLResource`

Field Summary
`static String`	`ADUSEPARATOR` Separator char in filter-adu key generation.
`private AmplifierCrosstalk`	`amplifiers` The object for crosstalk correction or null.
`private static String`	`DEFAMBTEMP` Default name of the ambient temperature field in the obs db.
`private static String`	`DEFCCDTEMP` Default name of the detector temperature field in the obs db.
`private static String`	`DEFDEROTSEPARATE` Default we derotator-separate on flats.
`private static boolean`	`DEFEXTRAPOLATE` We may not extrapolate.
`private static String`	`DEFFILTERAWAREADU` Default we filter-separate on science and flats.
`private static String`	`DEFFILTERSEPARATE` Default we filter-separate on science and flats.
`private static String`	`DEFGAIN` Gain per quadrant, as of 20100802.
`private static String`	`DEFIMCOPY` Default path to imcopy.
`private static String`	`DEFIMCOPYFLAG` Default extension for compress.
`private static double`	`DEFINTRINSIC` Fraction of span allowed for interpolation rule.
`private static String`	`DEFLOCALFITS` A re-useable file name for decompressed fit.
`static String`	`DEFLOCALNFS` A mapping of archive path to local accessible ones.
`private static String`	`DEFLOCALTMP` Directory where the (de)compressed files are stored.
`private static String`	`DEFMASTERHEAD` The default master file name start.
`private static long`	`DEFMAXEXPOSURETIME` Default maximum exposure time, more means new block.
`private static int`	`DEFQUADRANTS` The default number of quadrants, equals number of amplifiers.
`private static Dimension`	`DEFQUADRANTSIZE` The default number of quadrants, equals number of amplifiers.
`private static String`	`DEFREADNOISE` RN per quadrant, as of 20100802.
`private static String`	`DEFREDUCED` Start of directory tree of archive.
`private static double`	`DEFROTTOLERANCE` Default tolerance for derotator.
`private static String`	`DEFSITE` The default site is IZANA.
`private static String`	`DEFSTRICTADU` Default we are strict on all but science.
`private static String`	`DEFTELESCOPE` The default telescope name is STELLA1.
`static Date`	`FIRSTNIGHT` The first date of observations.
`private Map<String,String>`	`fits2set` These header keys whose values should replace the parameter per name.
`protected List<Double>`	`gain` The list of gains in ADU per electrons per amplifier.
`private List<HeaderCardDescriptor>`	`generate` These cards should be generated for all data products.
`static String`	`INPOL` Header key denoting the interpolation rule used in an intermediate.
`static String`	`INPOLREM` Header key denoting the interpolation rule used in an intermediate.
`static String`	`IPOLANTE` Header key denoting the fraction of the earlier master used.
`static String`	`IPOLANTEREM` Header key denoting the fraction of the earlier master used.
`static String`	`IPOLBASE` Header key denoting the master difference in the interpolation base.
`static String`	`IPOLBASEREM` Header key denoting the master difference in the interpolation base.
`static String`	`IPOLDRFT` Header key denoting the average ADU-difference between masters.
`static String`	`IPOLDRFTREM` Header key denoting the average ADU-difference between masters.
`static String`	`IPOLPOST` Header key denoting the fraction of the later master used.
`static String`	`IPOLPOSTREM` Header key denoting the fraction of the later master used.
`static String`	`IPOLTIME` Header key denoting the time difference between masters.
`static String`	`IPOLTIMEREM` Header key denoting the time difference between masters.
`static String`	`KEY_ALLOWSPANAMBTEMP` If a calibrations spans more than this, we discard the entire block.
`static String`	`KEY_ALLOWSPANCCDTEMP` If a calibrations spans more than this, we discard the entire block.
`static String`	`KEY_ALLOWSPANTIME` If a calibrations spans more than this, we discard the entire block.
`static String`	`KEY_AMBTEMP` Not-named field in obs db.
`static String`	`KEY_AMPLIFIERS` The properties of the amplifier crosstalk correction, if needed.
`static String`	`KEY_AVCARDS` Card key mapping of original to average-value cards.
`static String`	`KEY_BESTRULES` The order in which we should try Interpolate to get to best solution.
`static String`	`KEY_BESTSPANS` If given, apply the rule only if span is lower than this.
`static String`	`KEY_CCDTEMP` Not-named field in obs db.
`static String`	`KEY_COPYCARDS` This points to a list of fixed header cards.
`static String`	`KEY_DEROTSEPARATE` This key points to a list of blocks that separate on derotator.
`static String`	`KEY_EQUALCARDS` Card names that have to point to equal values during combine.
`static String`	`KEY_EXTCARDORDER` Header card order in extension header.
`static String`	`KEY_EXTRAPOLATE` Are we allowed to extrapolate a given rule?
`static String`	`KEY_FILTERAWAREADU` These types discern further into filter for ADU stat testing.
`static String`	`KEY_FILTERSEPARATE` This key points to a list of blocks that separate on filter.
`static String`	`KEY_FIRSTCARDS` Card key mapping of original to first-in-combine cards.
`static String`	`KEY_FITSSETBRIDGE` The mapping of fits keys to parameter names.
`static String`	`KEY_GAIN` The gain values as a list, per amplifier.
`static String`	`KEY_GENERATECARDS` A list of header cards generated from a parameter set.
`static String`	`KEY_IMCOPY` The path to the imcopy external program.
`static String`	`KEY_IMCOPYFLAG` The thing to append to the target fits for imcopy.
`static String`	`KEY_INTRINSIC` If the span in a single master exceed the difference between the two.
`static String`	`KEY_LASTCARDS` Card key mapping of original to first-in-combine cards.
`static String`	`KEY_LOCALFITS` A re-useable file name for decompressed fits.
`static String`	`KEY_LOCALNFS` A mapping of archive path to local accessible ones.
`static String`	`KEY_LOCALTMP` Directory where the compressed files are stored for transfer.
`static String`	`KEY_MASTERHEAD` Hom master calibration file names should be heading.
`static String`	`KEY_MAXCARDS` Card key mapping of original to minimum-value cards.
`static String`	`KEY_MAXEXPOSURETIME` This key points to a ms maximum duration of a single exposure.
`static String`	`KEY_MEDIANCARDS` Card key mapping of original to median-value cards.
`static String`	`KEY_MINCARDS` Card key mapping of original to minimum-value cards.
`static String`	`KEY_OBSDB` Special properties for the obs-db.
`static String`	`KEY_OVERSCAN` The properties of the overscan correction, if needed.
`static String`	`KEY_PRIMCARDORDER` Header card order in primary header.
`static String`	`KEY_QUADRANTS` The number of quadrants in all reducable files.
`static String`	`KEY_QUADRANTSIZE` The size of quadrants in all reducable files.
`static String`	`KEY_READNOISE` The read-out noise values as a list, per amplifier
`static String`	`KEY_REDUCED` Path to the main reduced dir.
`static String`	`KEY_ROTTOLERANCE` The tolerance for considering identical derotator position.
`static String`	`KEY_SET` The properties of the parameter set used for the header cards.
`static String`	`KEY_SITE` The name of the site we work on.
`static String`	`KEY_STRICTADU` These types discern further into filter for ADU stat testing.
`static String`	`KEY_TELESCOPE` The key to the telescope name in the database.
`private Longitude`	`lambda` The longitude of Tenerife.
`private ObserveSql`	`obssql` The access to the obs table.
`private Overscan`	`overscan` The object for overscan correction or null.
`static String`	`PREFIX_ADUMAX` Prefix for constructin max-type ADU statstic border values.
`static String`	`PREFIX_ADUMIN` Prefix for constructin min-type ADU statstic border values.
`static String`	`PREFIX_AMPCARDS` Prefix for constructin amplifier-specific cards.
`protected List<Double>`	`readnoise` The list of read-noise in electrons per amplifier.
`private ParameterClustering`	`set` The set used for calculating the non-fixed header cards.

Fields inherited from class util.PropertyBundles
`KEY_LOCALECOUNTRY, KEY_LOCALELANGUAGE, KEY_RESOURCEBUNDLES`

Fields inherited from class util.PropertyResources
`KEY_NOINITONCREATE, localurl, locate, POSTFIX_DIR, POSTFIX_EXT, POSTFIX_FILE, POSTFIX_LIST, POSTFIX_URL, urlset`

Fields inherited from class util.PropertyContainer
`KEY_LISTSEPARATOR, KEY_MAPKEYVALUECHAR, KEY_MAPSEPARATOR`

Fields inherited from interface util.ResourceSupplying
`KEY_URLRESOURCES, KEY_URLUSECONFIG, KEY_URLUSECURRENT, KEY_URLUSEHOME`

Fields inherited from interface util.PropertySupplying
`CONFIG, KEY_CLASS`

Constructor Summary
`ReductionMaster(Map<String,String> prop)` Defaults all.

Method Summary
`protected void`	`addAverageCards(Map<String,String> ckey2mkey, Collection<nom.tam.fits.Header> combine, Map<String,nom.tam.fits.HeaderCard> to)` We take a mapping from input header cards keys to outputkeys, then read all values from all headers, calculate the average and add this with the new header card key to the input mapping.
`protected void`	`addEqualCards(List<String> ekeys, Collection<nom.tam.fits.Header> combine, Map<String,nom.tam.fits.HeaderCard> to)` Adds all cards that are truely pointing to equal values in all headers.
`protected void`	`addGeneratorCards(Map<String,nom.tam.fits.HeaderCard> joint, List<HeaderCardDescriptor> add)` From a header card description we get a valid header card at run-time.
`protected void`	`addMaximumCards(Map<String,String> ckey2mkey, Collection<nom.tam.fits.Header> combine, Map<String,nom.tam.fits.HeaderCard> to)` We take a mapping from input header cards keys to outputkeys, then read all values from all headers, calculate the average and add this with the new header card key to the input mapping.
`protected void`	`addMedianCards(Map<String,String> ckey2mkey, Collection<nom.tam.fits.Header> combine, Map<String,nom.tam.fits.HeaderCard> to)` We take a mapping from input header cards keys to outputkeys, then read all values from all headers, calculate the average and add this with the new header card key to the input mapping.
`protected void`	`addMinimumCards(Map<String,String> ckey2mkey, Collection<nom.tam.fits.Header> combine, Map<String,nom.tam.fits.HeaderCard> to)` We take a mapping from input header cards keys to outputkeys, then read all values from all headers, calculate the average and add this with the new header card key to the input mapping.
`protected void`	`addSingleCards(Map<String,String> ckey2mkey, nom.tam.fits.Header combine, Map<String,nom.tam.fits.HeaderCard> to)` We take a mapping from input header cards keys to outputkeys and assign the value found in the original header to it
`protected List<FitsCheck.Failure>`	`aduFailure(ReductionMaster.ImageInfo ii, List<List<Double>> min, List<List<Double>> max, boolean strict)` Does a simple ADU-statistic check on the image data.
`protected Map<String,List<FitsCheck.Failure>>`	`aduStatistic(ReductionMaster.ImageBlock ib)` Does a simple ADU-statistic check on all file in the image block.
`boolean`	`allTransferred(ReductionMaster.ImageBlock ib)` Tests, if all fits are transfered in a single image block.
`protected Map<String,List<FitsCheck.Failure>>`	`alreadyTested(ReductionMaster.ImageBlock ib)` In the current version, this reads the failure files, but eventually this should be updated to reflect a db access.
`private boolean`	`applyCardsToSet(Map<String,nom.tam.fits.HeaderCard> to)`
`protected boolean`	`checkAmbTempSpan(ReductionMaster.ImageBlock ib, double maxspan)` We check if the total span in ambient temperature in this image block is below the stated maximal span.
`protected boolean`	`checkDetTempSpan(ReductionMaster.ImageBlock ib, double maxspan)` We check if the total span in CCD temperature in this image block is below the stated maximal span.
`protected boolean`	`compress(File src, File dest)` Uses imcopy [compress] to compress a fits file from the original path to the destination path.
`nom.tam.fits.Fits`	`crosstalkCorrect(nom.tam.fits.Fits raw)` Does an crosstalk correction if the amplifier's properties are set.
`private static void`	`enterHeaderCard(Map<String,nom.tam.fits.HeaderCard> to, String keyrem, double statval, Class<?> base)`
`private int`	`execute(String exe, long timeout)` Uses the given executable command line string and executes that with a specified timeout.
`File`	`existing(ReductionMaster.ImageInfo ii)` Gets the existing file of an image info.
`Map<String,File>`	`existingFits(ReductionMaster.ImageBlock ib)` Converts an image block into the mapping of unique id to their exisiting fits files.
`protected List<List<Double>>`	`getAduConstraints(String key)` Parses a double-linked list into a list of a list of doubles, representing in the outer index the moment, in the inner list the amplifier.
`static Statistic`	`getHeaderValueStatistic(String key, Collection<nom.tam.fits.Header> comb)` From a collection of different headers we grab all occurences of the specified key.
`protected Map<String,nom.tam.fits.HeaderCard>`	`getMasterCards(SortedMap<String,nom.tam.fits.Header> u2prim)` This creates a master mapping of fits keys to appropriate header cards from a collection of (primary) headers.
`Map<String,nom.tam.fits.HeaderCard>`	`getMasterCardsFromPrimary(Map<String,File> u2fits)` We read the primary header of all file given in the mapping and calculate averages, min, max etc.
`long`	`getMidnight(Date mid)` Converts a date to the ms of the closest midnight.
`protected File`	`getReducedParentDir(ReductionMaster.ImageBlock ib)` This is the parent dir to the reduced image block.
`protected File`	`getTemporalFits()` Returns a temporal fits file for storing uncompressed data.
`Map<String,File>`	`getUncompressed(Map<String,File> archive)` From the mapping of unique to possible compressed filenames, we get the pointing of uniques to uncompressed files.
`void`	`init()` Initializes the obs-db.
`protected nom.tam.fits.Fits`	`intermediate(ReductionMaster.MasterFits ante, ReductionMaster.MasterFits post, ReductionMaster.ImageInfo ii, List<ReductionMaster.Interpolate> rules, double sigtrend, List<Double> sigspans, List<String> wcards)` For calibration purposes, we offer the possibility to interpolate here between two given master calibration fits to optimally reflect the state of the instrument/environment at the time the image in question has been taken.
`protected boolean`	`markTested(ReductionMaster.ImageBlock ib, Map<String,List<FitsCheck.Failure>> checked)` In the current version we write failure file, but eventually this should be updated to reflect a db access.
`protected static nom.tam.fits.Header`	`merge(nom.tam.fits.Header h, Map<String,nom.tam.fits.HeaderCard> add)` Merge the mapping of header cards to a valid header.
`protected static nom.tam.fits.Header`	`merge(nom.tam.fits.Header h, Map<String,nom.tam.fits.HeaderCard> add, Collection<String> order)` Merge the mapping of header cards to a valid header and use the order as in the collection (which is normally a list).
`nom.tam.fits.ImageHDU`	`overscanCorrect(nom.tam.fits.ImageHDU in, boolean osexp)` Does an overscan correction on the specified extension.
`nom.tam.fits.ImageHDU`	`primaryHDU(Map<String,nom.tam.fits.HeaderCard> add, Collection<String> order)` Gets a primary empty HDU using the specified mapping for the header.
`private SortedMap<String,nom.tam.fits.Header>`	`readPrimary(Map<String,File> u2fits)` Reads all headers from the primary HDU.
`protected Map<String,nom.tam.fits.HeaderCard>`	`readPrimaryCards(nom.tam.fits.Header h, List<String> p)` This reads all header cards from the given ImagHDU, but purges the list.
`boolean`	`removeUncompressed(Collection<File> fits)` Removes left-over uncompressed files from the tmp directory.
`protected nom.tam.fits.Fits`	`restoreMasterFits(ReductionMaster.MasterFits mf)` Returns a fully read and readily closed fits file out of a master fits path element.
`List<ReductionMaster.ImageBlock>`	`scanNight(long midnight)` We scan an entire night and return all image blocks found.
`nom.tam.fits.ImageHDU`	`secondaryHDU(nom.tam.fits.ImageHDU naked, Map<String,nom.tam.fits.HeaderCard> add, Collection<String> order)` Gets a secondary HDU by augmenting the already existinh HDU with header data derived from the specified mapping.
`protected nom.tam.fits.ImageHDU`	`skipToHDU(int nr, File fits)` We skip HDU until we can read the HDU at the given index.
`protected boolean`	`uncompress(File src, File dest)` Uses imcopy to uncompress a fits file from the original path to the destination path.
`protected File`	`uncompressEventually(File fitz, boolean overwrite)`

Methods inherited from class util.PropertyBundles
`clone, getLocalized, getLocalized, getLocalizedString, getLocalizedString, loadResource`

Methods inherited from class util.PropertyResources
`createFrom, createFrom, createFrom, getApplet, getAsResources, getLocalClassLoader, getPropertiesToKey, getPropertiesToKey, getResource, getResourceAsStream, getResourceFromKey, getResources, keyCreate, keyCreate, reload, setApplet`

Methods inherited from class util.PropertyContainer
`augment, augment, augment, defaultBoolean, defaultChar, defaultDouble, defaultFloat, defaultInt, defaultLong, defaultObject, defaultObject, defaultProperties, defaultProperty, getAsBoolean, getAsChar, getAsDouble, getAsEnums, getAsFloat, getAsInt, getAsList, getAsLong, getAsMap, getAsMap, getAsObject, getAsObject, getProperties, getProperty, has, isNew, parseObject, reload, removeProperty, rescanned, setObject, setProperties, setProperty, stringProperties, toString`

Methods inherited from class util.PropertyContainer

augment, augment, augment, defaultBoolean, defaultChar, defaultDouble, defaultFloat, defaultInt, defaultLong, defaultObject, defaultObject, defaultProperties, defaultProperty, getAsBoolean, getAsChar, getAsDouble, getAsEnums, getAsFloat, getAsInt, getAsList, getAsLong, getAsMap, getAsMap, getAsObject, getAsObject, getProperties, getProperty, has, isNew, parseObject, reload, removeProperty, rescanned, setObject, setProperties, setProperty, stringProperties, toString

Methods inherited from class java.lang.Object
`equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

Methods inherited from interface util.ResourceSupplying
`getResource, getResourceAsStream, getResources`

Methods inherited from interface util.PropertySupplying
`defaultBoolean, defaultChar, defaultDouble, defaultFloat, defaultInt, defaultLong, defaultObject, defaultObject, defaultProperties, defaultProperty, getAsBoolean, getAsChar, getAsDouble, getAsFloat, getAsInt, getAsList, getAsLong, getAsMap, getAsObject, getAsObject, getProperties, getProperty, has, parseObject, removeProperty, setObject, setProperty, stringProperties`

Field Detail

FIRSTNIGHT

public static Date FIRSTNIGHT

The first date of observations.

INPOL

public static final String INPOL

Header key denoting the interpolation rule used in an intermediate.

See Also:: Constant Field Values

IPOLBASE

public static final String IPOLBASE

Header key denoting the master difference in the interpolation base.

See Also:: Constant Field Values

IPOLDRFT

public static final String IPOLDRFT

Header key denoting the average ADU-difference between masters.

See Also:: Constant Field Values

IPOLANTE

public static final String IPOLANTE

Header key denoting the fraction of the earlier master used.

See Also:: Constant Field Values

IPOLPOST

public static final String IPOLPOST

Header key denoting the fraction of the later master used.

See Also:: Constant Field Values

IPOLTIME

public static final String IPOLTIME

Header key denoting the time difference between masters.

See Also:: Constant Field Values

INPOLREM

public static final String INPOLREM

Header key denoting the interpolation rule used in an intermediate.

See Also:: Constant Field Values

IPOLBASEREM

public static final String IPOLBASEREM

Header key denoting the master difference in the interpolation base.

See Also:: Constant Field Values

IPOLDRFTREM

public static final String IPOLDRFTREM

Header key denoting the average ADU-difference between masters.

See Also:: Constant Field Values

IPOLANTEREM

public static final String IPOLANTEREM

Header key denoting the fraction of the earlier master used.

See Also:: Constant Field Values

IPOLPOSTREM

public static final String IPOLPOSTREM

Header key denoting the fraction of the later master used.

See Also:: Constant Field Values

IPOLTIMEREM

public static final String IPOLTIMEREM

Header key denoting the time difference between masters.

See Also:: Constant Field Values

ADUSEPARATOR

public static final String ADUSEPARATOR

Separator char in filter-adu key generation.

See Also:: Constant Field Values

PREFIX_AMPCARDS

public static final String PREFIX_AMPCARDS

Prefix for constructin amplifier-specific cards.

See Also:: Constant Field Values

PREFIX_ADUMIN

public static final String PREFIX_ADUMIN

Prefix for constructin min-type ADU statstic border values.

See Also:: Constant Field Values

PREFIX_ADUMAX

public static final String PREFIX_ADUMAX

Prefix for constructin max-type ADU statstic border values.

See Also:: Constant Field Values

KEY_MASTERHEAD

public static final String KEY_MASTERHEAD

Hom master calibration file names should be heading.

See Also:: Constant Field Values

KEY_QUADRANTS

public static final String KEY_QUADRANTS

The number of quadrants in all reducable files.

See Also:: Constant Field Values

KEY_QUADRANTSIZE

public static final String KEY_QUADRANTSIZE

The size of quadrants in all reducable files.

See Also:: Constant Field Values

KEY_READNOISE

public static final String KEY_READNOISE

The read-out noise values as a list, per amplifier

See Also:: Constant Field Values

KEY_GAIN

public static final String KEY_GAIN

The gain values as a list, per amplifier.

See Also:: Constant Field Values

KEY_AMPLIFIERS

public static final String KEY_AMPLIFIERS

The properties of the amplifier crosstalk correction, if needed.

See Also:: Constant Field Values

KEY_OVERSCAN

public static final String KEY_OVERSCAN

The properties of the overscan correction, if needed.

See Also:: Constant Field Values

KEY_COPYCARDS

public static final String KEY_COPYCARDS

This points to a list of fixed header cards.

See Also:: Constant Field Values

KEY_GENERATECARDS

public static final String KEY_GENERATECARDS

A list of header cards generated from a parameter set.

See Also:: Constant Field Values

KEY_SET

public static final String KEY_SET

The properties of the parameter set used for the header cards.

See Also:: Constant Field Values

KEY_FITSSETBRIDGE

public static final String KEY_FITSSETBRIDGE

The mapping of fits keys to parameter names.

See Also:: Constant Field Values

KEY_TELESCOPE

public static final String KEY_TELESCOPE

The key to the telescope name in the database.

See Also:: Constant Field Values

KEY_SITE

public static final String KEY_SITE

The name of the site we work on.

See Also:: Constant Field Values

KEY_IMCOPY

public static final String KEY_IMCOPY

The path to the imcopy external program.

See Also:: Constant Field Values

KEY_IMCOPYFLAG

public static final String KEY_IMCOPYFLAG

The thing to append to the target fits for imcopy.

See Also:: Constant Field Values

KEY_LOCALTMP

public static final String KEY_LOCALTMP

Directory where the compressed files are stored for transfer.

See Also:: Constant Field Values

KEY_LOCALFITS

public static final String KEY_LOCALFITS

A re-useable file name for decompressed fits.

See Also:: Constant Field Values

KEY_OBSDB

public static final String KEY_OBSDB

Special properties for the obs-db.

See Also:: Constant Field Values

KEY_MAXEXPOSURETIME

public static final String KEY_MAXEXPOSURETIME

This key points to a ms maximum duration of a single exposure.

See Also:: Constant Field Values

KEY_FILTERSEPARATE

public static final String KEY_FILTERSEPARATE

This key points to a list of blocks that separate on filter.

See Also:: Constant Field Values

KEY_DEROTSEPARATE

public static final String KEY_DEROTSEPARATE

This key points to a list of blocks that separate on derotator.

See Also:: Constant Field Values

KEY_ROTTOLERANCE

public static final String KEY_ROTTOLERANCE

The tolerance for considering identical derotator position.

See Also:: Constant Field Values

KEY_CCDTEMP

public static final String KEY_CCDTEMP

Not-named field in obs db.

See Also:: Constant Field Values

KEY_AMBTEMP

public static final String KEY_AMBTEMP

Not-named field in obs db.

See Also:: Constant Field Values

KEY_ALLOWSPANCCDTEMP

public static final String KEY_ALLOWSPANCCDTEMP

If a calibrations spans more than this, we discard the entire block.

See Also:: Constant Field Values

KEY_ALLOWSPANAMBTEMP

public static final String KEY_ALLOWSPANAMBTEMP

If a calibrations spans more than this, we discard the entire block.

See Also:: Constant Field Values

KEY_ALLOWSPANTIME

public static final String KEY_ALLOWSPANTIME

If a calibrations spans more than this, we discard the entire block.

See Also:: Constant Field Values

KEY_FILTERAWAREADU

public static final String KEY_FILTERAWAREADU

These types discern further into filter for ADU stat testing.

See Also:: Constant Field Values

KEY_STRICTADU

public static final String KEY_STRICTADU

These types discern further into filter for ADU stat testing.

See Also:: Constant Field Values

KEY_LOCALNFS

public static final String KEY_LOCALNFS

A mapping of archive path to local accessible ones.

See Also:: Constant Field Values

KEY_REDUCED

public static final String KEY_REDUCED

Path to the main reduced dir.

See Also:: Constant Field Values

KEY_EQUALCARDS

public static final String KEY_EQUALCARDS

Card names that have to point to equal values during combine.

See Also:: Constant Field Values

KEY_AVCARDS

public static final String KEY_AVCARDS

Card key mapping of original to average-value cards.

See Also:: Constant Field Values

KEY_MEDIANCARDS

public static final String KEY_MEDIANCARDS

Card key mapping of original to median-value cards.

See Also:: Constant Field Values

KEY_MINCARDS

public static final String KEY_MINCARDS

Card key mapping of original to minimum-value cards.

See Also:: Constant Field Values

KEY_MAXCARDS

public static final String KEY_MAXCARDS

Card key mapping of original to minimum-value cards.

See Also:: Constant Field Values

KEY_FIRSTCARDS

public static final String KEY_FIRSTCARDS

Card key mapping of original to first-in-combine cards.

See Also:: Constant Field Values

KEY_LASTCARDS

public static final String KEY_LASTCARDS

Card key mapping of original to first-in-combine cards.

See Also:: Constant Field Values

KEY_PRIMCARDORDER

public static final String KEY_PRIMCARDORDER

Header card order in primary header.

See Also:: Constant Field Values

KEY_EXTCARDORDER

public static final String KEY_EXTCARDORDER

Header card order in extension header.

See Also:: Constant Field Values

KEY_BESTRULES

public static final String KEY_BESTRULES

The order in which we should try Interpolate to get to best solution.

See Also:: Constant Field Values

KEY_EXTRAPOLATE

public static final String KEY_EXTRAPOLATE

Are we allowed to extrapolate a given rule?

See Also:: Constant Field Values

KEY_BESTSPANS

public static final String KEY_BESTSPANS

If given, apply the rule only if span is lower than this.

See Also:: Constant Field Values

KEY_INTRINSIC

public static final String KEY_INTRINSIC

If the span in a single master exceed the difference between the two.

See Also:: Constant Field Values

DEFMASTERHEAD

private static final String DEFMASTERHEAD

The default master file name start.

See Also:: Constant Field Values

DEFQUADRANTS

private static final int DEFQUADRANTS

The default number of quadrants, equals number of amplifiers.

See Also:: Constant Field Values

DEFQUADRANTSIZE

private static final Dimension DEFQUADRANTSIZE

The default number of quadrants, equals number of amplifiers.

DEFREADNOISE

private static final String DEFREADNOISE

RN per quadrant, as of 20100802.

See Also:: Constant Field Values

DEFGAIN

private static final String DEFGAIN

Gain per quadrant, as of 20100802.

See Also:: Constant Field Values

DEFTELESCOPE

private static final String DEFTELESCOPE

The default telescope name is STELLA1.

See Also:: Constant Field Values

DEFIMCOPY

private static final String DEFIMCOPY

Default path to imcopy.

See Also:: Constant Field Values

DEFIMCOPYFLAG

private static final String DEFIMCOPYFLAG

Default extension for compress.

See Also:: Constant Field Values

DEFLOCALTMP

private static final String DEFLOCALTMP

Directory where the (de)compressed files are stored.

See Also:: Constant Field Values

DEFLOCALFITS

private static final String DEFLOCALFITS

A re-useable file name for decompressed fit.

See Also:: Constant Field Values

DEFSITE

private static final String DEFSITE

The default site is IZANA.

See Also:: Constant Field Values

DEFFILTERSEPARATE

private static final String DEFFILTERSEPARATE

Default we filter-separate on science and flats.

See Also:: Constant Field Values

DEFDEROTSEPARATE

private static final String DEFDEROTSEPARATE

Default we derotator-separate on flats.

See Also:: Constant Field Values

DEFCCDTEMP

private static final String DEFCCDTEMP

Default name of the detector temperature field in the obs db.

See Also:: Constant Field Values

DEFAMBTEMP

private static final String DEFAMBTEMP

Default name of the ambient temperature field in the obs db.

See Also:: Constant Field Values

DEFMAXEXPOSURETIME

private static final long DEFMAXEXPOSURETIME

Default maximum exposure time, more means new block.

See Also:: Constant Field Values

DEFROTTOLERANCE

private static final double DEFROTTOLERANCE

Default tolerance for derotator.

See Also:: Constant Field Values

DEFEXTRAPOLATE

private static final boolean DEFEXTRAPOLATE

We may not extrapolate.

See Also:: Constant Field Values

DEFINTRINSIC

private static final double DEFINTRINSIC

Fraction of span allowed for interpolation rule.

See Also:: Constant Field Values

DEFFILTERAWAREADU

private static final String DEFFILTERAWAREADU

Default we filter-separate on science and flats.

See Also:: Constant Field Values

DEFSTRICTADU

private static final String DEFSTRICTADU

Default we are strict on all but science.

See Also:: Constant Field Values

DEFLOCALNFS

public static final String DEFLOCALNFS

A mapping of archive path to local accessible ones.

See Also:: Constant Field Values

DEFREDUCED

private static final String DEFREDUCED

Start of directory tree of archive.

See Also:: Constant Field Values

readnoise

protected List<Double> readnoise

The list of read-noise in electrons per amplifier.

gain

protected List<Double> gain

The list of gains in ADU per electrons per amplifier.

obssql

private ObserveSql obssql

The access to the obs table.

lambda

private Longitude lambda

The longitude of Tenerife.

amplifiers

private AmplifierCrosstalk amplifiers

The object for crosstalk correction or null.

overscan

private Overscan overscan

The object for overscan correction or null.

set

private ParameterClustering set

The set used for calculating the non-fixed header cards.

fits2set

private Map<String,String> fits2set

These header keys whose values should replace the parameter per name.

generate

private List<HeaderCardDescriptor> generate

These cards should be generated for all data products.

Constructor Detail

ReductionMaster

public ReductionMaster(Map<String,String> prop)

Defaults all.

Method Detail

init

public void init()

Initializes the obs-db.

Specified by:: init in interface Initializable
Overrides:: init in class PropertyResources

getMidnight

public long getMidnight(Date mid)

Converts a date to the ms of the closest midnight.

allTransferred

public boolean allTransferred(ReductionMaster.ImageBlock ib)

Tests, if all fits are transfered in a single image block.

existingFits

public Map<String,File> existingFits(ReductionMaster.ImageBlock ib)

Converts an image block into the mapping of unique id to their exisiting fits files. Unmatched uniques are silently ignored, but a quick check is to compare the size of the unique list of the block to the size of the mapping returned here.
Unfortunately, only the file extension can be used to indicate compressed (".fitz") or uncompressed data (all other extensions, but checked only for .fits).

existing

public File existing(ReductionMaster.ImageInfo ii)

Gets the existing file of an image info.

uncompressEventually

protected File uncompressEventually(File fitz,
                                    boolean overwrite)
                             throws IOException

Throws:: IOException

checkAmbTempSpan

protected boolean checkAmbTempSpan(ReductionMaster.ImageBlock ib,
                                   double maxspan)

We check if the total span in ambient temperature in this image block is below the stated maximal span.

checkDetTempSpan

protected boolean checkDetTempSpan(ReductionMaster.ImageBlock ib,
                                   double maxspan)

We check if the total span in CCD temperature in this image block is below the stated maximal span.

getUncompressed

public Map<String,File> getUncompressed(Map<String,File> archive)
                                 throws IOException

From the mapping of unique to possible compressed filenames, we get the pointing of uniques to uncompressed files. If the file are not compressed, the original mapping is returned, otherwise each compressed file is decompressed in #KEY_TMP with the identical filename, but ending FitsTools.FITSEXTENSION. Compression is deduced from the file ending only.

Throws:: IOException

removeUncompressed

public boolean removeUncompressed(Collection<File> fits)

Removes left-over uncompressed files from the tmp directory.

getTemporalFits

protected File getTemporalFits()

Returns a temporal fits file for storing uncompressed data.

compress

protected boolean compress(File src,
                           File dest)

Uses imcopy [compress] to compress a fits file from the original path to the destination path. See Error codes

uncompress

protected boolean uncompress(File src,
                             File dest)

Uses imcopy to uncompress a fits file from the original path to the destination path. See Error codes Because imcopy tends to fail for unknown reasons (error code 106), but on perfectly uncompressable files, we loop on this process at max 3 times.

overscanCorrect

public nom.tam.fits.ImageHDU overscanCorrect(nom.tam.fits.ImageHDU in,
                                             boolean osexp)
                                      throws nom.tam.fits.FitsException

Does an overscan correction on the specified extension.

Throws:: nom.tam.fits.FitsException

crosstalkCorrect

public nom.tam.fits.Fits crosstalkCorrect(nom.tam.fits.Fits raw)
                                   throws nom.tam.fits.FitsException,
                                          IOException

Does an crosstalk correction if the amplifier's properties are set. Works always together on all quadrants, thus we work on fits.

Throws:: nom.tam.fits.FitsException; IOException

intermediate

protected nom.tam.fits.Fits intermediate(ReductionMaster.MasterFits ante,
                                         ReductionMaster.MasterFits post,
                                         ReductionMaster.ImageInfo ii,
                                         List<ReductionMaster.Interpolate> rules,
                                         double sigtrend,
                                         List<Double> sigspans,
                                         List<String> wcards)
                                  throws IOException,
                                         nom.tam.fits.FitsException

For calibration purposes, we offer the possibility to interpolate here between two given master calibration fits to optimally reflect the state of the instrument/environment at the time the image in question has been taken. The two master fits should normally bracket the image described by the image info in a time-like manner. First, the difference between the two calibration images is calculated, the RMS of the difference is added with the #IPOLRMS to the primary header. A fraction of this difference is used to interpolate to the best calibration fits. Per default, the times of exposure are used to calculate this fraction, but depending on the rules, also other interpolation schemes can be utilized, namely those listed in #Interpolate. These other rules are only applied, firstly, if the RMS of the calibration master differences exceeds the value for a significant trend (if not, time interpolation is used). If the trend is significant, we try the first rule. This rule comes with a significant span associated with it, which is on appropriate position in the final argument. Only if the difference of the averages in the two masters (say, difference in CCD Temperature) exceeds the significance value, the rule may be applied, but only if the spans in the individual two masters (ccdtempspan) do not exceed the KEY_INTRINSIC times total difference. If a rule is discarded, the next rule is tested until no more rules are present, in which case again time sclicing is applied. The method (rule) used is recorded in the #INTPOL key. The obs-time difference in seconds is recorded in IPOLTIME, the difference of the two master rules used for interpolation is recorded in IPOLBASE. The two contributions (weights) are recorded in IPOLANTE and IPOLPOST.

Parameters:: wcards - A list of card keys in the amplifier extensions to keep as a weighted average in the intermediate fits.
Throws:: IOException; nom.tam.fits.FitsException

execute

private int execute(String exe,
                    long timeout)

Uses the given executable command line string and executes that with a specified timeout.

getMasterCardsFromPrimary

public Map<String,nom.tam.fits.HeaderCard> getMasterCardsFromPrimary(Map<String,File> u2fits)
                                                              throws nom.tam.fits.FitsException,
                                                                     IOException

We read the primary header of all file given in the mapping and calculate averages, min, max etc. out of it, following the prescriptions stored in the CARDS keys. Only keys requested for are kept, others discarded.

Throws:: nom.tam.fits.FitsException; IOException

getMasterCards

protected Map<String,nom.tam.fits.HeaderCard> getMasterCards(SortedMap<String,nom.tam.fits.Header> u2prim)
                                                      throws nom.tam.fits.HeaderCardException

This creates a master mapping of fits keys to appropriate header cards from a collection of (primary) headers.

Throws:: nom.tam.fits.HeaderCardException

readPrimaryCards

protected Map<String,nom.tam.fits.HeaderCard> readPrimaryCards(nom.tam.fits.Header h,
                                                               List<String> p)
                                                        throws nom.tam.fits.HeaderCardException

This reads all header cards from the given ImagHDU, but purges the list. Then applies it to the set and generate additional cards.

Throws:: nom.tam.fits.HeaderCardException

primaryHDU

public nom.tam.fits.ImageHDU primaryHDU(Map<String,nom.tam.fits.HeaderCard> add,
                                        Collection<String> order)

Gets a primary empty HDU using the specified mapping for the header. Ordering may be null, if the primary mapping already imposes a ordering or if the order is unimportant.

secondaryHDU

public nom.tam.fits.ImageHDU secondaryHDU(nom.tam.fits.ImageHDU naked,
                                          Map<String,nom.tam.fits.HeaderCard> add,
                                          Collection<String> order)

Gets a secondary HDU by augmenting the already existinh HDU with header data derived from the specified mapping. Ordering may be null, if the primary mapping already imposes a ordering or if the order is unimportant.

merge

protected static nom.tam.fits.Header merge(nom.tam.fits.Header h,
                                           Map<String,nom.tam.fits.HeaderCard> add)

Merge the mapping of header cards to a valid header.

merge

protected static nom.tam.fits.Header merge(nom.tam.fits.Header h,
                                           Map<String,nom.tam.fits.HeaderCard> add,
                                           Collection<String> order)

Merge the mapping of header cards to a valid header and use the order as in the collection (which is normally a list).

skipToHDU

protected nom.tam.fits.ImageHDU skipToHDU(int nr,
                                          File fits)
                                   throws nom.tam.fits.FitsException,
                                          IOException

We skip HDU until we can read the HDU at the given index. The file must be readable.

Throws:: nom.tam.fits.FitsException; IOException

readPrimary

private SortedMap<String,nom.tam.fits.Header> readPrimary(Map<String,File> u2fits)
                                                   throws nom.tam.fits.FitsException,
                                                          IOException

Reads all headers from the primary HDU.

Throws:: nom.tam.fits.FitsException; IOException

addEqualCards

protected void addEqualCards(List<String> ekeys,
                             Collection<nom.tam.fits.Header> combine,
                             Map<String,nom.tam.fits.HeaderCard> to)

Adds all cards that are truely pointing to equal values in all headers.

addSingleCards

protected void addSingleCards(Map<String,String> ckey2mkey,
                              nom.tam.fits.Header combine,
                              Map<String,nom.tam.fits.HeaderCard> to)
                       throws nom.tam.fits.HeaderCardException

We take a mapping from input header cards keys to outputkeys and assign the value found in the original header to it

Throws:: nom.tam.fits.HeaderCardException

applyCardsToSet

private boolean applyCardsToSet(Map<String,nom.tam.fits.HeaderCard> to)

addGeneratorCards

protected void addGeneratorCards(Map<String,nom.tam.fits.HeaderCard> joint,
                                 List<HeaderCardDescriptor> add)
                          throws nom.tam.fits.HeaderCardException

From a header card description we get a valid header card at run-time.

Throws:: nom.tam.fits.HeaderCardException

addMedianCards

protected void addMedianCards(Map<String,String> ckey2mkey,
                              Collection<nom.tam.fits.Header> combine,
                              Map<String,nom.tam.fits.HeaderCard> to)
                       throws nom.tam.fits.HeaderCardException

We take a mapping from input header cards keys to outputkeys, then read all values from all headers, calculate the average and add this with the new header card key to the input mapping.

Throws:: nom.tam.fits.HeaderCardException

addMinimumCards

protected void addMinimumCards(Map<String,String> ckey2mkey,
                               Collection<nom.tam.fits.Header> combine,
                               Map<String,nom.tam.fits.HeaderCard> to)
                        throws nom.tam.fits.HeaderCardException

We take a mapping from input header cards keys to outputkeys, then read all values from all headers, calculate the average and add this with the new header card key to the input mapping.

Throws:: nom.tam.fits.HeaderCardException

addMaximumCards

protected void addMaximumCards(Map<String,String> ckey2mkey,
                               Collection<nom.tam.fits.Header> combine,
                               Map<String,nom.tam.fits.HeaderCard> to)
                        throws nom.tam.fits.HeaderCardException

We take a mapping from input header cards keys to outputkeys, then read all values from all headers, calculate the average and add this with the new header card key to the input mapping.

Throws:: nom.tam.fits.HeaderCardException

addAverageCards

protected void addAverageCards(Map<String,String> ckey2mkey,
                               Collection<nom.tam.fits.Header> combine,
                               Map<String,nom.tam.fits.HeaderCard> to)
                        throws nom.tam.fits.HeaderCardException

We take a mapping from input header cards keys to outputkeys, then read all values from all headers, calculate the average and add this with the new header card key to the input mapping.

Throws:: nom.tam.fits.HeaderCardException

enterHeaderCard

private static void enterHeaderCard(Map<String,nom.tam.fits.HeaderCard> to,
                                    String keyrem,
                                    double statval,
                                    Class<?> base)
                             throws nom.tam.fits.HeaderCardException

Throws:: nom.tam.fits.HeaderCardException

getHeaderValueStatistic

public static Statistic getHeaderValueStatistic(String key,
                                                Collection<nom.tam.fits.Header> comb)

From a collection of different headers we grab all occurences of the specified key. The value of the card is converted to a double, either using the numerical value directly, or converting to milliseconds in case the key points to a date, or to 0/1 for false/true binary cards. In the latter two cases, the base class is annotated to the statistic.

getAduConstraints

protected List<List<Double>> getAduConstraints(String key)

Parses a double-linked list into a list of a list of doubles, representing in the outer index the moment, in the inner list the amplifier. This means a list like 10000,11000,12000,9000;300,-1,100,200 is interprated as, averages 10000,11000,12000,9000 for the four amplifiers and as variances 300,don't care, 100, and 200. It is used for the minima and maxima. If the maxima is less then the minima, this means this moment for this particular quadrant is ignored.

alreadyTested

protected Map<String,List<FitsCheck.Failure>> alreadyTested(ReductionMaster.ImageBlock ib)

In the current version, this reads the failure files, but eventually this should be updated to reflect a db access.

markTested

protected boolean markTested(ReductionMaster.ImageBlock ib,
                             Map<String,List<FitsCheck.Failure>> checked)

In the current version we write failure file, but eventually this should be updated to reflect a db access.

getReducedParentDir

protected File getReducedParentDir(ReductionMaster.ImageBlock ib)

This is the parent dir to the reduced image block.

aduStatistic

protected Map<String,List<FitsCheck.Failure>> aduStatistic(ReductionMaster.ImageBlock ib)

Does a simple ADU-statistic check on all file in the image block. Returns a mapping of unique ID to failure flag.

aduFailure

protected List<FitsCheck.Failure> aduFailure(ReductionMaster.ImageInfo ii,
                                             List<List<Double>> min,
                                             List<List<Double>> max,
                                             boolean strict)

Does a simple ADU-statistic check on the image data. If no failures are found, an empty list is returned, otherwise the ADU, SIGMA or MOMENT failure is set.

scanNight

public List<ReductionMaster.ImageBlock> scanNight(long midnight)
                                           throws SQLException

We scan an entire night and return all image blocks found. Neither of the files is checked for existence or if they satisfy the raw-quality check, this should be done on individual blocks. If the blocking should include separation for filter and/or derotator, the property KEY_FILTERSEPARATE and KEY_DEROTSEPARATE should be set and pointing to a list of image types.

Throws:: SQLException

restoreMasterFits

protected nom.tam.fits.Fits restoreMasterFits(ReductionMaster.MasterFits mf)
                                       throws IOException,
                                              nom.tam.fits.FitsException

Returns a fully read and readily closed fits file out of a master fits path element. Memory links are avoided (I think).

Throws:: IOException; nom.tam.fits.FitsException

Overview

Package

Class

Use

Tree

Deprecated

Index

Help

PREV CLASS NEXT CLASS

FRAMES NO FRAMES

SUMMARY: NESTED | FIELD | CONSTR | METHOD

DETAIL: FIELD | CONSTR | METHOD

astro.fits Class ReductionMaster

FIRSTNIGHT

INPOL

IPOLBASE

IPOLDRFT

IPOLANTE

IPOLPOST

IPOLTIME

INPOLREM

IPOLBASEREM

IPOLDRFTREM

IPOLANTEREM

IPOLPOSTREM

IPOLTIMEREM

ADUSEPARATOR

PREFIX_AMPCARDS

PREFIX_ADUMIN

PREFIX_ADUMAX

KEY_MASTERHEAD

KEY_QUADRANTS

KEY_QUADRANTSIZE

KEY_READNOISE

KEY_GAIN

KEY_AMPLIFIERS

KEY_OVERSCAN

KEY_COPYCARDS

KEY_GENERATECARDS

KEY_SET

KEY_FITSSETBRIDGE

KEY_TELESCOPE

KEY_SITE

KEY_IMCOPY

KEY_IMCOPYFLAG

KEY_LOCALTMP

KEY_LOCALFITS

KEY_OBSDB

KEY_MAXEXPOSURETIME

KEY_FILTERSEPARATE

KEY_DEROTSEPARATE

KEY_ROTTOLERANCE

KEY_CCDTEMP

KEY_AMBTEMP

KEY_ALLOWSPANCCDTEMP

KEY_ALLOWSPANAMBTEMP

KEY_ALLOWSPANTIME

KEY_FILTERAWAREADU

KEY_STRICTADU

KEY_LOCALNFS

KEY_REDUCED

KEY_EQUALCARDS

KEY_AVCARDS

KEY_MEDIANCARDS

KEY_MINCARDS

KEY_MAXCARDS

KEY_FIRSTCARDS

KEY_LASTCARDS

KEY_PRIMCARDORDER

KEY_EXTCARDORDER

KEY_BESTRULES

KEY_EXTRAPOLATE

KEY_BESTSPANS

KEY_INTRINSIC

DEFMASTERHEAD

DEFQUADRANTS

DEFQUADRANTSIZE

DEFREADNOISE

DEFGAIN

DEFTELESCOPE

DEFIMCOPY

DEFIMCOPYFLAG

DEFLOCALTMP

DEFLOCALFITS

DEFSITE

DEFFILTERSEPARATE

DEFDEROTSEPARATE

DEFCCDTEMP

DEFAMBTEMP

DEFMAXEXPOSURETIME

DEFROTTOLERANCE

DEFEXTRAPOLATE

astro.fits
Class ReductionMaster