astro.fits
Class AmplifierCrosstalk

java.lang.Object
  util.PropertyContainer
      util.PropertyResources
          util.PropertyBundles
              astro.fits.AmplifierCrosstalk

All Implemented Interfaces:

Cloneable, Initializable, LocalizedSupplying, PropertySupplying, ResourceSupplying

public class AmplifierCrosstalk
extends PropertyBundles

This class is designed to measure crosstalk between amplifiers and prepare reduction files to reduce it as far as possible.

Amplifier crosstalk can happen if a monolithic chip is read out using more than a single amplifier, like reported by Freyhammer et al. Experimental Astronomy 12, 147 (2001).
We write the measured ADU signal in amplifier i as

   ADU_i(x,y) = Σ_j α_ij(x,y) · N_j(x,y),
   

where N_j is the true signal at point x,y, summing over all quadrants, and α_ij(x,y) is a quadratic matrix of dimensionality equal to the number of amplifies. Consequentially, the true signal can be restored by

   N_i(x,y) = Σ_j α*_ij · ADU_j(x,y),
   

where α*_ij is the inverse of α_ij(x,y). Ideally, α_ij(x,y) collapses to the Kronecker δ_ij. Note that even if the linearity beween measured and true signal holds until the full-well capacity is reached, normal gain setup in CCD prevent a full recovery since the maximum ADU level (65535 in most cases) ist somewhere below fill-well capacity, in the linear regime of the CCD. Thus, a saturated signal of 65535 may lead to different responses in the affected quadrants, depending wether full-well was already reached. In this region, the correction will not be precise.

The purpose of this class is to measure the forward-crosstalk α*_ij by providing a set of N fits images, each with N extensions corresponding to the amplifiers, where each image ideally is illuminated in a single amplifier section only. We then identify the cross-talk signals in all the other than the exposed section my measuring the signal on x,y and correcting for background light, either by a constant level or a dedicated background image, or even a combination of both. The true signal in the illuminated amplifier is established as the measured signal after a possible backgorund subtraction. Using the entire set of images, the off-diagonal elements of α_ij(x,y) are thus defined, α_ii=1 is assumed. Matrix α_ij(x,y) is then inverted to yield α*_ij(x,y), which is then stored as N fits images with N extensions, allowing then direct recovery of the true signal for all science images taken.

To check the validity of the homogeneous (!) linear relation in the first place, one should use at least two different illumination levels and compare the results. Note, too, that high S/N is required to recover the weak crosstalk signal (an effect below order of 10^-4, hopefully), thus the input images to this class should be gained by averageing a couple of images. Eventually, median-averaging over boxes in the original image may help, thus loosing the pixel-to-pixel response of the reconstruction, but gaining a lot of signal.

Helper classes exists for creating empty HDUs of various sizes. This class is mainly intended for the WiFSIP instrument, which features a four-amp chip.

Nested Class Summary

static class	AmplifierCrosstalk.Gradient This class tries to get from real-world measurements to useful input images by subtracting a gradient in the background.
static class	AmplifierCrosstalk.LinearRestore This class takes the fits on the command line argument and does the amplifier correction using the properties stated.
static class	AmplifierCrosstalk.Matrix This class takes N (four for WiFSIP) single-quadrant illuminated fits files and calculates the correction matrix out of it.
static class	AmplifierCrosstalk.MatrixFit This class tries to get from real-world measurements to corrections over the entire amplifier surface.
static class	AmplifierCrosstalk.Restore This class takes the fits on the command line argument and does the amplifier correction using the properties stated.
static class	AmplifierCrosstalk.Simul Test class to generate artificial illumination fits files.

Nested classes/interfaces inherited from class util.PropertyResources

PropertyResources.URLResource

Field Summary

static double	ADULIMIT Numerical maximum ADU reachable in input image.
private double[][]	calpha If constant calibration is sufficient.
private static String	DEFCORRECTIONCONS From a series of mesured regulus images (Apr.
private static double	DEFMINILLUMINATION Default minimum illumination.
private static String	DEFOVERSPILL From science20100616A-0000EXP0005 the overspill.
static String	ILLUKEY Header key for indicating the amplifer the correction applies.
static String	ILLUREM Remark to above key.
static String	KEY_BACKGROUNDFITS If stated, the list of background images.
static String	KEY_BACKGROUNDLEVEL If stated, the double-tokenizable list of background levels.
static String	KEY_CORRECTIONCONS The ,;-list of constant correction elements, if sufficient.
static String	KEY_CORRECTIONFITS The list of the fits that hold the correction matrices.
static String	KEY_CROSSTALKFITS The list of the N single-amplifier exposed fits, each N extension.
static String	KEY_CROSSTALKSIGMA The sigmas to the N single-amplifier exposed fits, each N extension.
static String	KEY_MINILLUMINATION Minimum ADU to call pixel illuminated in an illu quadrant.
static String	KEY_OVERSPILL The ,;-list of overspill corrections for maximum overspill.
private double[][][][]	measures The calibrated measurements.
private double[][]	overspill Overspill correction for estimating correction above 65535.

Fields inherited from class util.PropertyBundles

KEY_LOCALECOUNTRY, KEY_LOCALELANGUAGE, KEY_RESOURCEBUNDLES

Fields inherited from class util.PropertyResources

KEY_NOINITONCREATE, localurl, locate, POSTFIX_DIR, POSTFIX_EXT, POSTFIX_FILE, POSTFIX_LIST, POSTFIX_URL, urlset

Fields inherited from class util.PropertyContainer

KEY_LISTSEPARATOR, KEY_MAPKEYVALUECHAR, KEY_MAPSEPARATOR

Fields inherited from interface util.ResourceSupplying

KEY_URLRESOURCES, KEY_URLUSECONFIG, KEY_URLUSECURRENT, KEY_URLUSEHOME

Fields inherited from interface util.PropertySupplying

CONFIG, KEY_CLASS

Constructor Summary

AmplifierCrosstalk(Map<String,String> prop)
Creates a new AmplifierCrosstalk.

Method Summary

private static nom.tam.fits.ImageHDU	allOnes(nom.tam.fits.BasicHDU size) Returns an image HDU with all double ones.
nom.tam.fits.Fits	applyCorrection(nom.tam.fits.Fits science) Applies the correction.
nom.tam.fits.Fits	applyLinearCorrection(nom.tam.fits.Fits science) Applies the linear correction.
static nom.tam.fits.Fits	correct(nom.tam.fits.Fits science, double[][][][] alpha, double[][] spill) Split a single science fits file into its image quadrants, correct them and convert back into a single fits object.
static nom.tam.fits.ImageHDU[]	correctQuadrants(nom.tam.fits.ImageHDU[] science, double[][][][] alpha, double[][] overspill) We apply the corrections on a single quadrant providing the four correct correction coefficients.
boolean	createCorrectionMatrix() This method is used to create the correction fits out of the crosstalk fits.
private static double	estimateOverspill(double[][] in, int x, int y) From an x,y position in a quadrant checked as source for overspill, we try to estimate the amount of overspill.
static nom.tam.fits.ImageHDU	fitBackground(nom.tam.fits.ImageHDU amplifier, nom.tam.fits.ImageHDU crstlk, nom.tam.fits.ImageHDU sigma, double minillu, Multidimensional[] model) If the input images are too noise, we can subtract a fit here.
static nom.tam.fits.ImageHDU[]	fitCorrectionMatrix(nom.tam.fits.ImageHDU corr, nom.tam.fits.ImageHDU sig, Multidimensional[] model) Normally, measurements of the crosstalk cannot be obtained across an entire quadrant, meaning that normally only a portion of the quadrant will be illuminated.
static List<nom.tam.fits.Fits>	getConversionFits(double[][][][] matrix) We store the crosstalk-correction matrices as fits files, each fits has a primary extension without data, followed by an extension per amplifier.
static double[][][][]	getCorrection(double[][][][] illu, double minadu) The input array consists of reduced illumination data.
static double[][][][]	getCrosstalk(List<nom.tam.fits.Fits> illu, List<nom.tam.fits.Fits> bias, double[][] offset) Creates the data array out of the image list, the background list, if applicable, and/or a list of constant offsets.
private List<nom.tam.fits.Fits>	getFitsList(List<String> ctf) Returns a list of fits files by either looking up the list entries at files, or, if not existing, as resources.
void	init() Always load the constant coefficients, this is no time.
private static boolean	isMax(double[][] in, int x, int y) Checks, if requested coordinate is in image range.
nom.tam.fits.ImageHDU[]	linearCorrectQuadrants(nom.tam.fits.ImageHDU[] science) We apply the corrections on a single quadrant providing the four correct correction coefficients.
static double[][][][]	loadConversionFits(List<nom.tam.fits.Fits> alpha) We load the correction matrix from alist of Fits files.
static Multidimensional[]	lowestTwoFit() Creates the model of a planar plus an xy term fit through the correction matrix.
static Multidimensional[]	parabolicFit() Parabolic fit.
static Multidimensional[]	planeFit() Creates the model of a planar fit through the correction matrix.
boolean	storeCorrectionMatrix() Stores the correction matrix in the fits files.
private static double[][]	stringMatrix(String commasemicolon) Converts a comma-semicolon matrix int a two-dimensional array.

Methods inherited from class util.PropertyBundles

clone, getLocalized, getLocalized, getLocalizedString, getLocalizedString, loadResource

Methods inherited from class util.PropertyResources

createFrom, createFrom, createFrom, getApplet, getAsResources, getLocalClassLoader, getPropertiesToKey, getPropertiesToKey, getResource, getResourceAsStream, getResourceFromKey, getResources, keyCreate, keyCreate, reload, setApplet

Methods inherited from class util.PropertyContainer

augment, augment, augment, defaultBoolean, defaultChar, defaultDouble, defaultFloat, defaultInt, defaultLong, defaultObject, defaultObject, defaultProperties, defaultProperty, getAsBoolean, getAsChar, getAsDouble, getAsEnums, getAsFloat, getAsInt, getAsList, getAsLong, getAsMap, getAsMap, getAsObject, getAsObject, getProperties, getProperty, has, isNew, parseObject, reload, removeProperty, rescanned, setObject, setProperties, setProperty, stringProperties, toString

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface util.ResourceSupplying

getResource, getResourceAsStream, getResources

Methods inherited from interface util.PropertySupplying

defaultBoolean, defaultChar, defaultDouble, defaultFloat, defaultInt, defaultLong, defaultObject, defaultObject, defaultProperties, defaultProperty, getAsBoolean, getAsChar, getAsDouble, getAsFloat, getAsInt, getAsList, getAsLong, getAsMap, getAsObject, getAsObject, getProperties, getProperty, has, parseObject, removeProperty, setObject, setProperty, stringProperties

Field Detail

ADULIMIT

public static final double ADULIMIT

Numerical maximum ADU reachable in input image.

See Also:

Constant Field Values

ILLUKEY

public static final String ILLUKEY

Header key for indicating the amplifer the correction applies.

See Also:

Constant Field Values

ILLUREM

public static final String ILLUREM

Remark to above key.

See Also:

Constant Field Values

KEY_BACKGROUNDFITS

public static final String KEY_BACKGROUNDFITS

If stated, the list of background images.

See Also:

Constant Field Values

KEY_BACKGROUNDLEVEL

public static final String KEY_BACKGROUNDLEVEL

If stated, the double-tokenizable list of background levels.

See Also:

Constant Field Values

KEY_CROSSTALKFITS

public static final String KEY_CROSSTALKFITS

The list of the N single-amplifier exposed fits, each N extension.

See Also:

Constant Field Values

KEY_CROSSTALKSIGMA

public static final String KEY_CROSSTALKSIGMA

The sigmas to the N single-amplifier exposed fits, each N extension.

See Also:

Constant Field Values

KEY_CORRECTIONFITS

public static final String KEY_CORRECTIONFITS

The list of the fits that hold the correction matrices.

See Also:

Constant Field Values

KEY_CORRECTIONCONS

public static final String KEY_CORRECTIONCONS

The ,;-list of constant correction elements, if sufficient.

See Also:

Constant Field Values

KEY_OVERSPILL

public static final String KEY_OVERSPILL

The ,;-list of overspill corrections for maximum overspill.

See Also:

Constant Field Values

KEY_MINILLUMINATION

public static final String KEY_MINILLUMINATION

Minimum ADU to call pixel illuminated in an illu quadrant.

See Also:

Constant Field Values

DEFMINILLUMINATION

private static final double DEFMINILLUMINATION

Default minimum illumination.

See Also:

Constant Field Values

DEFCORRECTIONCONS

private static final String DEFCORRECTIONCONS

From a series of mesured regulus images (Apr. 2010) the const. corr.

See Also:

Constant Field Values

DEFOVERSPILL

private static final String DEFOVERSPILL

From science20100616A-0000EXP0005 the overspill.

See Also:

Constant Field Values

measures

private double[][][][] measures

The calibrated measurements. First index amp, 2nd ext, 3+4 x&y

calpha

private double[][] calpha

If constant calibration is sufficient. This is measures without 3&4.

overspill

private double[][] overspill

Overspill correction for estimating correction above 65535.

Constructor Detail

AmplifierCrosstalk

public AmplifierCrosstalk(Map<String,String> prop)

Creates a new AmplifierCrosstalk. Depending on keys present, the entire cycle can be launched with getCrosstalk(java.util.List, java.util.List, double[][]).

Method Detail

init

public void init()

Always load the constant coefficients, this is no time.

Specified by:

init in interface Initializable

Overrides:

init in class PropertyResources

stringMatrix

private static double[][] stringMatrix(String commasemicolon)

Converts a comma-semicolon matrix int a two-dimensional array. The inner list is the second index, the outer one the first index. "1,2,3;4,5,6" would produce an array a[2][3], with a[0] = 1,2,3 and a[1] = 4,5,6

createCorrectionMatrix

public boolean createCorrectionMatrix()
                               throws nom.tam.fits.FitsException,
                                      IOException

This method is used to create the correction fits out of the crosstalk fits.

Throws:

nom.tam.fits.FitsException

IOException

storeCorrectionMatrix

public boolean storeCorrectionMatrix()
                              throws nom.tam.fits.FitsException,
                                     IOException

Stores the correction matrix in the fits files.

Throws:

nom.tam.fits.FitsException

IOException

applyCorrection

public nom.tam.fits.Fits applyCorrection(nom.tam.fits.Fits science)
                                  throws nom.tam.fits.FitsException,
                                         IOException

Applies the correction.

Throws:

nom.tam.fits.FitsException

IOException

applyLinearCorrection

public nom.tam.fits.Fits applyLinearCorrection(nom.tam.fits.Fits science)
                                        throws nom.tam.fits.FitsException,
                                               IOException

Applies the linear correction.

Throws:

nom.tam.fits.FitsException

IOException

getFitsList

private List<nom.tam.fits.Fits> getFitsList(List<String> ctf)
                                     throws nom.tam.fits.FitsException,
                                            IOException

Returns a list of fits files by either looking up the list entries at files, or, if not existing, as resources. Returns null if not found.

Throws:

nom.tam.fits.FitsException

IOException

getCrosstalk

public static double[][][][] getCrosstalk(List<nom.tam.fits.Fits> illu,
                                          List<nom.tam.fits.Fits> bias,
                                          double[][] offset)
                                   throws nom.tam.fits.FitsException,
                                          IOException

Creates the data array out of the image list, the background list, if applicable, and/or a list of constant offsets. The list of fits should have as mauch entries as there are amplifiers. Each fits should have a number of extensions that again equals the number of amplifiers. Each image should be generated with sufficient illumination of a single amplifier section, all other amplifier sections should be dark. The first fits should have the first amplifier section illuminated, i.e. in the first fits the first extension should show siginficant ADUs. The second fits should have the second amplifier section illuminated, i.e. in this fits the second extension should show significant ADUs.

Parameters:

illu - The list of individually illuminated amplifier sections.

bias - If non-null, this bias is subtracted from the illumination.

offset - If non-null subtracted from the illumination level.

Throws:

nom.tam.fits.FitsException

IOException

getCorrection

public static double[][][][] getCorrection(double[][][][] illu,
                                           double minadu)
                                    throws nom.tam.fits.FitsException,
                                           IOException

The input array consists of reduced illumination data. The first index describes the different images, thus that for each image only a single amplifier section has been illuminated. The second index corresponds to the different amplifier sections. The illumination data should consist of illuminaten of the n-th amplifier section, i.e. for index i illu[i][i] contains the truely illuminated image data. In the ideal case the illumination data will be maximum ADU in the amplifier section in question and zero in all others. The last two indices are the x/y coordinates of the individual amplifier sections. The number of illumination images and amplifiers (i.e. the first two indices) must be equal. On return, the resulting array is filled with the corresponding α* matrix, i.e. the matrix that can than be applied on science images to correct for the crosstalk.

Parameters:

minadu - If the illumination is less, ignore data.

Returns:

The inversion matrix to correct for crosstalk effects, with y/x exchanged for fits embedding.

Throws:

nom.tam.fits.FitsException

IOException

getConversionFits

public static List<nom.tam.fits.Fits> getConversionFits(double[][][][] matrix)
                                                 throws nom.tam.fits.FitsException

We store the crosstalk-correction matrices as fits files, each fits has a primary extension without data, followed by an extension per amplifier.

Throws:

nom.tam.fits.FitsException

loadConversionFits

public static double[][][][] loadConversionFits(List<nom.tam.fits.Fits> alpha)
                                         throws nom.tam.fits.FitsException,
                                                IOException

We load the correction matrix from alist of Fits files.

Throws:

nom.tam.fits.FitsException

IOException

correctQuadrants

public static nom.tam.fits.ImageHDU[] correctQuadrants(nom.tam.fits.ImageHDU[] science,
                                                       double[][][][] alpha,
                                                       double[][] overspill)
                                                throws nom.tam.fits.FitsException

We apply the corrections on a single quadrant providing the four correct correction coefficients. The returns HDU is always double.

Parameters:

alpha - correction matrix with x/y excahnged, i.e [n][n][y][x]

Throws:

nom.tam.fits.FitsException

linearCorrectQuadrants

public nom.tam.fits.ImageHDU[] linearCorrectQuadrants(nom.tam.fits.ImageHDU[] science)
                                               throws nom.tam.fits.FitsException

We apply the corrections on a single quadrant providing the four correct correction coefficients. The returns HDU is always float.

Parameters:

alpha - correction matrix with x/y excahnged, i.e [n][n][y][x]

Throws:

nom.tam.fits.FitsException

correct

public static nom.tam.fits.Fits correct(nom.tam.fits.Fits science,
                                        double[][][][] alpha,
                                        double[][] spill)
                                 throws nom.tam.fits.FitsException,
                                        IOException

Split a single science fits file into its image quadrants, correct them and convert back into a single fits object.

Throws:

nom.tam.fits.FitsException

IOException

planeFit

public static Multidimensional[] planeFit()

Creates the model of a planar fit through the correction matrix.

lowestTwoFit

public static Multidimensional[] lowestTwoFit()

Creates the model of a planar plus an xy term fit through the correction matrix.

parabolicFit

public static Multidimensional[] parabolicFit()

Parabolic fit.

allOnes

private static final nom.tam.fits.ImageHDU allOnes(nom.tam.fits.BasicHDU size)
                                            throws nom.tam.fits.FitsException

Returns an image HDU with all double ones.

Parameters:

size - The data template for size.

Throws:

nom.tam.fits.FitsException

estimateOverspill

private static double estimateOverspill(double[][] in,
                                        int x,
                                        int y)
                                 throws nom.tam.fits.FitsException

From an x,y position in a quadrant checked as source for overspill, we try to estimate the amount of overspill. Zero is returned if the value at x/y is not the maximum ADU (65535). If this pixel has 65535, we also check the surrounding 2 to each side in x/y and the four pixel diagonally adjacant to x/y. From this 13 pixel (including the one on x/y), we report the amount of pixel that have maximum ADU. This number is returned as the overspill correction.

Throws:

nom.tam.fits.FitsException

isMax

private static boolean isMax(double[][] in,
                             int x,
                             int y)
                      throws nom.tam.fits.FitsException

Checks, if requested coordinate is in image range. If so, return true, if ADU here is ADULIMIT, otherwise (or if outside) return false.

Throws:

nom.tam.fits.FitsException

fitBackground

public static nom.tam.fits.ImageHDU fitBackground(nom.tam.fits.ImageHDU amplifier,
                                                  nom.tam.fits.ImageHDU crstlk,
                                                  nom.tam.fits.ImageHDU sigma,
                                                  double minillu,
                                                  Multidimensional[] model)
                                           throws nom.tam.fits.FitsException,
                                                  IOException

If the input images are too noise, we can subtract a fit here. From a single input fits having N extensions, we grab the ImageHDU with the amplifier (given by first parameter) and the target HDU. From the target HDU, we extract all values that have an illumination *below* the KEY_MINILLUMINATION in the amplifier quadrant. To this extracted ADUs, a linear model is fitted. This model is then subtracted across the entire target HDU, returning the residuals in a separated ImageHDU. If the sigma for the input pixel is known, provide it as a ImageHDU.

Parameters:

sigma - Can be null

Throws:

nom.tam.fits.FitsException

IOException

fitCorrectionMatrix

public static nom.tam.fits.ImageHDU[] fitCorrectionMatrix(nom.tam.fits.ImageHDU corr,
                                                          nom.tam.fits.ImageHDU sig,
                                                          Multidimensional[] model)
                                                   throws nom.tam.fits.FitsException

Normally, measurements of the crosstalk cannot be obtained across an entire quadrant, meaning that normally only a portion of the quadrant will be illuminated. This method tries to fit to all valid points, i.e. those that are non-zero in the correction matrix by a smooth function using GeneralLinearRegression. The easiest model would be a plane according to m_ij = d_0+k_1*x_i+k_2*y_j, which has only three model functions

Returns:

An image hdu array with the fit and the residuals

Throws:

nom.tam.fits.FitsException

See Also:

planeFit()