Differences between revisions 45 and 99 (spanning 54 versions)
Revision 45 as of 2011-01-21 20:37:19
Size: 15281
Editor: gtang
Comment:
Revision 99 as of 2022-08-07 02:33:53
Size: 32213
Editor: SteveLudtke
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
## page was renamed from Eman2Metadata
Line 6: Line 7:
=== Native HDF5 Image Compression ===
EMAN2 supports lossless and lossy (through bit reduction) compression via native HDF5 mechanisms. To write an image with compression, the storage mode must be ''EM_COMPRESSED''. ''render_bits'', ''render_min'' and ''render_max'' should also be set (otherwise lossless compression in floating point mode will take place, which won't save much space. These parameters are not stored under these names in the file). ''render_compress_level'' may optionally be set (no quality impact, just speed/size tradeoff). Upon storage, the parameters: ''stored_rendermin'', ''stored_rendermax'' and ''stored_renderbits'' will be set, and when the file is read from disk, the values will be rescaled to match the original image, to within the limits permitted by render_bits.
Line 8: Line 12:
||Tag||Type||Description||
Line 16: Line 21:
||is_complex ||int ||Flag indicating that the image is complex (R/I or A/P pairs) ||
||is_complex_ri ||int ||Flag indicating that a complex image is R/I not A/P ||
||changecount ||int ||An integer which is incremented every time the image is marked as changed ||
||data_path ||string ||Used only in BDB files, to indicate that the binary data for an image should be read from an alternate location. Data cannot be written back to such objects. ||


==== These values are computed on the fly ====
||all_int||int||True if all values in the image are integers, otherwise false||

==== These values are computed only when requested, they are not part of get_attr_dict() ====
||Tag||Type||Description||
Line 27: Line 29:
||moment_inertia||float||Moment of inertia about the image center, considering only positive pixel values||
||radius_gyration||float||Radius of gyration about the image center, considering only positive pixel values||
Line 30: Line 34:
||Tag||Type||Description||
Line 31: Line 36:
||changecount ||int ||An integer which is incremented every time the image is marked as changed ||
Line 33: Line 39:
||class_ptcl_idxs ||tuple ||In a class-average, this is a list of particle numbers used in the final average (see class_ptcl_src) || ||class_ptcl_idxs ||tuple ||In a class-average, this is a list of particle numbers used in the final average (see class_ptcl_src and exc_class_ptcl_idxs) ||
||class_ptcl_qual ||tuple ||In a class-average, this is the mean quality of the particles vs the average ||
||class_ptcl_qual_sigma ||tuple ||In a class-average, this is the standard deviation of the quality of the particles vs the average ||
||class_qual ||float ||The similarity of the class-average to the alignment reference (usually a projection). As usual, smaller is better ||
||class_ssnr||float list||SSNR computed from the FSC between the class-average and a reference. N points equivalent to the radius(+1) of the particle||
Line 40: Line 50:
||data_n ||ing ||Used in virtual stacks. This is the image number || ||data_n ||int ||Used in virtual stacks. This is the image number ||
||datatype ||int ||Number format used when storing the file to disk, see format-specific parameters below for formats supported by each file. 0:EM_UNKNOWN, 1:EM_CHAR, 2:EM_UCHAR, 3:EM_SHORT, 4:EM_USHORT, 5:EM_INT, 6:EM_UINT, 7:EM_FLOAT, 8:EM_DOUBLE, 9:EM_SHORT_COMPLEX, 10:EM_USHORT_COMPLEX, 11:EM_FLOAT_COMPLEX,12:EM_COMPRESSED||
Line 42: Line 53:
||eval_contrast_lowres ||float ||Low-resoluton contrsast in the image (mean power from 25-200 A / mean power >20 A) ||
Line 43: Line 55:
||filter_curve ||float list ||Returned by some processors. List of floats representing an applied radial Fourier filter ||
||hadzeroedge ||int ||If set, indicates that the mask.zeroedgefill processor detected an edge in need of filling ||
||hostendian ||str ||Endianness of the computer in use. "big" or "little" ||
||imageendian ||str ||Endianness of the image. "big" or "little". Only set for image files where this is an important distinction. ||
||is_complex ||int ||Flag indicating that the image is complex (R/I or A/P pairs) ||
||is_complex_ri ||int ||Flag indicating that a complex image is R/I not A/P ||
||lst_comment ||string ||When a file is read from LSX format, any comment present will be stored here ||
Line 45: Line 64:
||microscope_voltage ||float ||Voltage of the microscope in kV ||
||microscope_cs ||float ||Cs of the microscope in mm ||
||model_id ||int ||in a projection during multi-model refinement, this is the index of the model for the current projection. For single model refinements always 1. In EMAN2.0x the first model was number 0, in EMAN2.1x the first model is number 1. ||
||movie_frames ||int ||If this is one particle from a direct detector movie, this is the total number of frames in the movie ||
||movie_n ||int ||If this is one particle from a direct detector movie, this is the sequence number within the movie (0 is the first frame as ususal) ||
||movie_cc ||float list ||Correlation for each frame in a particle movie ||
||movie_tx ||float list ||X translation for each frame in a particle movie ||
||movie_ty ||float list ||Y translation for each frame in a particle movie ||
||norm_mult ||float ||When images are normalized there is a linear transformation. This is the multiplicative component (happens first). Only some processors provide it. ||
||norm_add ||float ||When images are normalized there is a linear transformation. This is the additive component for processors that provide it. ||
Line 54: Line 83:
||render_min,max ||float ||Used when rendering an image to 8/16 bit integers. These are the values representing the minimum and maximum integer values || ||render_min,max ||float ||Used when rendering an image to integer formats. These are the values representing the minimum and maximum integer values. These values are never written to a file, but are set prior to saving. see stored_rendermin ||
||render_bits||int||Specify the number of bits to keep in integer representation of the data. Bit reduction is lossy, but will not save disk space unless saving with compression enabled. Special value -1 can be used to specify floating point for lossless compression. This is never stored in a file. See stored_renderbits||
||render_compress_level||int||Specify the compression level to use when writing compressed HDF files. Default is 1, which provides decent compression quickly. Valid range is 1-9. This value will be stored if set.||
||score_align||float||Score associated with xform.align2d or xform.align3d. Meaning varies with alignment algorithm, but smaller should be better. Not all alignment routines return this.||
Line 56: Line 88:
||sigma_presub ||float ||The sigma of the input image prior to reference subtraction, optionally returned by math.sub.optimal processor ||
||source_orig_path ||string ||When an image is copied from one file to another, this can be used to reference its original location ||
||source_orig_n ||string ||When an image is copied from one file to another, this can be used to reference its original number within the file ||
Line 57: Line 92:
||source_n ||string ||When an image is read from a file, this is set to the image number || ||source_n ||string ||When an image is read from a file, this is set to the image number (first image is 0) ||
||stored_rendermin||float ||When an image has been stored in integer mode, this value is used to restore the original floating point range when reading. Any value set for this property prior to saving will be ignored and overwritten. To set this value prior to saving, see render_min.||
||stored_rendermax||float ||When an image has been stored in integer mode, this value is used to restore the original floating point range when reading. Any value set for this property prior to saving will be ignored and overwritten. To set this value prior to saving, see render_max.||
||stored_renderbits||float ||The number of bits used when saving this file. Special value -1 can be used to specify floating point for lossless compression. To set this value prior to saving, see render_bits.||
||stored_truncated||int||Number of pixels with truncated values when saving with rendermin and rendermax||
Line 63: Line 102:
||tilt_angle || float||Tilt angle of a specific image in a tilt series, derived from recording software||
||tilt_defocus_est|| float||Estimated defocus of a tilt in a tilt series from recording software||
||tilt_dose|| float||Dose in a single tilt in a tilt series||
||tilt_dose_begin|| float||Dose expended in the tilt series at the beginning of a specific tilt image||
||tilt_seq|| int||Recorded sequence number of a single tilt in a tilt series, starting with 0||
||timestamp ||string ||When data for an image is being written this is updated with the current time. It is not updated for metadata changes, only when the image data is written ||
||symxform ||Transform ||A Transform object produced by e2symsearch.py ||
Line 71: Line 117:
||Tag||Type||Description||Comments||
Line 77: Line 124:


Line 88: Line 132:
==== Flags come form MRC file ==== ==== Parameters from MRC file ====
||Tag||Type||Description||
||datatype ||int ||pixel storage data type in EMAN format: EM_UCHAR, EM_SHORT, EM_USHORT, EM_SHORT_COMPLEX, EM_FLOAT, EM_FLOAT_COMPLEX ||
||apix_x,y,z ||float ||Angstroms per pixel on the x-axis (also _y and _z). If _y or _z are 0 or not present it will be assumed that they are the same as _x. Note that CTF objects have an independent value for A/pix ||
Line 92: Line 139:
||origin_x, y, z ||float ||image origin for x, y, z axis ||
Line 115: Line 163:
  ==== Flags come form IMAGIC file ====

==== Parameters from FEI MRC file (an extended MRC format for tomography) ====
||Tag||Type||Description||
||FEIMRC.a_tilt ||float ||Alpha tilt, in degrees ||
||FEIMRC.b_tilt ||float ||Beta tilt, in degrees ||
||FEIMRC.x_stage ||float ||Stage x position. Normally in SI units (meters), but some older files may be in micrometers. Check by looking at values for x,y,z. If one of these exceeds 1, it will be micrometers. ||
||FEIMRC.y_stage ||float ||Stage y position. For testing of units see x_stage. ||
||FEIMRC.z_stage ||float ||Stage z position. For testing of units see x_stage. ||
||FEIMRC.x_shift ||float ||Image shift x. For testing of units see x_stage. ||
||FEIMRC.y_shift ||float ||Image shift y. For testing of units see x_stage. ||
||FEIMRC.defocus ||float ||Defocus as read from microscope. For testing of units see x_stage. ||
||FEIMRC.exp_time ||float ||Exposure time in seconds. ||
||FEIMRC.mean_int ||float ||Mean value of image. ||
||FEIMRC.tilt_axis ||float ||The orientation of the tilt axis in the image in degrees. Vertical to the top is 0=B0, the direction of positive rotation is anti-clockwise. ||
||FEIMRC.pixel_axis ||float ||The pixel size of the images in SI units (meters). ||
||FEIMRC.magnification ||float ||The magnification used for recording the images. ||
||FEIMRC.ht ||float ||Value of the high tension in SI units. (volts) ||
||FEIMRC.binning ||float ||The binning of the CCD or STEM acquisition. ||
||FEIMRC.appliedDefocus ||float ||The intended application defocus in SI units (meters), as defined for example in the tomography parameter view. ||
||FEIMRC.remainder ||float(x128) ||Not used. ||

==== Parameters from EER file ====
||Tag||Type||Description||
||EER.compression ||int ||65001 for EER compressed data 7-bit, COMPRESSION_NONE (1) for final image data ||
||EER.exposure_time ||float ||Exposure time of the acquisition in s ||
||EER.total_dose ||float ||Average amount of electrons per pixel in the entire acquisition in e/pixel ||
||EER.mean_dose_rate ||float ||Average amount of electrons per pixel per second during the acquisition in e/pixel/s ||
||EER.number_of_frames ||int ||Total number of frames in the acquisition ||
||EER.sensor_image_height ||int ||Height of the individual images ||
||EER.sensor_image_width ||int ||Width of the individual images ||



==== Parameters from IMAGIC file ====
||Tag||Type||Description||
||datatype ||int ||pixel storage data type in EMAN format: EM_UCHAR, EM_USHORT, EM_FLOAT, EM_FLOAT_COMPLEX ||
Line 134: Line 217:
||ptcl_repr ||int ||raw images represented by this image ||
||orientation_convention ||string ||orientation convention ||
||euler_alt ||float ||euler angle alt ||
||euler_az ||float ||euler angle az ||
||euler_phi ||float ||euler angle phi ||

==== Flags come form SPIDER file ====
||ptcl_repr ||int ||raw images represented by this image. Note: non-standard use ||
||xform.projection ||Transform ||particle orientation, set from the orientation flags(alt, az, phi) in the IMAGIC header ||
||xform.align3d ||Transform ||particle orientation for 3D image, set from the orientation flags(alt, az, phi) in the IMAGIC header ||


==== Parameters from IMAGIC2 file ====
||Tag||Type||Description||
||datatype ||int ||pixel storage data type in EMAN format: EM_UCHAR, EM_USHORT, EM_FLOAT, EM_FLOAT_COMPLEX ||
||IMAGIC.imgnum ||int ||image location number (1,2,3,...) ||
||IMAGIC.count ||int ||total # images in file (1st record only), total number of images - 1 (0,1,2,...) ||
||IMAGIC.error ||int ||error code for this image during IMAGIC run ||
||IMAGIC.month ||int ||creation month ||
||IMAGIC.day ||int ||creation day ||
||IMAGIC.year ||int ||creation year ||
||IMAGIC.hour ||int ||creation hour ||
||IMAGIC.minute ||int ||creation minute ||
||IMAGIC.sec ||int ||creation second ||
||IMAGIC.rsize ||int ||image size in bytes ||
||IMAGIC.izold ||int ||top left Z co-ordinate before THREED-CUT ||
||IMAGIC.type ||char(4) ||4 characters determining the image type; only 'REAL', or 'INTG' are implemented for EMAN ||
||IMAGIC.ixold ||int ||top left X co-ordinate before CUT-IMAGE (boxing) ||
||IMAGIC.iyold ||int ||top left Y co-ordinate before CUT-IMAGE (boxing) ||
||IMAGIC.complex ||int ||label indicating that data is always complex ||
||IMAGIC.defocus1 ||float ||defocus value 1 ||
||IMAGIC.defocus2 ||float ||defocus value 2 ||
||IMAGIC.defangle ||float ||defocus angle ||
||IMAGIC.sinostart ||float ||start angle if image is a sinogram ||
||IMAGIC.sinoend ||float ||end angle if image is a sinogram ||
||IMAGIC.label ||char(80) ||coded NAME/TITLE of the image ||
||IMAGIC.ccc3d ||float ||3D similarity criteria ||
||IMAGIC.ref3d ||int ||3D membership ||
||IMAGIC.mident ||int ||micrograph identification number ||
||IMAGIC.ezshift ||int ||equivalent shift in Z direction ||
||IMAGIC.ealpha ||int ||equivalent Euler angle alpha ||
||IMAGIC.ebeta ||int ||equivalent Euler angle beta ||
||IMAGIC.egamma ||int ||equivalent Euler angle gamma ||
||IMAGIC.nalisum ||int ||number of image summed ||
||IMAGIC.pgroup ||int ||point-group symmetry in international notation (622, for example) ||
||IMAGIC.i4lp ||int ||number of objects in file ||
||IMAGIC.alpha ||float ||Euler angle alpha (3D and Angular Reconst. ||
||IMAGIC.beta ||float ||Euler angle beta (3D and Angular Reconst.) ||
||IMAGIC.gamma ||float ||Euler angle gamma (3D and Angular Reconst.) ||
||IMAGIC.imavers ||int ||IMAGIC version, which created the file (yyyymmdd) ||
||IMAGIC.realtype ||int ||floating point type, machine stamp ||
||IMAGIC.angle ||float ||last rotation angle ||
||IMAGIC.voltage ||float ||acceleration voltage (kv) ||
||IMAGIC.spaberr ||int ||sperical aberration (mm) ||
||IMAGIC.pcoher ||int ||partial coherence ||
||IMAGIC.ccc ||float ||cross correlation peak hight ||
||IMAGIC.errar ||float ||error in angular reconstitution, if -1.0: the file is a special file (FABOSA) ||
||IMAGIC.err3d ||float ||error in 3D reconstruction ||
||IMAGIC.ref ||int ||(multi-) reference number ||
||IMAGIC.classno ||float ||class number in MSA classification ||
||IMAGIC.locold ||float ||location number before CUT-IMAGE (boxing), or before copy in ANG-RECONST and EX-COPY ||
||IMAGIC.repqual ||float ||representation quality, used in MSA-RUN and MSA (eigen) filtering ||
||IMAGIC.zshift ||float ||last shift in Z direction ||
||IMAGIC.xshift ||float ||last shift in X direction ||
||IMAGIC.yshift ||float ||last shift in Y direction ||
||IMAGIC.numcls ||float ||number members in the class specified in CLASSNO, if this image represents a class average (class-sum image) ||
||IMAGIC.ovqual ||float ||overall quality of the class in CLASSNO ||
||IMAGIC.eangle ||float ||equivalent angle ||
||IMAGIC.exshift ||float ||equivalent shift in X direction ||
||IMAGIC.eyshift ||float ||equivalent shift in Y direction ||
||IMAGIC.cmtotvar ||float ||total variance in data matrix relative to center of mass (MSA calculations) ||
||IMAGIC.informat ||float ||Gauss norm / real*FT Space information of the data set ||
||IMAGIC.numeigen ||int ||number of eigen values in MSA ||
||IMAGIC.niactive ||int ||number of active images in MSA calculations ||
||IMAGIC.fabosa1 ||float ||Euler angle alpha (from projection matching), Special FABOSA variables if DAT1(105) = -1.0 ||
||IMAGIC.fabosa2 ||float ||Euler angle beta (from projection matching), Special FABOSA variables if DAT1(105) = -1.0 ||
||IMAGIC.fabosa3 ||float ||Euler angle gamma (from projection matching), Special FABOSA variables if DAT1(105) = -1.0 ||
||IMAGIC.alpha2 ||float ||Euler angle alpha (from projection matching), Special FABOSA variables if DAT1(105) = -1.0 ||
||IMAGIC.beta2 ||float ||Euler angle beta (from projection matching), Special FABOSA variables if DAT1(105) = -1.0 ||
||IMAGIC.gamma2 ||float ||Euler angle gamma (from projection matching), Special FABOSA variables if DAT1(105) = -1.0 ||
||IMAGIC.nmetric ||float ||Metric used in MSA calculations ||
||IMAGIC.actmsa ||float ||a flag indicating whether the "image" is active or not. Used during MSA calculations ||
||IMAGIC.coosmsa ||float(69) ||co-ordinates of "image" along factorial axis ||
||IMAGIC.history ||char(228) ||coded history of image (228 characters) ||
||ptcl_repr ||int ||raw images represented by this image. Note: non-standard use ||
||xform.projection ||Transform ||particle orientation, set from the orientation flags(alt, az, phi) in the IMAGIC header ||
||xform.align3d ||Transform ||particle orientation for 3D image, set from the orientation flags(alt, az, phi) in the IMAGIC header ||


==== Parameters from SPIDER file ====
||Tag||Type||Description||
||datatype ||int ||pixel storage data type in EMAN format: EM_FLOAT ||
Line 168: Line 329:

==== Flags come form EMIM file ====
||pixel ||float ||pixel/voxel size in anstrom ||
||micrograph_id ||int ||micrograph number or CCD frame ||

==== Flags come form PGM file ====
||max_gray ||int ||maximum value for grey level ||
||min_gray ||int ||minimum value for grey level ||

==== Flags come form SAL file ====
||pixel ||float ||pixel size ||

==== Flags come form TIFF file ====
||bitspersample ||unsigned short ||bits per pixel sample ||
||resolution_x ||float ||x dimension resolution ||
||resolution_y ||float ||y dimension resolution ||

==== Flags come form XPLOR file ====
||xform.projection ||Transform ||particle orientation, set from the orientation flags(phi, theta, psi, tx, ty, tz, scale) in the SPIDER header ||
||xform.align3d ||Transform ||particle orientation for 3D image, set from the orientation flags(phi, theta, psi, tx, ty, tz, scale) in the SPIDER header ||


==== Parameters from PGM file ====
||Tag||Type||Description||
||PGM.max_gray ||int ||maximum value for grey level ||
||PGM.min_gray ||int ||minimum value for grey level ||


==== Parameters from SAL file ====
||Tag||Type||Description||
||datatype ||int ||pixel storage data type in EMAN format: EM_SHORT ||
||SAL.pixel ||float ||pixel size ||


==== Parameters from TIFF file ====
||Tag||Type||Description||
||datatype ||int ||pixel storage data type in EMAN format: EM_UCHAR, EM_USHORT, EM_FLOAT ||
||TIFF.bitspersample ||unsigned short ||bits per pixel sample ||
||TIFF.resolution_x ||float ||x dimension resolution ||
||TIFF.resolution_y ||float ||y dimension resolution ||


==== Parameters from Gatan DM3 files ====
||Tag||Type||Description||
||DM3.acq_date ||string ||Acquisition date ||
||DM3.acq_time ||string ||Acquisition time ||
||DM3.actual_mag ||float ||Calibrated magnification ||
||DM3.antiblooming ||int || ||
||DM3.binning_x ||int ||Binning (X) ||
||DM3.binning_y ||int ||Binning (Y) ||
||DM3.camera_x ||int ||Camera size (X) ||
||DM3.camera_y ||int ||Camera size (Y) ||
||DM3.cs ||float ||Microscope Cs ||
||DM3.exposure_number ||int ||Camera exposure number ||
||DM3.exposure_time ||float ||Exposure time ||
||DM3.frame_type ||string ||Frame type ||
||DM3.indicated_mag ||float ||Indicated magnification ||
||DM3.name ||string ||Filename ||
||DM3.pixel_size ||float ||Pixel size (microns) ||
||DM3.source ||string ||Camera name ||
||DM3.voltage ||float ||Microscope voltage ||
||DM3.zoom ||float || ||


==== Parameters from XPLOR file ====
||Tag||Type||Description||
||apix_x,y,z ||float ||Angstroms per pixel on the x-axis (also _y and _z). If _y or _z are 0 or not present it will be assumed that they are the same as _x. Note that CTF objects have an independent value for A/pix ||

Parameters/Metadata stored in EMData Objects

The EMData object, and its representation on disk in the BDB local database, XML files, and when serialized in Python using 'pickle', supports the concept of arbitrary header parameters also known as metadata. This metadata are key/value pairs. The keys are always simple ascii text, and the values may be virtually anything represented as an EMObject, including simple ints, floats and strings, as well as more complicated objects such as Transform classes. If an EMData object is stored in some fashion other than the 3 mechanisms above, some loss of metadata is almost guaranteed. The file i/o objects will try to preserve some of the basic metadata, but most of the cryoEM formats simply don't support arbitrary header-data. The EMAN2 convention is to use BDB for most internal purposes, and to use HDF5 for data transfer/exchange.

This page will serve as a repository for the officially supported parameter key/value pairs. While you are free to set any metadata keys/values you like in an EMData object, the names listed here may be interpreted in specific ways by specific modules within EMAN, so it would be unwise to abuse them. Also, if you make up your own name for some purpose, it wouldn't hurt to register it here in the 'unofficial' section, to avoid it being used by someone else for a different purpose.

Native HDF5 Image Compression

EMAN2 supports lossless and lossy (through bit reduction) compression via native HDF5 mechanisms. To write an image with compression, the storage mode must be EM_COMPRESSED. render_bits, render_min and render_max should also be set (otherwise lossless compression in floating point mode will take place, which won't save much space. These parameters are not stored under these names in the file). render_compress_level may optionally be set (no quality impact, just speed/size tradeoff). Upon storage, the parameters: stored_rendermin, stored_rendermax and stored_renderbits will be set, and when the file is read from disk, the values will be rescaled to match the original image, to within the limits permitted by render_bits.

Special tags (read-only for getting image info)

These values are cached and only recomputed if the image changes

Tag

Type

Description

nx,ny,nz

int

Dimensions of the image, also available as get_xsize(),etc. Note: Assignments such as e["nx"] = 30 are an (not preferred) alternative to set_size(nx,ny,nz)

minimum

float

Smallest value in the image

maximum

float

Largest value in the image

mean

float

The average pixel value in the image

sigma

float

The standard deviation of the pixel values in the image

square_sum

float

Sum of the squares of the pixel values

mean_nonzero

float

The mean value of all nonzero pixels

sigma_nonzero

float

The standard deviation of the pixels ignoring pixels which are zero

all_int

int

True if all values in the image are integers, otherwise false

These values are computed only when requested, they are not part of get_attr_dict()

Tag

Type

Description

skewness

float

Skewness of the pixel values

kurtosis

float

Kurtosis of the pixel values

median

float

Median value of the pixel values

nonzero_median

float

Median value of nonzero pixels

moment_inertia

float

Moment of inertia about the image center, considering only positive pixel values

radius_gyration

float

Radius of gyration about the image center, considering only positive pixel values

Official tags (used in EMAN2/SPARX as distributed):

Tag

Type

Description

apix_x,y,z

float

Angstroms per pixel on the x-axis (also _y and _z). If _y or _z are 0 or not present it will be assumed that they are the same as _x. Note that CTF objects have an independent value for A/pix

changecount

int

An integer which is incremented every time the image is marked as changed

class_id

int

Set by classification routines to indicate which class number the particle is in

class_ptcl_src

string

In a class-average, this is the file containing the raw images used to create the average

class_ptcl_idxs

tuple

In a class-average, this is a list of particle numbers used in the final average (see class_ptcl_src and exc_class_ptcl_idxs)

class_ptcl_qual

tuple

In a class-average, this is the mean quality of the particles vs the average

class_ptcl_qual_sigma

tuple

In a class-average, this is the standard deviation of the quality of the particles vs the average

class_qual

float

The similarity of the class-average to the alignment reference (usually a projection). As usual, smaller is better

class_ssnr

float list

SSNR computed from the FSC between the class-average and a reference. N points equivalent to the radius(+1) of the particle

ctf

Ctf subclass

A subclass of Ctf containing all CTF parameters

ctf_phase_flipped

bool

Set to true if the CTF phases have been flipped

ctf_wiener_filtered

bool

Set to true if a Wiener filter has been applied

ctf_snr_total

float list

Set in class-averages by some averagers indicating the total estimated radial SNR of the average

data_path

string

Used for virtual stacks. References binary file and location for image data as file*location

data_source

string

Used in virtual stacks. This is a reference back to the source image from which this image was derived

data_n

int

Used in virtual stacks. This is the image number

datatype

int

Number format used when storing the file to disk, see format-specific parameters below for formats supported by each file. 0:EM_UNKNOWN, 1:EM_CHAR, 2:EM_UCHAR, 3:EM_SHORT, 4:EM_USHORT, 5:EM_INT, 6:EM_UINT, 7:EM_FLOAT, 8:EM_DOUBLE, 9:EM_SHORT_COMPLEX, 10:EM_USHORT_COMPLEX, 11:EM_FLOAT_COMPLEX,12:EM_COMPRESSED

eigval

float

Eigenvalue, only set for images which represent Eigenvectors

eval_contrast_lowres

float

Low-resoluton contrsast in the image (mean power from 25-200 A / mean power >20 A)

exc_class_ptcl_idxs

tuple

In a class-average, this is a list of particle numbers provided to the averager, but excluded from the final average (see class_ptcl_src)

filter_curve

float list

Returned by some processors. List of floats representing an applied radial Fourier filter

hadzeroedge

int

If set, indicates that the mask.zeroedgefill processor detected an edge in need of filling

hostendian

str

Endianness of the computer in use. "big" or "little"

imageendian

str

Endianness of the image. "big" or "little". Only set for image files where this is an important distinction.

is_complex

int

Flag indicating that the image is complex (R/I or A/P pairs)

is_complex_ri

int

Flag indicating that a complex image is R/I not A/P

lst_comment

string

When a file is read from LSX format, any comment present will be stored here

match_n

int

used to represent the number of a reference particle this particle best matched

match_qual

float

used to represent the quality associated with match_n, smaller is a better match

microscope_voltage

float

Voltage of the microscope in kV

microscope_cs

float

Cs of the microscope in mm

model_id

int

in a projection during multi-model refinement, this is the index of the model for the current projection. For single model refinements always 1. In EMAN2.0x the first model was number 0, in EMAN2.1x the first model is number 1.

movie_frames

int

If this is one particle from a direct detector movie, this is the total number of frames in the movie

movie_n

int

If this is one particle from a direct detector movie, this is the sequence number within the movie (0 is the first frame as ususal)

movie_cc

float list

Correlation for each frame in a particle movie

movie_tx

float list

X translation for each frame in a particle movie

movie_ty

float list

Y translation for each frame in a particle movie

norm_mult

float

When images are normalized there is a linear transformation. This is the multiplicative component (happens first). Only some processors provide it.

norm_add

float

When images are normalized there is a linear transformation. This is the additive component for processors that provide it.

projection_image

string

In a class-average, this represents the image file which was used for initial alignment references

projection_image_idx

string

In a class-average, this represents the specific image number in projection_image

ptcl_repr

int

If an image/volume represents the combination of one or more other images, this is the count of the number of particles that went into the average

ptcl_helix_coords

tuple

The two endpoints and a box width that defines a helix box (x1, y1, x2, y2, box_width)

ptcl_source_coord

tuple

The central coordinate of a boxed particle in terms of its source image, normally (x,y), may be (x,y,z) for subtomograms

ptcl_source_image

string

The name of the image from which the particle was extracted. Full path, may be in bdb syntax

reconstruct_norm

float

Normalization factor applied to a single projection/class-average during reconstruction

reconstruct_qual

float

Quality of a single projection/class-average relative to others during reconstruction. Unlike with comparators, larger values are better.

reconstruct_preproc

bool

Set if the image has been preprocessed for use with a reconstructor

render_min,max

float

Used when rendering an image to integer formats. These are the values representing the minimum and maximum integer values. These values are never written to a file, but are set prior to saving. see stored_rendermin

render_bits

int

Specify the number of bits to keep in integer representation of the data. Bit reduction is lossy, but will not save disk space unless saving with compression enabled. Special value -1 can be used to specify floating point for lossless compression. This is never stored in a file. See stored_renderbits

render_compress_level

int

Specify the compression level to use when writing compressed HDF files. Default is 1, which provides decent compression quickly. Valid range is 1-9. This value will be stored if set.

score_align

float

Score associated with xform.align2d or xform.align3d. Meaning varies with alignment algorithm, but smaller should be better. Not all alignment routines return this.

segment_centers

float list

Used when a volume has been segmented into regions. Set of 3*nregions floats in x1,y1,z1,x2,y2,z2,... order, indicating the center of each region as defined by the specific algorithm

sigma_presub

float

The sigma of the input image prior to reference subtraction, optionally returned by math.sub.optimal processor

source_orig_path

string

When an image is copied from one file to another, this can be used to reference its original location

source_orig_n

string

When an image is copied from one file to another, this can be used to reference its original number within the file

source_path

string

When an image is read from a file, this is set to the filename

source_n

string

When an image is read from a file, this is set to the image number (first image is 0)

stored_rendermin

float

When an image has been stored in integer mode, this value is used to restore the original floating point range when reading. Any value set for this property prior to saving will be ignored and overwritten. To set this value prior to saving, see render_min.

stored_rendermax

float

When an image has been stored in integer mode, this value is used to restore the original floating point range when reading. Any value set for this property prior to saving will be ignored and overwritten. To set this value prior to saving, see render_max.

stored_renderbits

float

The number of bits used when saving this file. Special value -1 can be used to specify floating point for lossless compression. To set this value prior to saving, see render_bits.

stored_truncated

int

Number of pixels with truncated values when saving with rendermin and rendermax

subvolume_x0,y0,z0

int

Used when the EMData stores only a portion of a larger image in certain contexts (notably direct Fourier inversion. This represents the location of the origin of 'this' in the larger virtual volume

subvolume_full_nx,ny,nz

int

Used with subvolume_x0,... Specifies the size of the virtual volume that 'this' is a part of

threed_ptcl_src

string

In a 3-D map, this is the file containing the raw images used to create the reconstruction

threed_ptcl_idxs

tuple

In a 3-D map, this is a list of particle numbers used in the final reconstruction (see threed_ptcl_src)

threed_excl_ptcl_idxs

tuple

In a 3-D map, this is a list of particle numbers excluded from the final map (see threed_ptcl_src)

tilt_angle

float

Tilt angle of a specific image in a tilt series, derived from recording software

tilt_defocus_est

float

Estimated defocus of a tilt in a tilt series from recording software

tilt_dose

float

Dose in a single tilt in a tilt series

tilt_dose_begin

float

Dose expended in the tilt series at the beginning of a specific tilt image

tilt_seq

int

Recorded sequence number of a single tilt in a tilt series, starting with 0

timestamp

string

When data for an image is being written this is updated with the current time. It is not updated for metadata changes, only when the image data is written

symxform

Transform

A Transform object produced by e2symsearch.py

xform.projection

Transform

A Transform object used by Projectors. It is applied to a 3-D model prior to projecting by summing along Z. The inverse of this Transform is used by Reconstructors

xform.align2d

Transform

A Transform object representing a 2-D transformation used to align this EMData object to a reference in 2-D

xform.align3d

Transform

A Transform object representing a 3-D transformation used to align this (3-D) EMData object to a (3-D) reference

Proposed new Official tags (comments welcome):

Tag

Type

Description

Comments

apix_scan

float

Scan pixel size in Angstroms

(there are already parameters for apix_x/y/z above. Is this different in some way ?, you could provide this as supplementary information, but apix_x/y/z remain the 'official' values. Is this what you want ?)

box_location

int_array

4 values, x0, y0, xsize, ysize representing the location of the particle in the original (box_source) image

(Pawel suggested this be changed to the reduced image, because normally we first reduce the micrograph, the window it.) (No, the box location should be in the coordinates of the referenced 'parent' image. If you want to reference a reduced image you can, but I think it makes a lot more sense to provide the capability of refining the coordinates from the reduced image when you return to the original image.) Pawel: the coordinates in boxer refer to the image from windowing was done. This is is (or can be) a reduced micrograph. There is more confusion here: name apix_scan suggests this is pixel size of the scan, NOT THE REDUCED MICROGRAPH FROM WHICH PARTICLES WERE WINDOWED. Incidentally, both pixel sizes are needed and have to be in the header. Steve: This differs from the philosophy of e2boxer, which is that you always provide e2boxer with the original image, it may internally downscale it for boxing, but the final box positions are in terms of the original image. I don't understand why you would want to externally downscale, then have to rescale the coordinates again later ? This is very messy. Anyway, any single image has a particluar A/pix value associated with it. I have no objections to something like apix_original, but what exactly is the point ? You aren't likely to rescale reduced images back to their original size...

box_score

float

A value representing the relative quality (meaning may vary) of this particle compared to others

box_source

string

Filename (not full path) of the image this particle was extracted from

box_source_id

string

Other (database) identifier of the image the raw data came from if available

Unofficial tags (To prevent reuse, used by someone in their own code or for testing):

Tags used to store header information derived from other file formats:

Parameters from MRC file

Tag

Type

Description

datatype

int

pixel storage data type in EMAN format: EM_UCHAR, EM_SHORT, EM_USHORT, EM_SHORT_COMPLEX, EM_FLOAT, EM_FLOAT_COMPLEX

apix_x,y,z

float

Angstroms per pixel on the x-axis (also _y and _z). If _y or _z are 0 or not present it will be assumed that they are the same as _x. Note that CTF objects have an independent value for A/pix

MRC.minimum

float

Minimum density value

MRC.maximum

float

Maximum density value

MRC.mean

float

Mean density value

origin_x, y, z

float

image origin for x, y, z axis

MRC.nxstart

int

No. of first column in map

MRC.nystart

int

No. of first row in map

MRC.nzstart

int

No. of first section in map

MRC.mx

int

Number of intervals along X

MRC.my

int

Number of intervals along Y

MRC.mz

int

Number of intervals along Z

MRC.nx

int

number of columns

MRC.ny

int

number of rows

MRC.nz

int

number of sections

MRC.xlen

float

Cell dimensions (Angstroms)

MRC.ylen

float

Cell dimensions (Angstroms)

MRC.zlen

float

Cell dimensions (Angstroms)

MRC.alpha

float

Cell angles (Degrees)

MRC.beta

float

Cell angles (Degrees)

MRC.gamma

float

Cell angles (Degrees)

MRC.mapc

int

Which axis corresponds to Columns

MRC.mapr

int

Which axis corresponds to Rows

MRC.maps

int

Which axis corresponds to Sections

MRC.ispg

int

Space group number (0 for images)

MRC.nsymbt

int

Number of chars used for storing symmetry operators

MRC.machinestamp

int

machine stamp in CCP4 convention: big endian=0x11110000 little endian=0x44440000

MRC.rms

float

rms deviation of map from mean density

MRC.nlabels

int

Number of labels being used

Parameters from FEI MRC file (an extended MRC format for tomography)

Tag

Type

Description

FEIMRC.a_tilt

float

Alpha tilt, in degrees

FEIMRC.b_tilt

float

Beta tilt, in degrees

FEIMRC.x_stage

float

Stage x position. Normally in SI units (meters), but some older files may be in micrometers. Check by looking at values for x,y,z. If one of these exceeds 1, it will be micrometers.

FEIMRC.y_stage

float

Stage y position. For testing of units see x_stage.

FEIMRC.z_stage

float

Stage z position. For testing of units see x_stage.

FEIMRC.x_shift

float

Image shift x. For testing of units see x_stage.

FEIMRC.y_shift

float

Image shift y. For testing of units see x_stage.

FEIMRC.defocus

float

Defocus as read from microscope. For testing of units see x_stage.

FEIMRC.exp_time

float

Exposure time in seconds.

FEIMRC.mean_int

float

Mean value of image.

FEIMRC.tilt_axis

float

The orientation of the tilt axis in the image in degrees. Vertical to the top is 0=B0, the direction of positive rotation is anti-clockwise.

FEIMRC.pixel_axis

float

The pixel size of the images in SI units (meters).

FEIMRC.magnification

float

The magnification used for recording the images.

FEIMRC.ht

float

Value of the high tension in SI units. (volts)

FEIMRC.binning

float

The binning of the CCD or STEM acquisition.

FEIMRC.appliedDefocus

float

The intended application defocus in SI units (meters), as defined for example in the tomography parameter view.

FEIMRC.remainder

float(x128)

Not used.

Parameters from EER file

Tag

Type

Description

EER.compression

int

65001 for EER compressed data 7-bit, COMPRESSION_NONE (1) for final image data

EER.exposure_time

float

Exposure time of the acquisition in s

EER.total_dose

float

Average amount of electrons per pixel in the entire acquisition in e/pixel

EER.mean_dose_rate

float

Average amount of electrons per pixel per second during the acquisition in e/pixel/s

EER.number_of_frames

int

Total number of frames in the acquisition

EER.sensor_image_height

int

Height of the individual images

EER.sensor_image_width

int

Width of the individual images

Parameters from IMAGIC file

Tag

Type

Description

datatype

int

pixel storage data type in EMAN format: EM_UCHAR, EM_USHORT, EM_FLOAT, EM_FLOAT_COMPLEX

IMAGIC.imgnum

int

image number, index from 1 to n

IMAGIC.count

int

total number of images - 1

IMAGIC.error

int

Error code for this image

IMAGIC.headrec

int

number of header records/image (always 1)

IMAGIC.mday

int

image creation date

IMAGIC.month

int

image creation month

IMAGIC.year

int

image creation year

IMAGIC.hour

int

image creation hour

IMAGIC.minute

int

image creation minute

IMAGIC.sec

int

image creation second

IMAGIC.reals

int

image size in reals

IMAGIC.pixels

int

image size in pixels

IMAGIC.type

char(4)

PACK, INTG, REAL, COMP, RECO

IMAGIC.ixold

int

Top left X-coord. in image before windowing

IMAGIC.iyold

int

Top left Y-coord. in image before windowing

IMAGIC.oldav

float

old average density

IMAGIC.label

char(80)

image id string

ptcl_repr

int

raw images represented by this image. Note: non-standard use

xform.projection

Transform

particle orientation, set from the orientation flags(alt, az, phi) in the IMAGIC header

xform.align3d

Transform

particle orientation for 3D image, set from the orientation flags(alt, az, phi) in the IMAGIC header

Parameters from IMAGIC2 file

Tag

Type

Description

datatype

int

pixel storage data type in EMAN format: EM_UCHAR, EM_USHORT, EM_FLOAT, EM_FLOAT_COMPLEX

IMAGIC.imgnum

int

image location number (1,2,3,...)

IMAGIC.count

int

total # images in file (1st record only), total number of images - 1 (0,1,2,...)

IMAGIC.error

int

error code for this image during IMAGIC run

IMAGIC.month

int

creation month

IMAGIC.day

int

creation day

IMAGIC.year

int

creation year

IMAGIC.hour

int

creation hour

IMAGIC.minute

int

creation minute

IMAGIC.sec

int

creation second

IMAGIC.rsize

int

image size in bytes

IMAGIC.izold

int

top left Z co-ordinate before THREED-CUT

IMAGIC.type

char(4)

4 characters determining the image type; only 'REAL', or 'INTG' are implemented for EMAN

IMAGIC.ixold

int

top left X co-ordinate before CUT-IMAGE (boxing)

IMAGIC.iyold

int

top left Y co-ordinate before CUT-IMAGE (boxing)

IMAGIC.complex

int

label indicating that data is always complex

IMAGIC.defocus1

float

defocus value 1

IMAGIC.defocus2

float

defocus value 2

IMAGIC.defangle

float

defocus angle

IMAGIC.sinostart

float

start angle if image is a sinogram

IMAGIC.sinoend

float

end angle if image is a sinogram

IMAGIC.label

char(80)

coded NAME/TITLE of the image

IMAGIC.ccc3d

float

3D similarity criteria

IMAGIC.ref3d

int

3D membership

IMAGIC.mident

int

micrograph identification number

IMAGIC.ezshift

int

equivalent shift in Z direction

IMAGIC.ealpha

int

equivalent Euler angle alpha

IMAGIC.ebeta

int

equivalent Euler angle beta

IMAGIC.egamma

int

equivalent Euler angle gamma

IMAGIC.nalisum

int

number of image summed

IMAGIC.pgroup

int

point-group symmetry in international notation (622, for example)

IMAGIC.i4lp

int

number of objects in file

IMAGIC.alpha

float

Euler angle alpha (3D and Angular Reconst.

IMAGIC.beta

float

Euler angle beta (3D and Angular Reconst.)

IMAGIC.gamma

float

Euler angle gamma (3D and Angular Reconst.)

IMAGIC.imavers

int

IMAGIC version, which created the file (yyyymmdd)

IMAGIC.realtype

int

floating point type, machine stamp

IMAGIC.angle

float

last rotation angle

IMAGIC.voltage

float

acceleration voltage (kv)

IMAGIC.spaberr

int

sperical aberration (mm)

IMAGIC.pcoher

int

partial coherence

IMAGIC.ccc

float

cross correlation peak hight

IMAGIC.errar

float

error in angular reconstitution, if -1.0: the file is a special file (FABOSA)

IMAGIC.err3d

float

error in 3D reconstruction

IMAGIC.ref

int

(multi-) reference number

IMAGIC.classno

float

class number in MSA classification

IMAGIC.locold

float

location number before CUT-IMAGE (boxing), or before copy in ANG-RECONST and EX-COPY

IMAGIC.repqual

float

representation quality, used in MSA-RUN and MSA (eigen) filtering

IMAGIC.zshift

float

last shift in Z direction

IMAGIC.xshift

float

last shift in X direction

IMAGIC.yshift

float

last shift in Y direction

IMAGIC.numcls

float

number members in the class specified in CLASSNO, if this image represents a class average (class-sum image)

IMAGIC.ovqual

float

overall quality of the class in CLASSNO

IMAGIC.eangle

float

equivalent angle

IMAGIC.exshift

float

equivalent shift in X direction

IMAGIC.eyshift

float

equivalent shift in Y direction

IMAGIC.cmtotvar

float

total variance in data matrix relative to center of mass (MSA calculations)

IMAGIC.informat

float

Gauss norm / real*FT Space information of the data set

IMAGIC.numeigen

int

number of eigen values in MSA

IMAGIC.niactive

int

number of active images in MSA calculations

IMAGIC.fabosa1

float

Euler angle alpha (from projection matching), Special FABOSA variables if DAT1(105) = -1.0

IMAGIC.fabosa2

float

Euler angle beta (from projection matching), Special FABOSA variables if DAT1(105) = -1.0

IMAGIC.fabosa3

float

Euler angle gamma (from projection matching), Special FABOSA variables if DAT1(105) = -1.0

IMAGIC.alpha2

float

Euler angle alpha (from projection matching), Special FABOSA variables if DAT1(105) = -1.0

IMAGIC.beta2

float

Euler angle beta (from projection matching), Special FABOSA variables if DAT1(105) = -1.0

IMAGIC.gamma2

float

Euler angle gamma (from projection matching), Special FABOSA variables if DAT1(105) = -1.0

IMAGIC.nmetric

float

Metric used in MSA calculations

IMAGIC.actmsa

float

a flag indicating whether the "image" is active or not. Used during MSA calculations

IMAGIC.coosmsa

float(69)

co-ordinates of "image" along factorial axis

IMAGIC.history

char(228)

coded history of image (228 characters)

ptcl_repr

int

raw images represented by this image. Note: non-standard use

xform.projection

Transform

particle orientation, set from the orientation flags(alt, az, phi) in the IMAGIC header

xform.align3d

Transform

particle orientation for 3D image, set from the orientation flags(alt, az, phi) in the IMAGIC header

Parameters from SPIDER file

Tag

Type

Description

datatype

int

pixel storage data type in EMAN format: EM_FLOAT

SPIDER.nslice

int

number of slices in volume; 1 for a 2D image

SPIDER.type

int

file type

SPIDER.irec

float

total number of records in the file (unused)

SPIDER.angvalid

int

1 if tilt angles have been computed

SPIDER.phi

float

tilt angle phi

SPIDER.theta

float

tilt angle theta

SPIDER.gamma

float

tilt angle gamma

SPIDER.headrec

int

number of records in header

SPIDER.headlen

int

header length in bytes

SPIDER.reclen

int

record length in bytes

SPIDER.dx

float

x translation

SPIDER.dy

float

y translation

SPIDER.dz

float

z translation

SPIDER.istack

int

0 for simple 2D or 3D (non-stack) files. for stacked image, istack=2 in overall header, istack =-1 in following individual images.

SPIDER.maxim

int

maxim is only used in the overall header for a stacked image file. It is the number of the highest image currently used in the stack. The number is updated, if necessary, when an image is added or deleted from the stack.

SPIDER.imgnum

int

imgnum is only used in a stacked image header. It is the number of the current image or zero if the image is unused.

SPIDER.Kangle

int

flag that additional angles are present in header. 1 = one additional rotation is present, 2 = additional rotation that preceeds the rotation that was stored in words 15..20.

SPIDER.phi1

float

SPIDER.theta1

float

SPIDER.psi1

float

SPIDER.phi2

float

SPIDER.theta2

float

SPIDER.psi2

float

SPIDER.date

char(11)

creation date

SPIDER.time

char(8)

creation time

SPIDER.title

char(160)

title

SPIDER.scale

float

scale factor

xform.projection

Transform

particle orientation, set from the orientation flags(phi, theta, psi, tx, ty, tz, scale) in the SPIDER header

xform.align3d

Transform

particle orientation for 3D image, set from the orientation flags(phi, theta, psi, tx, ty, tz, scale) in the SPIDER header

Parameters from PGM file

Tag

Type

Description

PGM.max_gray

int

maximum value for grey level

PGM.min_gray

int

minimum value for grey level

Parameters from SAL file

Tag

Type

Description

datatype

int

pixel storage data type in EMAN format: EM_SHORT

SAL.pixel

float

pixel size

Parameters from TIFF file

Tag

Type

Description

datatype

int

pixel storage data type in EMAN format: EM_UCHAR, EM_USHORT, EM_FLOAT

TIFF.bitspersample

unsigned short

bits per pixel sample

TIFF.resolution_x

float

x dimension resolution

TIFF.resolution_y

float

y dimension resolution

Parameters from Gatan DM3 files

Tag

Type

Description

DM3.acq_date

string

Acquisition date

DM3.acq_time

string

Acquisition time

DM3.actual_mag

float

Calibrated magnification

DM3.antiblooming

int

DM3.binning_x

int

Binning (X)

DM3.binning_y

int

Binning (Y)

DM3.camera_x

int

Camera size (X)

DM3.camera_y

int

Camera size (Y)

DM3.cs

float

Microscope Cs

DM3.exposure_number

int

Camera exposure number

DM3.exposure_time

float

Exposure time

DM3.frame_type

string

Frame type

DM3.indicated_mag

float

Indicated magnification

DM3.name

string

Filename

DM3.pixel_size

float

Pixel size (microns)

DM3.source

string

Camera name

DM3.voltage

float

Microscope voltage

DM3.zoom

float

Parameters from XPLOR file

Tag

Type

Description

apix_x,y,z

float

Angstroms per pixel on the x-axis (also _y and _z). If _y or _z are 0 or not present it will be assumed that they are the same as _x. Note that CTF objects have an independent value for A/pix

XPLOR.alpha

float

alpha angle of the cell

XPLOR.beta

float

beta angle of the cell

XPLOR.gamma

float

gamma angle of the cell

EMAN2/Eman2Metadata (last edited 2022-08-07 02:33:53 by SteveLudtke)