Diff for "EMAN2/Programs/e2refine"

Differences between revisions 1 and 10 (spanning 9 versions)

e2refine

This program is the heart of single particle reconstruction in EMAN2. It embodies the concept of an iterative 3-D single particle reconstruction in a single step, starting with a 3-D starting model and a set of preprocessed particle data. The overall strategy is similar to that used in EMAN1, with a number of improvements for speed and accuracy. The general idea is that the 3-D orientation of each particle is determined by comparison to a set of projections of the current 3-D model. Particles in near-identical orientations are then aligned and averaged in 2-D. These averages are then used to construct a new 3-D model, which is then reprojected for use in the next cycle of refinement. This process of reference-based classification is somewhat unique to EMAN, and is one reason why it can converge so rapidly to the correct answer even with a poor starting model.

EMAN2 refinement has many more options than EMAN1, and permits much more precise control over the refinement process. This can be both a blessing and a curse. We suggest launching your refinements from the workflow interface which simplifies specifying all of the necessary options. For those in search of detail, we document everything here.

Command Line Arguments

General Options

	--version	bool	show program's version number and exit
-h	--help	bool	show this help message and exit
-c	--check	bool	Checks the contents of the current directory to verify that e2refine.py command will work - checks for the existence of the necessary starting files and checks their dimensions.
-v	--verbose	int	verbose level [0-9], higner number means higher level of verboseness

Options impacting the overall refinement

	--iter	int	The total number of refinement iterations to perform
	--startiter	int	If a refinement crashes, this can be used to pick it up where it left off. This should NOT be used to change parameters, but only to resume an incomplete run.
	--model	string	The name 3D image that will seed the refinement
	--input	string	The name of the image containing the particle data
	--usefilt	string	Note: some unresolved bugs may exist with this option (6/2011) Specify a particle data file that has been low pass or Wiener filtered. Has a one to one correspondence with your particle data. If specified will be used in projection matching routines, and elsewhere.
	--path	string	The name of a directory where results are placed. If not specified (suggested), will use a path of the form refine_xx.
	--mass	float	The mass of the particle in kilodaltons, used to run normalize.bymass. If unspecified nothing happens. Requires the --apix argument.
	--apix	float	The angstrom per pixel of the input particles. This argument is required if you specify the --mass argument. If unspecified, the convergence plot is generated using either the project apix, or an apix of 1.
	--sym	string	Symmetry to be imposed throughout: c<n>, d<n>, h<n>, tet, oct, icos. Omit this option or specify 'c1' for asymmetric reconstructions.
	--lowmem	bool	Make limited use of memory when possible. Slight speed penalty.
-P	--parallel	string	Run in parallel, specify type:<option>=<value>:<option>:<value>. See EMAN2/Parallel

Options related to making projections

Options for comparing particles to projections

See also EMAN2/Programs/e2simmx

--twostage	int	Optionally run a faster 2-stage similarity matrix, ~5-30x faster, almost identical results. Value specifies shrink factor for first stage, typ 1-3
--shrink	int	Optionally shrink the input particles by an integer amount prior to computing similarity scores. For speed. If used with --twostage, this specifies the second stage shrink factor.
--simcmp	string	The name of a comparator to be used in comparing the aligned images
--simalign	string	The name of an aligner to use prior to comparing the images
--simaligncmp	string	Name and options for a comparator to use in first stage alignment for classification
--simralign	string	The name and parameters of the second stage aligner which refines the results of the first alignment. Currently this is either not specified or is 'refine'.
--simraligncmp	string	The name and parameters of the comparator used by the second stage aligner. Default is dot.
--simmask	string	A file containing a single 0/1 image to apply as a mask before comparison but after alignment
--prefilt	bool	Filter each reference (c) to match the power spectrum of each particle (r) before alignment and comparison

These parameters are used by e2simmx, a program that compares each particle to each projection and records quality scores. To do this the particles must first be aligned to the projections using the aligners you specify. Once aligned the 'Main comparator' is used to record the quality score. These quality values are recorded to an image matrix on handed on to the next stage in the refinement process.

The shrink parameter causes all projections and particles to be shrunken by the given amount prior to comparison. This can provide a significant time advantage, though at the expense of resolution. Note however that the class averaging stage, which can involve iterative alignment, does not use shrunken data.
2 stage simmx is still experimental. If set to 2 instead of zero, classification will be performed in two stages resulting in a 5-25x speedup, but with a potential decrease in accuracy.
PS match ref will force the power spectra of the particle and reference to be the same before comparison. Necessary for some comparators.
Main comparator is used to decide which reference a particle most looks-like (e2help.py cmps -v2)
Aligner - use default
Align comparator and refine align comparator allow you to select which comparators are used for particle->reference alignment. In most cases ccc is adequate, but sometimes you may wish to match the main comparator.
Refine align - if set to 'refine', alignments will be more accurate, and thus classification will be more accurate. Severe speed penalty.

For comparators here are some possible choices:

ccc (no options) - Simple dot product. Fast, can work well, but in some situations will cause a deterministic orientation bias (like GroEL side views which end up tilted). Works poorly for very noisy data unless usefilt particles are used for alignment.
frc zeromask=1:snrweight=1 - Fourier Ring Correlation with signal to noise ratio weighting and reference based masks. Works poorly without SNR weighting. Masking is optional, but a good idea.
phase zeromask=1:snrweight=1 - Mean phase error. same options as for frc. Do NOT use phase without snrweight=1
sqeuclidean normto=1:zeromask=1 - similar to ccc, but with additional options to better match densities. Only works well in conjunction with PS match ref, and usefilt with Wiener filtered particles.

Options for classifying particles based on similarity matrix

Options for generating class-averages

Options related to 3-D Reconstruction of Class-averages and post-processing

--pad	int	To reduce Fourier artifacts, the model is typically padded by ~25% - only applies to Fourier reconstruction. Please read EMAN2/BoxSize
--recon	string	reconstructor to use. Main choices are 'fourier' or 'wiener_fourier'
--m3dkeep	float	The percentage of slices to keep in e2make3d.py
--m3dkeepsig	bool	Similar to classkeepsig above, changes the meaning of --m3dkeep to be in terms of standard deviations
--m3dsetsf	bool	Filters the final 3-D map to match the precomputed structure factor (stored in the project database). Normally used with a --m3dpostprocess=filter.lowpass.* option
--m3diter	int	The number of times the 3D reconstruction should be iterated. 2 and 3 are the only valid values. 2 is faster and normally has sufficient accuracy.
--m3dpreprocess	string	Normalization processor applied before 3D reconstruction
--m3dpostprocess	string	Post processor to be applied to the 3D volume once the reconstruction is completed
--automask3d	string	The 5 parameters of the mask.auto3d processor, applied after 3D reconstruction. These parameters are, in order: isosurface threshold,radius,nshells and nshellsgauss. From e2proc3d.py you could achieve the same thing using --process=mask.auto3d:threshold=1.1:radius=30:nshells=5:nshellsgauss=5. e2help.py processors -v2 for more information on mask.auto3d.

Parameters for 3D reconstruction:

Use the 'fourier' or 'wiener_fourier' reconstructor
pad - should be some number a bit larger than your box size. This should be a 'good' box size as well (see wiki)
m3dkeep - similar to keep in class-averaging, but determines how many class averages are excluded from the reconstruction
m3dsetsf - This will force the reconstruction to be filtered to match the structure factor determined during CTF correction. If used it should be combined with a gaussian lowpass filter at the targeted resolution
m3dpostprocess - This is an optional filter to apply to the model as a final step, filter.lowpass.gauss with 'cutoff_freq=<1/resolution>' is good with set SF. If set SF is not used, note that the model will already be somewhat filtered even without this.

Please note that the box-sizes suggested in EMAN2 are larger than those in EMAN1 for improved CTF correction. Details on box size selection are here.

The refinement process produces a large number of different output files in databases within directories named refine_xx. The easiest way to browse these files is with EMAN2/Programs/e2display, the file browser. For documentation of the file contents, please see the items towards the bottom of this page.

-  ⇤ ← Revision 1 as of 2009-07-24 00:30:02 → 
  Size: 9614
  Editor: 10
  Comment:
+   ← Revision 10 as of 2012-09-21 11:59:03 → ⇥
  Size: 14484
  Editor: SteveLudtke
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
-| [[#args|Command line arguments]] |  [[#checkfunc|Check functionality]] | [[EMAN2/e2refinefaq|e2refine FAQ]] |
+== e2refine ==
 Line 3:
+This program is the heart of single particle reconstruction in EMAN2. It embodies the concept of an iterative 3-D single particle reconstruction in a single step, starting with a [[EMAN2/Programs/e2initialmodel|3-D starting model]] and a set of preprocessed particle data. The overall strategy is similar to that used in [[EMAN1]], with a number of improvements for speed and accuracy. The general idea is that the 3-D orientation of each particle is determined by comparison to a set of projections of the current 3-D model. Particles in near-identical orientations are then aligned and averaged in 2-D. These averages are then used to construct a new 3-D model, which is then reprojected for use in the next cycle of refinement. This process of reference-based classification is somewhat unique to EMAN, and is one reason why it can converge so rapidly to the correct answer even with a poor starting model.
-Line 4:
+Line 5:
-<<TableOfContents>>
== e2refine.py ==
|| {{attachment:e2refine.png}} ||


e2refine.py runs in much the same way as [[EMAN1/Programs/Refine|refine]] in [[EMAN1]]

This programs oversees iterative single particle reconstruction. The overall process is to take a pre-existing 3D image and a set of 2D images and to run a variety of (often intensive) image processing applications which produces a refined 3D model. In particular, the program iteratively executes a sequence of python scripts which perform specific tasks, starting with with 3D projection (e2project3d.py), comparision of particle data to projections (e2simmx.py),  classification (e2classify.py), the generation of class averages (e2classaverage.py), and finally the generation of a new 3D model (e2make3d.py). This pipeline is depicted graphically in '''Figure 1''' below, along with accompanying data inputs and outputs.
|| {{attachment:refinepipeline_small.png}} ||


'''Fig. 1. Overview of data inputs and outputs in the EMAN2 refinement pipeline. Pink objects are data supplied by the user, blue objects are programs, and green objects are data output by EMAN2 programs.'''



<<Anchor(args)>>
+EMAN2 refinement has many more options than EMAN1, and permits much more precise control over the refinement process. This can be both a blessing and a curse. We suggest launching your refinements from the [[EMAN2/Programs/e2workflow|workflow interface]] which simplifies specifying all of the necessary options. For those in search of detail, we document everything here.
-Line 22:
+Line 8:
-Most of the command line arguments have defaults, those which are absolutely. The user must atleast specify the total number of iterations, the symmetry and the proportional distribution or number of projections.
-Line 24:
+Line 9:
-==== General parameters ====
+==== General Options ====
|| ||--version||bool||show program's version number and exit||
||-h||--help||bool||show this help message and exit||
||-c||--check||bool||Checks the contents of the current directory to verify that e2refine.py command will work - checks for the existence of the necessary starting files and checks their dimensions. ||
||-v||--verbose||int||verbose level [0-9], higner number means higher level of verboseness||
-Line 26:
+Line 15:
-|| version || bool || Show program's version number and exit ||
|| h, help || bool || Show help ||
|| c, check || bool || Checks the contents of the current directory to verify that e2refine.py will work ||
|| v, verbose || int || Toggle verbose mode - prints extra infromation to the command line while executing ||
|| input|| string || The input image stack of 2D particles||
|| iter|| int|| The number of refinement iterations ||
|| lowmem|| boolean || A low memory flag used to indicate memory should be used as sparsely as possible ||
|| model|| string || The seeding 3D model ||
|| path || string || The directory where output will be stored ||
|| sym|| string|| The [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1Symmetry3D.html|symmetry]] being output 3D models and the limit the range of generated projections ||
+==== Options impacting the overall refinement ====
|| ||--iter||int||The total number of refinement iterations to perform||
|| ||--startiter||int||If a refinement crashes, this can be used to pick it up where it left off. This should NOT be used to change parameters, but only to resume an incomplete run.||
|| ||--model||string||The name 3D image that will seed the refinement||
|| ||--input||string||The name of the image containing the particle data||
|| ||--usefilt||string||''Note: some unresolved bugs may exist with this option (6/2011)'' Specify a particle data file that has been low pass or Wiener filtered. Has a one to one correspondence with your particle data. If specified will be used in projection matching routines, and elsewhere.||
|| ||--path||string||The name of a directory where results are placed. If not specified (suggested), will use a path of the form ''refine_xx''.||
|| ||--mass||float||The mass of the particle in kilodaltons, used to run normalize.bymass. If unspecified nothing happens. Requires the --apix argument.||
|| ||--apix||float||The angstrom per pixel of the input particles. This argument is required if you specify the --mass argument. If unspecified, the convergence plot is generated using either the project apix, or an apix of 1.||
|| ||--sym||string||[[EMAN2/Modular/Symmetry|Symmetry]] to be imposed throughout: c<n>, d<n>, h<n>, tet, oct, icos. Omit this option or specify 'c1' for asymmetric reconstructions. ||
|| ||--lowmem||bool||Make limited use of memory when possible. Slight speed penalty.||
||-P||--parallel||string||Run in parallel, specify type:<option>=<value>:<option>:<value>. See [[EMAN2/Parallel]]||
-Line 37:
+Line 28:
-==== Arguments used to execute e2project3d.py ====
+==== Options related to making projections ====
See also [[EMAN2/Programs/e2project3d]]
|| ||--projector||string||[[EMAN2/Modular/Projectors|Projector]] to use. 'standard' is the default||
|| ||--orientgen||string||The [[EMAN2/Modular/OrientGens|orientation generation]] argument for e2project3d.py. Typically something like: ''--orientgen=eman:delta=2.0:inc_mirror=0''||
-Line 39:
+Line 33:
-See [[e2project3d|e2project3d.py.]]
+These  parameters are used by e2project3d. Several orientation generation techniques provide alternative methods for distributing orientations in the asymmetric unit. Orientations can be generated based on your desired angular spacing, or alternatively on the desired total number of projections. In the latter case EMAN2 will generate a number as close as possible to the specified number, but note that there is no guarantee of a perfect match. You can also vary the method by which projections are generated. If you check the 'include mirror' option you should be sure to use aligners to that do not perform mirror alignment.
-Line 41:
+Line 35:
-|| orientgen|| string:args || The [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1OrientationGenerator.html|OrientationGenerator]] and parameters used for generation orientations in the asymmetric unit of the 3D model ||
|| projector || string:args || The [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1Projector.html|projector]] used for generating projections ||
+==== Options for comparing particles to projections ====
See also [[EMAN2/Programs/e2simmx]]
|| ||--twostage||int||Optionally run a faster 2-stage similarity matrix, ~5-30x faster, almost identical results. Value specifies shrink factor for first stage, typ 1-3||
|| ||--shrink||int||Optionally shrink the input particles by an integer amount prior to computing similarity scores. For speed. If used with --twostage, this specifies the second stage shrink factor.||
|| ||--simcmp||string||The name of a [[EMAN2/Modular/Cmps|comparator]] to be used in comparing the aligned images||
|| ||--simalign||string||The name of an [[EMAN2/Modular/Aligners|aligner]] to use prior to comparing the images||
|| ||--simaligncmp||string||Name and options for a [[EMAN2/Modular/Cmps|comparator]] to use in first stage alignment for classification||
|| ||--simralign||string||The name and parameters of the second stage [[EMAN2/Modular/Aligners|aligner]] which refines the results of the first alignment. Currently this is either not specified or is 'refine'.||
|| ||--simraligncmp||string||The name and parameters of the [[EMAN2/Modular/Cmps|comparator]] used by the second stage aligner. Default is dot.||
|| ||--simmask||string||A file containing a single 0/1 image to apply as a mask before comparison but after alignment||
|| ||--prefilt||bool||Filter each reference (c) to match the power spectrum of each particle (r) before alignment and comparison||
-Line 44:
+Line 47:
+These  parameters are used by e2simmx, a program that compares each particle to each projection and records quality scores. To do this the particles must first be aligned to the projections using the aligners you specify. Once aligned the 'Main comparator' is used to record the quality score. These quality values are recorded to an image matrix on handed on to the next stage in the refinement process.
 * The shrink parameter causes all projections and particles to be shrunken by the given amount prior to comparison. This can provide a significant time advantage, though at the expense of resolution. Note however that the class averaging stage, which can involve iterative alignment, does not use shrunken data.
 * 2 stage simmx is still experimental. If set to 2 instead of zero, classification will be performed in two stages resulting in a 5-25x speedup, but with a potential decrease in accuracy.
 * PS match ref will force the power spectra of the particle and reference to be the same before comparison. Necessary for some comparators.
 * Main comparator is used to decide which reference a particle most looks-like (e2help.py cmps -v2)
 * Aligner - use default
 * Align comparator and refine align comparator allow you to select which comparators are used for particle->reference alignment. In most cases ccc is adequate, but sometimes you may wish to match the main comparator.
 * Refine align - if set to 'refine', alignments will be more accurate, and thus classification will be more accurate. Severe speed penalty.
-Line 45:
+Line 56:
-==== Arguments used to execute e2simmx.py ====
+For comparators here are some possible choices:
-Line 47:
+Line 58:
-See [[e2simmx|e2simmx.py.]]
+ * ccc (no options) - Simple dot product. Fast, can work well, but in some situations will cause a deterministic orientation bias (like GroEL side views which end up tilted). Works poorly for very noisy data unless usefilt particles are used for alignment.
-Line 49:
+Line 60:
-|| shrink || int || The shrink factor applied to particles prior to generation of the similarity matrix (e2simmx.py) ||
|| simalign || string:args || The main [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1Aligner.html|aligner]] used during similarity matrix generation  ||
|| simaligncmp || string:args || The [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1Cmp.html|comparator]] used by the main aligner during similarity matrix generation ||
|| simcmp|| string:args || The [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1Cmp.html|comparator]] used to generate the final score which is stored in the similarity matrix ||
|| simralign|| string:args || The refinement [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1Aligner.html|aligner]] used during similarity matrix generation ||
|| simraligncmp || string:args || The [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1Cmp.html|comparator]] used by the refine align in similarity matrix generation ||
+ * frc zeromask=1:snrweight=1 - Fourier Ring Correlation with signal to noise ratio weighting and reference based masks. Works poorly without SNR weighting. Masking is optional, but a good idea.
-Line 56:
+Line 62:
-==== Arguments used to execute e2classify.py ====
+ * phase zeromask=1:snrweight=1 - Mean phase error. same options as for frc. Do NOT use phase without snrweight=1
-Line 58:
+Line 64:
+ * sqeuclidean normto=1:zeromask=1 - similar to ccc, but with additional options to better match densities. Only works well in conjunction with PS match ref, and usefilt with Wiener filtered particles.
-Line 59:
+Line 66:
-See [[e2classify|e2classify.py]]
+==== Options for classifying particles based on similarity matrix ====
See also [[EMAN2/Programs/e2classify]]
|| ||--sep||int||The number of classes a particle can contribute towards (default is 1). This is a precursor to full maximum-liklihood.||
-Line 61:
+Line 70:
-|| sep || int || The number of classes each particles can be associated with ||
+ * sep - puts each particle in the best 'n' classes, a larger number here combined with a small angular step can somewhat mimic maximum liklihood methods (smoother map at the cost of resolution)
-Line 63:
+Line 72:
-==== Arguments used to execute e2classaverage.py ====
+==== Options for generating class-averages ====
See also [[EMAN2/Programs/e2classaverage]]
|| ||--classiter||int||The number of iterations to perform. Default is 1. Larger values reduce model/noise bias, but slightly decrease resolution.||
|| ||--classcmp||string||The name and parameters of the comparitor used to generate similarity scores, when class averaging. Default is 'dot:normalize=1'||
|| ||--classalign||string||If doing more than one iteration, this is the name and parameters of the [[EMAN2/Modular/Aligners|aligner]] used to align particles to the previous class average.||
|| ||--classaligncmp||string||This is the name and parameters of the [[EMAN2/Modular/Cmps|comparator]] used by the fist stage aligner  Default is dot.||
|| ||--classralign||string||The second stage aligner which refines the results of the first alignment in class averaging. Currently this is either not specified or is 'refine'.||
|| ||--classraligncmp||string||The [[EMAN2/Modular/Cmps|comparator]] used by the second stage aligner in class averageing. Default is dot:normalize=1.||
|| ||--classaverager||string||The [[EMAN2/Modular/Averagers|averager]] used to generate the class averages. Default is 'mean'.||
|| ||--classkeep||float||The fraction of particles to keep in each class, based on the similarity score generated by the --cmp argument (see also --classkeepsig).||
|| ||--classkeepsig||bool||Change the keep ('--keep') criterion from fraction-based to sigma-based. eg - with this set, 1.0 would correspond to discarding particles more than 1 standard deviation from the mean||
|| ||--classnormproc||string||Normalization [[EMAN2/Modular/Processors|processor]] and options applied during class averaging. Typically 'normalize.edgemean'||
|| ||--classrefsf||bool||This will impose the 1-D structure factor of each model projection onto the corresponding class-average to improve its filtration. This is an alternative to Wiener filtration if the map resolution is regulated.||
|| ||--classautomask||bool||Experimental. This will apply a 2-D automask to the class-average during iterative alignment for better accuracy. The final class averages are unmasked.||
-Line 65:
+Line 87:
-See [[e2classaverage|e2classaverage.py]].
+These parameters address how class-averages are made. For the comparators see the previous tab:
 * classiter - Use 3-6 when doing initial refinements to eliminate model bias. Use 1 when pushing resolution
 * keep - determines how many 'bad' particles are thrown away either in terms of sigma, or an absolute value (1 keeps 100%, .8 keeps 80%)
 * averager - either ctf.auto for ctf amplitude correction or mean for no CTF amplitude correction
 * classrefsf - this will filter the class-averages to match the radial power spectrum of the projections. Not good for initial rounds of refinement, but may be useful later.
-Line 67:
+Line 93:
-|| classalign || string:args || The [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1Aligner.html|aligner]] used for alignment during iterative class averaging ||
|| classaligncmp|| string:args || [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1Cmp.html|Comparator]] used by the main aligner during iterative class averaging ||
|| classaverager|| string::args || [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1Averager.html|Averager]] used for averaging the images in each class ||
|| classcmp || string:args || The main [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1Cmp.html|comparator]] used to quality and exclude bad particles in iterative class averaging ||
|| classiter || int || The number of class averaging iterations ||
|| classkeep|| float || The keep threshold used for excluding bad particles in iterative class averaging ||
|| classnormproc|| string:args || The normalization [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1Processor.html|processor]] used in class averaging ||
|| classralign|| string:args || The refinement [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1Aligner.html|aligner]] used in iterative class averagin ||
|| classraligncmp || string:args || The [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1Cmp.html|comparator]] used by the refinement aligner in iterative class averaging ||
+==== Options related to 3-D Reconstruction of Class-averages and post-processing ====
|| ||--pad||int||To reduce Fourier artifacts, the model is typically padded by ~25% - only applies to Fourier reconstruction. Please read [[EMAN2/BoxSize]]||
|| ||--recon||string||[[EMAN2/Modular/Reconstructors|reconstructor]] to use. Main choices are 'fourier' or 'wiener_fourier'||
|| ||--m3dkeep||float||The percentage of slices to keep in e2make3d.py||
|| ||--m3dkeepsig||bool||Similar to classkeepsig above, changes the meaning of --m3dkeep to be in terms of standard deviations||
|| ||--m3dsetsf||bool||Filters the final 3-D map to match the precomputed structure factor (stored in the project database). Normally used with a ''--m3dpostprocess=filter.lowpass.*'' option||
|| ||--m3diter||int||The number of times the 3D reconstruction should be iterated. 2 and 3 are the only valid values. 2 is faster and normally has sufficient accuracy.||
|| ||--m3dpreprocess||string||Normalization [[EMAN2/Modular/Processors|processor]] applied before 3D reconstruction||
|| ||--m3dpostprocess||string||Post [[EMAN2/Modular/Processors|processor]] to be applied to the 3D volume once the reconstruction is completed||
|| ||--automask3d||string||The 5 parameters of the mask.auto3d processor, applied after 3D reconstruction. These parameters are, in order: isosurface threshold,radius,nshells and nshellsgauss. From e2proc3d.py you could achieve the same thing using ''--process=mask.auto3d:threshold=1.1:radius=30:nshells=5:nshellsgauss=5''. ''e2help.py processors -v2'' for more information on mask.auto3d.||
-Line 77:
+Line 104:
-==== Arguments used to execute e2make3d.py  ====
+Parameters for 3D reconstruction:
 * Use the 'fourier' or 'wiener_fourier' reconstructor
 * pad - should be some number a bit larger than your box size. This should be a 'good' box size as well (see wiki)
 * m3dkeep - similar to keep in class-averaging, but determines how many class averages are excluded from the reconstruction
 * m3dsetsf - This will force the reconstruction to be filtered to match the structure factor determined during CTF correction. If used it should be combined with a gaussian lowpass filter at the targeted resolution
 * m3dpostprocess - This is an optional filter to apply to the model as a final step, filter.lowpass.gauss with 'cutoff_freq=<1/resolution>' is good with set SF. If set SF is not used, note that the model will already  be somewhat filtered even without this.
Please note that the box-sizes suggested in EMAN2 are larger than those in EMAN1 for improved CTF correction. Details on box size selection are [[EMAN2/BoxSize|here]].
-Line 79:
+Line 112:
-See [[e2make3d|e2make3d.py.]]

|| m3diter|| int || The number of iterations used my make3d when performing the Fourier inversion method of 3D reconstruction ||
|| m3dkeep|| float || The keep threshold used by e2make3d for the purpose of slice exclusion during 3D reconstruction ||
|| m3dpreprocess|| string:args || The normalization [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1Processor.html|processor]] applied prior to slice insertion during 3D reconstruction ||
|| pad || int || The amount of padding used by the Fourier inversion 3D reconstruction technique ||
|| recon|| string:args || The [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1Reconstructor.html|reconstructor]] used for performing 3D reconstruction ||

==== Arguments used to post process the 3D reconstruction  ====

The ByMass links will resolve on January 22

|| mass || float || The estimated mass of the particle in kilodalton that, along with the apix argument, is used to run the  [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1NormalizeByMassProcessor.html| normalize.bymass processor]] immediately after 3D reconstruction ||
|| apix || float || The physical distance represented by a single pixel. This parameter, along with the mass argument, is used to run the  [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1NormalizeByMassProcessor.html| normalize.bymass processor]] immediately after 3D reconstruction. The apix argument is also used for generating the x-axis of the automatically generated convergence plots. ||
|| automask3d || float,int,int,int || The threshold, radius, nshells and nshellsgauss parameters, respectively, of the  [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1AutoMask3D2Processor.html| mask.auto3d processor]], which is applied directly after the application of the [[http://blake.bcm.edu/eman2/doxygen_html/classEMAN_1_1NormalizeByMassProcessor.html| normalize.bymass processor]]. ||



<<Anchor(checkfunc)>>

=== Check functionality ===
By specifying the --check flag, e2refine.py checks only whether the specified parameters are valid, and nothing more. Example output is shown below.

{{{
[someone@localhost]$ e2refine.py --check
#### Testing directory contents and command line arguments for e2refine.py
Error: you must specify the --it argument
start.img contains 2498 images of dimensions 100x100
threed.0a.mrc has dimensions 100x100x100
e2refine.py test.... FAILED
#### Test executing projection command: e2project3d.py threed.0a.mrc -f --sym=None --projector=standard --out=e2proj.img --check
Error: you must specify one of either --prop or --numproj
Error: none is an invalid symmetry type. You must specify the --sym argument
e2project3d.py command line arguments test.... FAILED
#### Test executing simmx command: e2simmx.py e2proj.img start.img e2simmx.img -f --saveali --cmp=dot:normalize=1 --align=rotate_translate --aligncmp=dot --check --nofilecheck
e2simmx.py command line arguments test.... PASSED
#### Test executing classify command: e2classify.py e2simmx.img e2classify.img --sep=2 -f --check --nofilecheck
e2classify.py command line arguments test.... PASSED
#### Test executing classaverage command: e2classaverage.py start.img e2classify.img e2classes.1.img --ref=e2proj.img --iter=3 -f --keepsig=1.000000 --cmp=dot:normalize=1 --align=rotate_translate --aligncmp=phase --check --nofilecheck
e2classaverage.py command line arguments test.... PASSED
#### Test executing make3d command: e2make3d.py e2classes.1.img --sym=None --iter=4 -f --recon=fourier --out=threed.0a.mrc --keepsig=1.000000 --check --nofilecheck
Error: none is an invalid symmetry type. You must specify the --sym argument
e2make3d.py command line arguments test.... FAILED
}}}
This functionality will be useful for people who have to submit their jobs to queues - being able to check that the script will work will ensure its successful execution.
+The refinement process produces a large number of different output files in databases within directories named ''refine_xx''. The easiest way to browse these files is with ''[[EMAN2/Programs/e2display]]'', the file browser. For documentation of the file contents, please see the items towards the bottom of [[EMAN2/Concepts|this page]].

	--projector	string	Projector to use. 'standard' is the default
	--orientgen	string	The orientation generation argument for e2project3d.py. Typically something like: --orientgen=eman:delta=2.0:inc_mirror=0

--classiter	int	The number of iterations to perform. Default is 1. Larger values reduce model/noise bias, but slightly decrease resolution.
--classcmp	string	The name and parameters of the comparitor used to generate similarity scores, when class averaging. Default is 'dot:normalize=1'
--classalign	string	If doing more than one iteration, this is the name and parameters of the aligner used to align particles to the previous class average.
--classaligncmp	string	This is the name and parameters of the comparator used by the fist stage aligner Default is dot.
--classralign	string	The second stage aligner which refines the results of the first alignment in class averaging. Currently this is either not specified or is 'refine'.
--classraligncmp	string	The comparator used by the second stage aligner in class averageing. Default is dot:normalize=1.
--classaverager	string	The averager used to generate the class averages. Default is 'mean'.
--classkeep	float	The fraction of particles to keep in each class, based on the similarity score generated by the --cmp argument (see also --classkeepsig).
--classkeepsig	bool	Change the keep ('--keep') criterion from fraction-based to sigma-based. eg - with this set, 1.0 would correspond to discarding particles more than 1 standard deviation from the mean
--classnormproc	string	Normalization processor and options applied during class averaging. Typically 'normalize.edgemean'
--classrefsf	bool	This will impose the 1-D structure factor of each model projection onto the corresponding class-average to improve its filtration. This is an alternative to Wiener filtration if the map resolution is regulated.
--classautomask	bool	Experimental. This will apply a 2-D automask to the class-average during iterative alignment for better accuracy. The final class averages are unmasked.