Binary Installation Instructions v2.31

All Systems

Official releases are in reverse chronological order. The Continuous Build is rebuilt any time a developer makes a change they think users should have access too. It is normally reasonably stable, and will contain the latest pre-publication features. Alternatively, the highest numbered version will contain a stable and tested release. Windows10 users please see the instructions below before downloading.

• Download EMAN2

Mac OS X

Note: the neural network code in EMAN2 works best on GPUs, which are available only on the Linux installations. It can still run on Mac, but will be quite slow.

1. If you have previously installed EMAN2:
   • Please remove or rename any existing installed EMAN2 folder you might have.
   • LD_LIBRARY_PATH, DYLD_LIBRARY_PATH and PYTHONPATH are NO LONGER USED, and should be removed if you have them set.
   • If you have any of these shell variables set for use with other software, it may be necessary to remove those settings as well. If the tests below fail after installation, this is the first thing to check.
2. Download eman2.X.MacOS.sh.
3. Run:

   bash <path-to-EMAN2-installer>

   • You will be prompted for a location to install EMAN2. Note that you cannot rename this folder after installation! You must reinstall if you wish to move the installation.

   • You will be asked if you want to add  export PATH=...  to your .profile file.

   • If you use a different shell, such as tcsh or zsh, you may need to edit the appropriate file yourself.
   • You should not normally need to run the OpenMPI reinstallation scripts. The copy of OpenMPI/Pydusa now distributed with the binaries should work on Macs in most cases.
   • Don't forget to restart your shell if you changed the .profile or other scripts.

   • If you don't understand what the .profile instructions are talking about, this may help: https://stackoverflow.com/questions/7501678/set-environment-variables-on-mac-os-x-lion

4. Run these programs to see if the install worked:

   e2version.py
   e2speedtest.py
   e2display.py
   e2proc2d.py :64:64:1 test.hdf --process mask.sharp:outer_radius=24

5. If you have problems with any of these programs, the first thing to check is whether you have PYTHONPATH, LD_LIBRARY_PATH or DYLD_LIBRARY_PATH set in your shell. While used in previous versions of EMAN, these variables are no longer necessary. If they are set to make some other software package work, but they interfere with the programs above, you will have to unset them, and set them only when you need the other software.

Linux Workstations (not clusters)

1. If you have previously installed EMAN2:
   • Please remove or rename any existing installed EMAN2 folder you might have.
   • LD_LIBRARY_PATH, DYLD_LIBRARY_PATH and PYTHONPATH are NO LONGER USED, and should be removed if you have them set.
   • If you have any of these shell variables set for use with other software, it may be necessary to remove those settings as well. If the tests below fail after installation, this is the first thing to check.
2. Run:

   bash <path-to-EMAN2-installer>

   • You will be prompted for a location to install EMAN2. Note that you cannot rename this folder after installation! You must reinstall if you wish to move the installation.

   • You will be asked if you want to add  export PATH=...  to your .profile file.

   • If you use a different shell, such as tcsh or zsh, you may need to edit the appropriate file yourself.
   • You should not normally need to run the OpenMPI reinstallation scripts. The copy of OpenMPI/Pydusa now distributed with the binaries should work on Linux workstations in most cases.
   • Don't forget to restart your shell if you changed the .profile or other scripts.

3. The new neural network based routines are much faster running on a GPU. If you have an NVidia graphics card, see Using the GPU section below.

4. Run these programs to see if the install worked:

   e2version.py
   e2speedtest.py
   e2display.py
   e2proc2d.py :64:64:1 test.hdf --process mask.sharp:outer_radius=24
   e2display.py test.hdf

5. If you have problems with any of these programs, the first thing to check is whether you have PYTHONPATH or LD_LIBRARY_PATH set in your shell. While used in previous versions of EMAN, these variables are no longer used, and in some cases may interfere with Anaconda. If they have been set to make some other software package work, but they interfere with the programs above, you will have to unset them, and set them only when you need the other software.

6. Specifically, if only the last command fails and you are using a Nvidia graphic card, it is likely caused by a graphic card driver incompatibility. Updating the Nvidia driver usually fix the problem. On recent Ubuntu systems, running apt-get install nvidia-current works. On other systems, you may need to follow the installation guide from Nvidia.

Linux Clusters

• Follow the Linux workstation instructions above.
• When using EMAN2/SPARX/SPHIRE on a cluster, the version of OpenMPI provided with the EMAN2.31 binaries may not be aware of the batch queuing system used to launch jobs on the cluster, and may only be able to run on one node at a time unless you follow the OpenMPI reinstallation instructions below. Follow only ONE of the sets of instructions below.

• You will need conda-build for the instructions to work. Binaries include it, but for source installations, it needs to be installed.  conda install -n base conda-build=3.17.8 -c defaults 

• If you have a file named .condarc in your home directory, temporarily rename or move it for the following instructions to work properly.

• Reinstall pytz. Binary installation script installs it to a wrong location.

  conda install --force-reinstall pytz

Use system OpenMPI

Most Linux clusters will have at least one OpenMPI installation on the cluster. In some cases there may be more than one, and you may have to select a "module" to get the correct one. It is also critical that OpenMPI be compiled with the --disable-dlopen option. If you don't understand this statement, please consult with your cluster sysadmin.

1. Remove the OpenMPI we provided:

   conda remove openmpi --force

2. Make sure that the correct OpenMPI for your cluster is in your path. You should be able to run 'mpicc' and get a message like 'gcc: no input files'.
3. Rebuild Pydusa using the system installed OpenMPI.

   export GIT_PYDUSA_BRANCH=v20180515
   conda build <path-to-EMAN2-directory>/recipes/pydusa --numpy 1.13 --no-test

   • Warning: If you see an error after this process like:

     Can't build /home/stevel/EMAN2/recipes/fftw-mpi due to environment creation error:
     Downloaded bytes did not match Content-Length
       url: http://www.fftw.org/fftw-3.3.6-pl1.tar.gz
       target_path: /home/stevel/EMAN2/conda-bld/src_cache/fftw-3.3.6.tar.gz
       Content-Length: 4179807
       downloaded bytes: 208916

     this means the fftw download failed. You will need to re-run this step, but first, delete the failed download :  rm EMAN2/conda-bld/src_cache/fftw-3.3.6.tar.gz 

   • Warning: If you see an error during this process like:

          configure: error: could not find mpi library for --enable-mpi

     this means the fftw build failed. On our system, this happened due to a libuuid library mismatch in combination with the omnipath cluster architecture. To workaround this situation try:

         cd <path-to-EMAN2-directory>/lib
         rm libuuid.so.1
         ln -s /lib64/libuuid.so.1.3.0 libuuid.so.1 (Try  'ldconfig -p | grep libuuid'  to find the correct libuuid location and version)
         cd -

     Run this step again and revert this change after the whole installation process is complete.

         cd <path-to-EMAN2-directory>/lib
         rm libuuid.so.1
         ln -s libuuid.so.1.0.0 libuuid.so.1
         cd -

4. Finally, install the compiled Pydusa:

   conda install fftw-mpi --force-reinstall --override-channels -c file://<path-to-EMAN2-directory>/conda-bld -c defaults
   conda install 'pydusa=1.15=np113*' --force-reinstall --override-channels -c file://<path-to-EMAN2-directory>/conda-bld -c defaults

Rebuild your own OpenMPI

This option insures that --disable-dlopen is used when compiling OpenMPI, but may lack some system-specific optimizations provided by your sysadmin.

1. Remove the OpenMPI we provided:

   conda remove openmpi --force

2. Rebuild OpenMPI.

   It is important to have the hwloc library available on the system. It might be available with 'module load hwloc'. It is also important to have the libraries of the used queuing system available (slurm, Torque/PBS, SGE, ...).

   conda build <path-to-EMAN2-directory>/recipes/openmpi -c defaults -c conda-forge -c conda-forge/label/cf201901
   conda install openmpi --override-channels -c file://<path-to-EMAN2-directory>/conda-bld -c defaults

3. Rebuild Pydusa using the rebuilt OpenMPI:

   export GIT_PYDUSA_BRANCH=v20180515
   conda build <path-to-EMAN2-directory>/recipes/pydusa --numpy 1.13 --no-test

   • Warning: If you see an error after this process like:

     Can't build /home/stevel/EMAN2/recipes/fftw-mpi due to environment creation error:
     Downloaded bytes did not match Content-Length
       url: http://www.fftw.org/fftw-3.3.6-pl1.tar.gz
       target_path: /home/stevel/EMAN2/conda-bld/src_cache/fftw-3.3.6.tar.gz
       Content-Length: 4179807
       downloaded bytes: 208916

     this means the fftw download failed. You will need to re-run this step, but first, delete the failed download :  rm EMAN2/conda-bld/src_cache/fftw-3.3.6.tar.gz 

   • Warning: If you see an error during this process like:

          configure: error: could not find mpi library for --enable-mpi

     this means the fftw build failed. On our system, this happened due to a libuuid library mismatch in combination with the omnipath cluster architecture. To workaround this situation try:

         cd <path-to-EMAN2-directory>/lib
         rm libuuid.so.1
         ln -s /lib64/libuuid.so.1.3.0 libuuid.so.1 (Try  'ldconfig -p | grep libuuid'  to find the correct libuuid location and version)
         cd -

     Run this step again and revert this change after the whole installation process is complete.

         cd <path-to-EMAN2-directory>/lib
         rm libuuid.so.1
         ln -s libuuid.so.1.0.0 libuuid.so.1
         cd -

4. Finally, install the compiled Pydusa:

   •   conda install fftw-mpi --force-reinstall --override-channels -c file://<path-to-EMAN2-directory>/conda-bld -c defaults
       conda install 'pydusa=1.15=np113*' --force-reinstall --override-channels -c file://<path-to-EMAN2-directory>/conda-bld -c defaults

Windows

We are finally able to provide 64 bit Windows binaries for EMAN2, however, please see the Windows 10- Linux/Bash shell option below for what may be a better alternative. Notes:

• SPARX/SPHIRE are not supported.
• EMAN2 functionality may not be complete using this first approach. You may get more complete functionality, but with some additional effort using the Linux/Bash shell approach below.
• Even for EMAN2, Windows support remains somewhat marginal, and is provided primarily for utility functions and basic GUI tools, like micrograph evaluation and particle picking. Complete refinements may not work well under Windows. You are welcome to ask questions in the mailing list, but there may be limited help we can provide because we simply don't have Windows machines around for testing.

Native Win7/10 64 bit

1. Download eman2.X.win64.exe.
2. Launch the installer, and answer any security questions you are prompted for.
3. Start a command prompt by clicking Start menu and typing cmd.exe in the dialog at the bottom.
4. On the command prompt, type

   setx path "%path%;c:\<path-to-EMAN2-directory>\Library\bin"

5. Close the command prompt and open a new one.

6. In most cases you will want to install: Python Launcher.

7. Run these programs to see if the install worked:

   e2version.py
   e2speedtest.py
   e2display.py
   e2proc2d.py :64:64:1 test.hdf --process mask.sharp:outer_radius=24

Windows 10 - Linux/Bash shell

Windows 10 includes an embedded Ubuntu Linux environment. It is possible to run the EMAN2 Linux binaries within this Win10 environment, but you will need to install some additional dependencies to do so. Also, you will effectively be running at a Linux command prompt, so you will have to become a bit familiar with Linux to do this, but it does avoid installing an additional operating system on your machine.

1. Install "Bash on Windows 10", https://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/.

2. When prompted to set a user name, enter root. This should give you an account without a password.

Install OpenGL and X Server, set environment variables

1. Install OpenGL.

   sudo apt-get update
   sudo apt-get install libsm-dev libxrender-dev build-essential libgl1-mesa-dev mesa-utils mesa-common-dev libglu1-mesa libglu1-mesa-dev
   sudo apt-get autoremove

2. Install Xming X Server for Windows.

   • Don't forget the fonts and Mesa (OpenGL) modules! If it seems to work, but the letters are black boxes, or you have other visual artifacts, the problem is probably with OpenGL support.
3. Set environment variables.

   export DISPLAY=:0
   glxinfo | grep OpenGL
   export KMP_AFFINITY=disabled # per https://github.com/Microsoft/BashOnWindows/issues/785#issuecomment-238079769

4. Download and install eman2.X.linux64.centos7.sh.

5. Start X Server before running eman2 programs.
6. Run these programs to see if the install worked:

   e2version.py
   e2speedtest.py
   e2display.py
   e2proc2d.py :64:64:1 test.hdf --process mask.sharp:outer_radius=24

Using the GPU

Currently, the GPU is only used for neural network operations in tomogram annotation and in particle picking. It provides a ~10 fold or more speed up in neural network training. The new GPU developments are currently based on TensorFlow. From about 2006-2012 EMAN2 had its own internal CUDA code, which could be compiled into the C++ library. This has been deprecated, and likely no longer works, though the code is still present. We are working on a new GPU support strategy moving forward.

Note that due to the NumPy version in the 2.2 release, we are still using the 'old' GPU backend for compatibility (i.e. NOT the libgpuarray backend), so be sure to use device=gpu in your .theanorc. Hopefully this will change in the 2.21 update (unfortunately it did not...but it is upgraded in the daily build newer than 2.21).

On a freshly installed Ubuntu 16.10, an easy way to install the GPU support is to run the following command after successful binary installation.

apt-get install nvidia-cuda-toolkit

If you are using a new GPU (like GTX1080 or better), you may need the newest version of CUDA to support the hardware. The following command should work.

apt-get install cuda-8-0

If this does not work, you may consider downloading the latest CUDA from Nvidia through the following link and follow their instruction to install. https://developer.nvidia.com/cuda-downloads

• After the CUDA installation (run "nvcc --version" to check if it works), create a text file called .theanorc in your $HOME directory with the following content:

  [global]
  device = gpu
  floatX = float32

Then try running any neural network related program or simply run e2.py then import theano. If you see print out message like Using gpu device ******* (CNMeM is disabled, cuDNN ****), it means the GPU is now being used. For other system (or if the guide above does not work), you may follow the instruction from Theano and Nvidia. Keep in mind that CUDA installation can be a painful process on some computers especially when some of the hardware are old. CUDA also has internal incompatibility issue with newer version of gcc, so it might also break other software you have installed. So be careful and good luck...

CUDA 9.0 support

(requires EMAN2.21 or newer)

In Theano, we now support CUDA9.0 and the gpuarray backend. To upgrade to CUDA9.0, you will need:

• Latest version of EMAN2 from GitHub (2017-10-25 or later).

• Edit ~/.theanorc file to:

  [global]
  device = cuda
  floatX = float32

• Upgrade Theano to latest GitHub version:

  conda remove theano
  pip install --no-deps git+https://github.com/Theano/Theano.git#egg=Theano

• Upgrade pygpu

  conda install -c conda-forge pygpu=0.7

• The latest Theano tries to use CuDNN by default. CuDNN speeds up neural network training, although the improvement is not very significant for the size of networks we are using. So you need to either install the latest CuDNN from Nvidia (https://developer.nvidia.com/cudnn), or disable it by adding this to ~/.theanorc

  [dnn]
  enabled=False

• If you get a very long string of errors a few seconds after trying to train a network, and see references to "narrowing" in the errors, this probably means your C++ compiler is newer than Theano expects. You can try adding the following to your .theanorc file:

  [gcc]
  cxxflags = -Wno-narrowing