Differences between revisions 3 and 34 (spanning 31 versions)
Revision 3 as of 2015-03-06 02:01:51
Size: 14310
Editor: MichaelBell
Comment: Fixed titles (hopefully)
Revision 34 as of 2015-05-11 15:01:14
Size: 20159
Editor: MichaelBell
Comment: Cleaned up the text a bit. Removed some unnecessary stuff.
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
Homebrew installation instructions
==================

The following install was successful (barring yet unseen bugs!) on Mountain Lion. If you wish to skip the fun of compiling by hand, you can follow the instructions below.

Install homebrew
==================

Homebrew is a package management system for Mac. It is under active development and has strict policies to protect the user from damage via software. They also seek to protect users from themselves by avoiding sudo commands as much as possible. It is recommended that you do NOT install homebrew alongside other package managers such as macports/fink. It seems to do just fine on its own.
= Compiling EMAN2 on Mac OS X with Homebrew =

<<TableOfContents>>

Homebrew is a package management system for Mac. It is under active development and has strict policies to protect the user from damage via software. They also seek to protect users from themselves by avoiding sudo commands as much as possible. It is recommended that you do NOT install homebrew alongside other package managers such as macports/fink.


The following install was successful on OS X 10.8 (Mountain Lion) and OS X 10.10 (Yosemite). If for any reason this doesn't work for you, I recommend refreshing your terminal (1st) and restarting your computer (2nd) since many of the packages downloaded might require sourcing. If neither solve your specific problem, please send an e-mail to a member of the lab and we'll try to address it as soon as possible.

If you find yourself upgrading from OS X 10.8 to 10.9 or 10.10, you will need to reinstall all homebrew packages or start from scratch. Luckily, there is an easy way to do the former:

{{{
brew list | xargs brew reinstall -v
}}}

This should reinstall all packages in their current versions. You will then need to reinstall older versions of berkeley-db (5.3.28) and cmake (3.1.3) as shown below (replacing the package name as appropriate, of course). To do so you must first remove the existing version of these two programs created by the above reinstallation command.

= Install homebrew =
Line 13: Line 22:
 ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" {{{
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
}}}
Line 17: Line 28:
 brew doctor
 
{{{
brew doctor
}}}
Line 21: Line 34:
 export PATH=/usr/local/bin:$PATH

Install important stuff
==================

CVS/GIT
----------------
{{{
export PATH=/usr/local/bin:$PATH
}}}

= Required Packages =

== Xcode ==

Starting with Mac OS X 10.7 and Xcode 4.3, the developer tools are distributed through the Mac App Store instead of an installer package.

You'll also need the command line tools, which you can install with:

{{{
xcode-select --install
}}}

== GCC/LLVM ==

To compile EMAN2's C++ code, we need to download a compiler. Here we install the homebrew versions of gcc and llvm, which are more up-to-date than those available on Mac by default.

{{{
brew install gcc --with-all-languages
brew install llvm --with-clang --with-asan
}}}

== CMake ==

CMake is a family of tools designed to build, test and package software. It is used to generate the Makefiles for compiling EMAN2. We install the homebrew version as follows:

{{{
brew install cmake
}}}

== OpenGL ==

{{{
brew install glew glfw3
}}}

== Qt4 ==

Qt is a graphics library for cross-platform user interfaces.

{{{
brew install qt
}}}

If you need to incorporate additional qt features, check the output of

{{{
brew options qt
}}}

to see what else you can include.

== Boost ==

Boost is a set of useful, standardized libraries for C++. It is used to generate the Python-C++ interface in EMAN2. Homebrew comes with two 'boost' packages, and we'll need both to generate the libraries required for compiling EMAN2:

{{{
brew install boost --with-mpi # or leave out the mpi option
brew install boost-build boost-python
}}}

== GSL ==

The GNU Scientific Library provides a wide range of mathematical routines such as random number generators, special functions and least-squares fitting.

{{{
brew install gsl
}}}

A personal concern is that there is no option to '--enable-float'; however, I have not run into any problems with this yet.

== FFTW3 ==

FFTW is a C subroutine library for computing the discrete Fourier transform (DFT) in one or more dimensions, of arbitrary input size, and of both real and complex data (as well as of even/odd data, i.e. the discrete cosine/sine transforms or DCT/DST). The most recent version is available from homebrew as follows:

{{{
brew install fftw
}}}

== Berkeley DB ==

The Oracle Berkeley DB family of open source, embeddable databases provides developers with fast, reliable, local persistence with zero administration. EMAN2 uses Berkeley DB as a mechanism for local storage of project metadata.

Oracle has made their most recent version very VERY proprietary. You need an account on their website to download the package, and you need a license to use 6.0.1. Instead, we can use homebrew, but we need to get a previous version since homebrew defaults to 6.0.1 (as of March, 2015). I managed to find instructions on Stack Overflow ([[https://stackoverflow.com/questions/3987683/homebrew-install-specific-version-of-formula]]) that explain how to go about this and have reproduced them below:

{{{
brew tap homebrew/boneyard
brew versions berkeley-db # Copy the command after '5.3.28'
cd $( brew --prefix )
git checkout fdc17de /usr/local/Library/Formula/berkeley-db.rb
cd ${HOME}
}}}

Now we can install berkeley-db just as we would any other package.

{{{
brew install berkeley-db
}}}

Finally, add the following to your shell's 'rc' file to enable bsddb3 to know where berkeley-db is located:

{{{
export BERKELEYDB_DIR=/usr/local/BerkeleyDB.5.3
}}}

At this point, I suggest that you refresh your shell.

If you're using Yosemite, you'll just want to define the environment variable as /usr/local instead.

== Freetype ==

On Mac OS X 10.8 and later, X11 and Freetype are no longer included by default. Even if you are using an earlier Mac OS X, I recommend installing your own Freetype. Homebrew offers the most recent version of freetype, but we'll need to explicitly include it in our configuration step later on.

{{{
brew install freetype
}}}

== FTGL ==

FTGL is a free, open source library to enable developers to use arbitrary fonts in their OpenGL applications. We can install it as follows:

{{{
brew install ftgl
}}}

== PNG ==

The PNG (Portal Network Graphics) format provides lossless-compressed image support. Mac OS X 10.7 Lion ships with libpng 1.5, in which some deprecated features in the PNG API were removed. EMAN2 is being updated to use these new methods. For now, use libpng 1.2 if you are compiling on 10.7.

{{{
brew install libpng
}}}

== HDF5 ==

HDF5 is a data model, library, and file format for storing and managing data. It supports an unlimited variety of datatypes, and is designed for flexible and efficient I/O and for high volume and complex data.

{{{
brew install hdf5
}}}

= Optional (Recommended) Packages =

== Readline ==

We can easily install 'readline' to parse command history in python as follows:

{{{
brew install readline
}}}

== JPEG ==

Library for writing JPEG image files. Though optional, it's very easy to install this package:

{{{
brew install jpeg
}}}

== TIFF ==

Library for reading and writing TIFF image files. Install TIFF as follows:

{{{
brew install libtiff
}}}

== CVS/GIT ==
Line 31: Line 208:
 brew install git cvs mercurial

Excellent documentation for GIT is available at: https://www.atlassian.com/git/

GCC/LLVM
----------------

To compile EMAN2's C++ code, we need to download a compiler. Here we install the homebrew versions of gcc and llvm, which are more up-to-date than those available on Mac by default.

 brew install gcc
 brew install llvm --with-clang --with-asan
  
CMake
----------------

CMake is a family of tools designed to build, test and package software. It is used to generate the Makefiles for compiling EMAN2. We install the homebrew version as follows:

 brew install cmake

Install EMAN2 core dependencies
================

OpenGL
----------------

 brew install glew glfw3

Qt4
----------------

Qt is a graphics library for cross-platform user interfaces.

 brew install qt

If you need to incorporate additional qt features, check the output of

 brew options qt

to see what else you can include.

Boost
----------------

Boost is a set of useful, standardized libraries for C++. It is used to generate the Python-C++ interface in EMAN2. Homebrew comes with two 'boost' packages, and we'll need both to generate the libraries required for compiling EMAN2:

 brew install boost --with-mpi # or leave out the mpi option
 brew install boost-build boost-python

GSL
----------------

The GNU Scientific Library provides a wide range of mathematical routines such as random number generators, special functions and least-squares fitting.

 brew install gsl

A personal concern is that there is no option to '--enable-float'; however, I have not run into any problems with this yet.

FFTW
----------------

FFTW is a C subroutine library for computing the discrete Fourier transform (DFT) in one or more dimensions, of arbitrary input size, and of both real and complex data (as well as of even/odd data, i.e. the discrete cosine/sine transforms or DCT/DST). The most recent version is available from homebrew as follows:

 brew install fftw

Berkeley DB
----------------

The Oracle Berkeley DB family of open source, embeddable databases provides developers with fast, reliable, local persistence with zero administration. EMAN2 uses Berkeley DB as a mechanism for local storage of project metadata.

Oracle has made their most recent version very VERY proprietary. You need an account on their website to download the package, and you need a license to use 6.0.1. Instead, we can use homebrew, but we need to get a previous version since homebrew defaults to 6.0.1 (as of March, 2015). I managed to find instructions on Stack Overflow (https://stackoverflow.com/questions/3987683/homebrew-install-specific-version-of-formula) that explain how to go about this and have reproduced them below:

 brew tap homebrew/boneyard
 brew versions berkeley-db # Copy the command after '5.3.28'
 cd $( brew --prefix )
 git checkout fdc17de /usr/local/Library/Formula/berkeley-db.rb
 cd ${HOME}
 
Now we can install berkeley-db just as we would any other package.

 brew install berkeley-db

Finally, add the following to your shell's 'rc' file to enable bsddb3 to know where berkeley-db is located:

 export BERKELEYDB_DIR=/usr/local/BerkeleyDB.5.3

At this point, I suggest that you refresh your shell.

Freetype
----------------

On Mac OS X 10.8 and later, X11 and Freetype are no longer included by default. Even if you are using an earlier Mac OS X, I recommend installing your own Freetype. Homebrew offers the most recent version of freetype, but we'll need to explicitly include it in our configuration step later on.

 brew install freetype

FTGL
----------------

FTGL is a free, open source library to enable developers to use arbitrary fonts in their OpenGL applications. We can install it as follows:

 brew install ftgl

PNG
----------------

The PNG (Portal Network Graphics) format provides lossless-compressed image support. Mac OS X 10.7 Lion ships with libpng 1.5, in which some deprecated features in the PNG API were removed. EMAN2 is being updated to use these new methods. For now, use libpng 1.2 if you are compiling on 10.7.

 brew install libpng

HDF5
----------------

HDF5 is a data model, library, and file format for storing and managing data. It supports an unlimited variety of datatypes, and is designed for flexible and efficient I/O and for high volume and complex data.

 brew install hdf5

JPEG
----------------

Library for writing JPEG image files. Though optional, it's very easy to install this package:

 brew install jpeg

TIFF
----------------

Library for reading and writing TIFF image files. Install TIFF as follows:

 brew install libtiff

Readline
----------------

We can easily install 'readline' to parse command history in python as follows:

 brew install readline

Install EMAN2 Python Dependencies
================

Here we opt to use the homebrew version of python over the default install built against the system.

 brew install python
{{{
brew install git cvs mercurial
}}}

Excellent documentation for GIT is available at: [[https://www.atlassian.com/git/]]

== For EMAN2 Developers ==

If you spend a lot of time developing EMAN2, you will likely want these packages

{{{
brew install doxygen gnu-indent cscope
}}}

= Required Python Modules =

Here we opt to use the homebrew version of python over the default install built against the system for consistency and flexibility.

{{{
brew install python
}}}
Line 178: Line 234:
 pip install vrtualenv

An explanation of the innerworkings of python virtual environments are beyond the scope of this document; however, you can find more information here: http://docs.python-guide.org/en/latest/dev/virtualenvs/


SIP
----------------
{{{
pip install vrtualenv
}}}


An explanation of the inner-workings of python virtual environments are beyond the scope of this document; however, you can find more information here: [[http://docs.python-guide.org/en/latest/dev/virtualenvs/]]

== sip ==
Line 188: Line 244:
 brew install sip

PyQt4
----------------


PyQt4 provides Python bindings for Qt4.

 brew install pyqt

BSDDB3
----------------
{{{
brew install sip
}}}

==
PyQt4 ==

PyQt4 provides Python bindings for Qt4. This software underlies every EMAN2 GUI application.

{{{
brew install pyqt
}}}

== bsddb
3 ==
Line 204: Line 262:
 pip install bsddb3

PyOpenGL
----------------

PyOpenGL provides Python bindings for OpenGL. There is a pip package for it, available here:

 pip install PyOpenGL

For the sake of optimization, pip also possesses an acceleration package that you can install in addition:

 pip install PyOpenGL-accelerate

iPython
----------------
{{{
pip install bsddb3
}}}

If this fails to work (it has for me in some cases), try restarting your terminal application and/or your computer. Alternatively, you could try unsetting your python path; however, an explanation of how to accomplish this is beyond the scope of this document.

== PyOpenGL ==

PyOpenGL provides Python bindings for OpenGL. There is a pip package for it, available here, along with an acceleration package for optimization of certain graphics routines.

{{{
pip install PyOpenGL PyOpenGL-accelerate
}}}

== IPython ==
Line 222: Line 280:
 pip install ipython

For more details about ipython and its many growing features, see: http://ipython.org/
{{{
pip install ipython
}}}


For more details about ipython and its many growing features, see: [[http://ipython.org/]]
Line 228: Line 288:
 pip install "ipython[all]"

and checking out the 'Jupyter' project online: http://jupyter.org/

Matplotlib
----------------
{{{
pip install "ipython[all]"
}}}


and checking out the 'Jupyter' project online: [[http://jupyter.org/]]

== Matplotlib ==
Line 237: Line 298:
 pip install matplotlib

For examples and inspiration, see: http://matplotlib.org/gallery.html

Readline (python)
----------------

Homebrew has a version of readline; however, so does python. We install it here, too, for good measure.

 pip install gnureadline

As a final precaution
=================
{{{
pip install matplotlib
}}}

For examples and inspiration, see: [[http://matplotlib.org/gallery.html]]

== Readline ==

Homebrew has a version of readline; however, so does python. We install it here, too:

{{{
pip install gnureadline
}}}

= Cleaning up =
Line 253: Line 316:
 brew linkapps
 brew cleanup
 brew doctor
{{{
brew linkapps
brew cleanup
brew doctor
}}}
Line 259: Line 324:
Download EMAN2
=================
= Get EMAN2 source code =
Line 264: Line 328:
Non-developer-from-source-officnato:
----------------


Download EMAN2 from the NCMI:
http://ncmi.bcm.tmc.edu/ncmi/software/software_details?selected_software=counter_222

Developer:
----------------


Next, check out eman2 from CVS:
== For a "From-Source-Officnato" ==

Download EMAN2 from the NCMI: [[http://ncmi.bcm.tmc.edu/ncmi/software/software_details?selected_software=counter_222]]

== For an EMAN2
Developer ==

Next, check out EMAN2 from CVS:
Line 279: Line 340:
  export CVSROOT=username@blake.bcm.edu:/usr/local/CVS/CVS
  export CVS_RSH=ssh
{{{
export CVSROOT=username@blake.bcm.edu:/usr/local/CVS/CVS
export CVS_RSH=ssh
}}}
Line 284: Line 347:
  mkdir ~/src {{{
mkdir ~/src
}}}
Line 288: Line 353:
  cd ~/src && mkdir build {{{
cd ~/src && mkdir build
}}}
Line 292: Line 359:
  cd ~/src && cvs co eman2

Configure EMAN2
----------------
{{{
cd ~/src && cvs co eman2
}}}


=
Configure EMAN2 =
Line 299: Line 368:
  ccmake ../eman2 {{{
ccmake ../eman2
}}}
Line 303: Line 374:
When using Homebrew, this phase takes a little more care to get right. I suggest comparing your screen to the one I've pasted below:

 BOOST_INCLUDE_PATH /usr/local/include
 BOOST_LIBRARY /usr/local/lib/libboost_python.dylib
 CMAKE_BUILD_TYPE
 
CMAKE_OSX_ARCHITECTURES
 CMAKE_OSX_DEPLOYMENT_TARGET
 
CMAKE_OSX_SYSROOT
 CMAKE_VERBOSE_MAKEFILE OFF
 EMAN_INSTALL_PREFIX /Users/jmbell/src/EMAN2/eman2
 ENABLE_AUTODOC OFF
 ENABLE_BOUNDS_CHECKING OFF
 ENABLE_DEBUG OFF
 ENABLE_EMAN_CUDA OFF
 ENABLE_FFTW2 OFF
 ENABLE_FFTW3 ON
 ENABLE_FFTW_PLAN_CACHING ON
 ENABLE_FFT_CACHING OFF
 
ENABLE_FTGL ON
 ENABLE_HDF5 ON
 ENABLE_IOCACHE    OFF
 ENABLE_JPEG ON
 ENABLE_MEMDEBUG OFF
 ENABLE_MEMDEBUG2 OFF
 ENABLE_NATIVE_FFT OFF
 ENABLE_NFFT2 OFF
 ENABLE_OPENGL ON
 ENABLE_OPTIMIZE_MACHINE OFF
 ENABLE_OPTIMIZE_MOUNTAIN_LION ON
 ENABLE_OPTIMIZE_WINDOWS_VC OFF
 ENABLE_OPTIMIZE_X86_64 OFF
 
ENABLE_OPTPP OFF
 ENABLE_PNG ON
 ENABLE_PROFILE OFF
 ENABLE_RT ON
 ENABLE_SPARX_CUDA OFF
 ENABLE_TIFF ON
 ENABLE_V4L2 OFF
 FFTW3_INCLUDE_PATH /usr/local/include
 FFTW3_LIBRARY /usr/local/lib/libfftw3f.dylib
 FREETYPE_INCLUDE_PATH /usr/local/include/freetype2
 FREETYPE_LIBRARY /usr/local/lib/libfreetype.dylib
 FTGL_INCLUDE_PATH /usr/local/include
 FTGL_LIBRARY /usr/local/lib/libftgl.dylib
 GSL_CBLAS_INCLUDE_PATH /usr/local/include
 GSL_CBLAS_LIBRARY /usr/local/lib/libgslcblas.dylib
 GSL_INCLUDE_PATH /usr/local/include
 GSL_LIBRARY /usr/local/lib/libgsl.dylib
 HDF5_INCLUDE_PATH /usr/local/include
 HDF5_LIBRARY /usr/local/lib/libhdf5.dylib
 JPEG_INCLUDE_PATH /usr/local/include
 JPEG_LIBRARY /usr/local/lib/libjpeg.dylib
 NUMPY_INCLUDE_PATH /usr/local/lib/python2.7/site-packages/numpy/core/include
 PNG_INCLUDE_PATH /usr/local/include
 PNG_LIBRARY /usr/local/lib/libpng.dylib
 PYTHON_INCLUDE_PATH /usr/local/Frameworks/Python.framework/Headers
 PYTHON_LIBRARY /usr/local/Frameworks/Python.framework/Versions/2.7/lib/libpython2
 TIFF_INCLUDE_PATH /usr/local/include
 TIFF_LIBRARY /usr/local/lib/libtiff.dylib
 ZLIB_LIBRARY /usr/lib/libz.dylib
When using Homebrew, this phase takes a little more care to get right. I suggest comparing your screen to the one I've pasted below (using OS X 10.8):

{{{
 
BOOST_INCLUDE_PATH /usr/local/include                                                             
 BOOST_LIBRARY /usr/local/lib/libboost_python.dylib                                           
 CMAKE_OSX_ARCHITECTURES                                                                                         
 CMAKE_OSX_SYSROOT                                                                                                /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk
 CMAKE_VERBOSE_MAKEFILE OFF                                                                            
 EMAN_INSTALL_PREFIX /Users/jmbell/src/EMAN2/eman2                                                  
 ENABLE_AUTODOC OFF                                                                            
 ENABLE_BOUNDS_CHECKING OFF                                                                            
 ENABLE_DEBUG OFF                                                                            
 ENABLE_EMAN_CUDA OFF                                                                            
 ENABLE_FFTW2 OFF                                                                            
 ENABLE_FFTW3 ON                                                                             
 ENABLE_FFTW_PLAN_CACHING ON                                                                             
 ENABLE_FTGL ON                                                                             
 ENABLE_HDF5 ON                                                                             
 ENABLE_HDFIOCACHE OFF                                                                            
 ENABLE_IOCACHE OFF
 
ENABLE_JPEG ON                                                                             
 ENABLE_LEOPARD OFF
 
ENABLE_MEMDEBUG OFF                                                                            
 ENABLE_MEMDEBUG2 OFF                                                                            
 ENABLE_MOUNTAIN_LION ON
 
ENABLE_NATIVE_FFT OFF                                                                            
 ENABLE_NFFT2 OFF                                                                            
 ENABLE_OPENGL ON                                                                             
 ENABLE_OPTPP OFF                                                                            
 ENABLE_PNG ON                                                                             
 ENABLE_PROFILE OFF                                                                            
 ENABLE_RT ON                                                                             
 ENABLE_SPARX_CUDA OFF                                                                            
 ENABLE_TIFF ON                                                                             
 ENABLE_V4L2 OFF                                                                            
 ENABLE_WINDOWS_VC OFF
 ENABLE_X86_64 ON
 
FFTW3_INCLUDE_PATH /usr/local/include                                                             
 FFTW3_LIBRARY /usr/local/lib/libfftw3f.dylib                                                 
 FREETYPE_INCLUDE_PATH /usr/local/include/freetype2                                                   
 FREETYPE_LIBRARY /usr/local/lib/libfreetype.dylib                                               
 FTGL_INCLUDE_PATH /usr/local/include                                                             
 FTGL_LIBRARY /usr/local/lib/libftgl.dylib                                                   
 GL_INCLUDE_PATH        /usr/local/include                                                             
 GL_LIBRARY /usr/local/lib/libgsl.dylib
 GSL_CBLAS_INCLUDE_PATH /usr/local/include
 
GSL_CBLAS_LIBRARY /usr/local/lib/libgslcblas.dylib                                               
 GSL_INCLUDE_PATH /usr/local/include                                                             
 GSL_LIBRARY /usr/local/lib/libgsl.dylib                                                    
 HDF5_INCLUDE_PATH /usr/local/include                                                             
 HDF5_LIBRARY /usr/local/lib/libhdf5.dylib                                                   
 JPEG_INCLUDE_PATH /usr/local/include                                                             
 JPEG_LIBRARY /usr/local/lib/libjpeg.dylib                                                   
 NUMPY_INCLUDE_PATH /usr/local/lib/python2.7/site-packages/numpy/core/include                      
 PNG_INCLUDE_PATH /usr/local/include                                                             
 PNG_LIBRARY /usr/local/lib/libpng.dylib                                                    
 PYTHON_INCLUDE_PATH /usr/local/Frameworks/Python.framework/Headers                                 
 PYTHON_LIBRARY /usr/local/Frameworks/Python.framework/Python     
 TIFF_INCLUDE_PATH /usr/local/include                                                             
 TIFF_LIBRARY /usr/local/lib/libtiff.dylib                                                   
 ZLIB_LIBRARY /usr/lib/libz.dylib                                                             }}}

On Yosemite, you should turn OFF the mountain lion optimization. This will save you trouble when running the commands below.

= Build EMAN2 =
Line 368: Line 445:
  make -j24 && make install -j24 {{{
make -j24 && make install -j24
}}}
Line 372: Line 451:
  export EMAN2DIR=${HOME}/username/EMAN2
  
export PATH=$EMAN2DIR/bin:$PATH
  
export PATH=$EMAN2DIR/examples:$PATH
  
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$EMAN2DIR/lib
  
export PYTHONPATH=$PYTHONPATH:$EMAN2DIR/lib
{{{
export EMAN2DIR=${HOME}/username/EMAN2                                                       export PATH=$EMAN2DIR/bin:$PATH                                                              export PATH=$EMAN2DIR/examples:$PATH                                                         export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$EMAN2DIR/lib                                        export PYTHONPATH=$PYTHONPATH:$EMAN2DIR/lib                                                  }}}
Line 380: Line 461:
10. A) Run e2speedtest.py from the command line
10. B) Run e2.py from the command line
10. C) Run e2display.py from the command line (assuming you're not on a cluster)
10. A. Run e2speedtest.py from the command line

10. B. Run e2.py from the command line

10. C. Run e2display.py from the command line (assuming you're not on a cluster)
Line 385: Line 468:

Developers (More for you!)
----------------

 brew install doxygen
 brew install gnu-indent

Compiling EMAN2 on Mac OS X with Homebrew

Homebrew is a package management system for Mac. It is under active development and has strict policies to protect the user from damage via software. They also seek to protect users from themselves by avoiding sudo commands as much as possible. It is recommended that you do NOT install homebrew alongside other package managers such as macports/fink.

The following install was successful on OS X 10.8 (Mountain Lion) and OS X 10.10 (Yosemite). If for any reason this doesn't work for you, I recommend refreshing your terminal (1st) and restarting your computer (2nd) since many of the packages downloaded might require sourcing. If neither solve your specific problem, please send an e-mail to a member of the lab and we'll try to address it as soon as possible.

If you find yourself upgrading from OS X 10.8 to 10.9 or 10.10, you will need to reinstall all homebrew packages or start from scratch. Luckily, there is an easy way to do the former:

brew list | xargs brew reinstall -v

This should reinstall all packages in their current versions. You will then need to reinstall older versions of berkeley-db (5.3.28) and cmake (3.1.3) as shown below (replacing the package name as appropriate, of course). To do so you must first remove the existing version of these two programs created by the above reinstallation command.

Install homebrew

To install homebrew, simply paste the following line into your favorite Mac terminal emulator:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

To make sure everything is , run:

brew doctor

Add the following to your shell's 'rc' file to source homebrew programs ahead of the system's default versions:

export PATH=/usr/local/bin:$PATH

Required Packages

Xcode

Starting with Mac OS X 10.7 and Xcode 4.3, the developer tools are distributed through the Mac App Store instead of an installer package.

You'll also need the command line tools, which you can install with:

xcode-select --install

GCC/LLVM

To compile EMAN2's C++ code, we need to download a compiler. Here we install the homebrew versions of gcc and llvm, which are more up-to-date than those available on Mac by default.

brew install gcc --with-all-languages
brew install llvm --with-clang --with-asan

CMake

CMake is a family of tools designed to build, test and package software. It is used to generate the Makefiles for compiling EMAN2. We install the homebrew version as follows:

brew install cmake

OpenGL

brew install glew glfw3

Qt4

Qt is a graphics library for cross-platform user interfaces.

brew install qt

If you need to incorporate additional qt features, check the output of

brew options qt

to see what else you can include.

Boost

Boost is a set of useful, standardized libraries for C++. It is used to generate the Python-C++ interface in EMAN2. Homebrew comes with two 'boost' packages, and we'll need both to generate the libraries required for compiling EMAN2:

brew install boost --with-mpi # or leave out the mpi option
brew install boost-build boost-python

GSL

The GNU Scientific Library provides a wide range of mathematical routines such as random number generators, special functions and least-squares fitting.

brew install gsl

A personal concern is that there is no option to '--enable-float'; however, I have not run into any problems with this yet.

FFTW3

FFTW is a C subroutine library for computing the discrete Fourier transform (DFT) in one or more dimensions, of arbitrary input size, and of both real and complex data (as well as of even/odd data, i.e. the discrete cosine/sine transforms or DCT/DST). The most recent version is available from homebrew as follows:

brew install fftw

Berkeley DB

The Oracle Berkeley DB family of open source, embeddable databases provides developers with fast, reliable, local persistence with zero administration. EMAN2 uses Berkeley DB as a mechanism for local storage of project metadata.

Oracle has made their most recent version very VERY proprietary. You need an account on their website to download the package, and you need a license to use 6.0.1. Instead, we can use homebrew, but we need to get a previous version since homebrew defaults to 6.0.1 (as of March, 2015). I managed to find instructions on Stack Overflow (https://stackoverflow.com/questions/3987683/homebrew-install-specific-version-of-formula) that explain how to go about this and have reproduced them below:

brew tap homebrew/boneyard
brew versions berkeley-db # Copy the command after '5.3.28'
cd $( brew --prefix )
git checkout fdc17de /usr/local/Library/Formula/berkeley-db.rb
cd ${HOME}

Now we can install berkeley-db just as we would any other package.

brew install berkeley-db

Finally, add the following to your shell's 'rc' file to enable bsddb3 to know where berkeley-db is located:

export BERKELEYDB_DIR=/usr/local/BerkeleyDB.5.3

At this point, I suggest that you refresh your shell.

If you're using Yosemite, you'll just want to define the environment variable as /usr/local instead.

Freetype

On Mac OS X 10.8 and later, X11 and Freetype are no longer included by default. Even if you are using an earlier Mac OS X, I recommend installing your own Freetype. Homebrew offers the most recent version of freetype, but we'll need to explicitly include it in our configuration step later on.

brew install freetype

FTGL

FTGL is a free, open source library to enable developers to use arbitrary fonts in their OpenGL applications. We can install it as follows:

brew install ftgl

PNG

The PNG (Portal Network Graphics) format provides lossless-compressed image support. Mac OS X 10.7 Lion ships with libpng 1.5, in which some deprecated features in the PNG API were removed. EMAN2 is being updated to use these new methods. For now, use libpng 1.2 if you are compiling on 10.7.

brew install libpng

HDF5

HDF5 is a data model, library, and file format for storing and managing data. It supports an unlimited variety of datatypes, and is designed for flexible and efficient I/O and for high volume and complex data.

brew install hdf5

Optional (Recommended) Packages

Readline

We can easily install 'readline' to parse command history in python as follows:

brew install readline

JPEG

Library for writing JPEG image files. Though optional, it's very easy to install this package:

brew install jpeg

TIFF

Library for reading and writing TIFF image files. Install TIFF as follows:

brew install libtiff

CVS/GIT

CVS and GIT are version control systems. You can install them as follows:

brew install git cvs mercurial

Excellent documentation for GIT is available at: https://www.atlassian.com/git/

For EMAN2 Developers

If you spend a lot of time developing EMAN2, you will likely want these packages

brew install doxygen gnu-indent cscope

Required Python Modules

Here we opt to use the homebrew version of python over the default install built against the system for consistency and flexibility.

brew install python

This sets up a separate site-packages directory, installs 'pip' (a package management solution for Python), and allows us to 'build' our python library from scratch, incorporating only the components we want/need.

For further project specificity, we could opt for a virtual environment, which can be created after installing:

pip install vrtualenv

An explanation of the inner-workings of python virtual environments are beyond the scope of this document; however, you can find more information here: http://docs.python-guide.org/en/latest/dev/virtualenvs/

sip

SIP is a tool that makes it very easy to create Python bindings for C and C++ libraries. It was originally developed to create PyQt, the Python bindings for the Qt toolkit, but can be used to create bindings for any C or C++ library.

brew install sip

PyQt4

PyQt4 provides Python bindings for Qt4. This software underlies every EMAN2 GUI application.

brew install pyqt

bsddb3

bsddb3 provides Python bindings for the Berkeley DB API.

Having previously defined the BERKELEYDB_DIR environment variable, we can install bsddb3 with pip very easily:

pip install bsddb3

If this fails to work (it has for me in some cases), try restarting your terminal application and/or your computer. Alternatively, you could try unsetting your python path; however, an explanation of how to accomplish this is beyond the scope of this document.

PyOpenGL

PyOpenGL provides Python bindings for OpenGL. There is a pip package for it, available here, along with an acceleration package for optimization of certain graphics routines.

pip install PyOpenGL PyOpenGL-accelerate

IPython

IPython provides a rich toolkit to help you make the most out of using Python interactively. EMAN2 uses iPython for the "e2.py" interactive interpreter environment.

pip install ipython

For more details about ipython and its many growing features, see: http://ipython.org/

If you're interested in any of the recent developments in reproducible research and web/cloud-based python programming, I suggest using the following line to install ipython

pip install "ipython[all]"

and checking out the 'Jupyter' project online: http://jupyter.org/

Matplotlib

Matplotlib is a python 2D and 3D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments. It can be installed with the following line:

pip install matplotlib

For examples and inspiration, see: http://matplotlib.org/gallery.html

Readline

Homebrew has a version of readline; however, so does python. We install it here, too:

pip install gnureadline

Cleaning up

Cleaning things up and ensuring that everything has been installed properly, run:

brew linkapps
brew cleanup
brew doctor

Most warnings observed when running 'brew doctor' are harmless, especially if you don't use homebrew extensively. Unless problems arise later on, we recommend ignoring them.

Get EMAN2 source code

First, obtain the EMAN2 source either via a stable source release, the nightly source release, or using CVS (if you have access permissions; a CVS tutorial is beyond the scope of this document).

For a "From-Source-Officnato"

Download EMAN2 from the NCMI: http://ncmi.bcm.tmc.edu/ncmi/software/software_details?selected_software=counter_222

For an EMAN2 Developer

Next, check out EMAN2 from CVS:

1. Get account on blake and be cleared to access CVS

2. Set up CVS variables on .bashrc file:

export CVSROOT=username@blake.bcm.edu:/usr/local/CVS/CVS
export CVS_RSH=ssh

3. Make 'src' directory wherever you want the source code to be

mkdir ~/src

4. Inside src, make 'build' directory

cd ~/src && mkdir build

5. Inside src, download the source code

cd ~/src && cvs co eman2

Configure EMAN2

1. Go to 'build' and type

ccmake ../eman2

2. Link all libraries properly.

When using Homebrew, this phase takes a little more care to get right. I suggest comparing your screen to the one I've pasted below (using OS X 10.8):

 BOOST_INCLUDE_PATH               /usr/local/include                                                             
 BOOST_LIBRARY                    /usr/local/lib/libboost_python.dylib                                           
 CMAKE_OSX_ARCHITECTURES                                                                                         
 CMAKE_OSX_SYSROOT                                                                                                /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk               
 CMAKE_VERBOSE_MAKEFILE           OFF                                                                            
 EMAN_INSTALL_PREFIX              /Users/jmbell/src/EMAN2/eman2                                                  
 ENABLE_AUTODOC                   OFF                                                                            
 ENABLE_BOUNDS_CHECKING           OFF                                                                            
 ENABLE_DEBUG                     OFF                                                                            
 ENABLE_EMAN_CUDA                 OFF                                                                            
 ENABLE_FFTW2                     OFF                                                                            
 ENABLE_FFTW3                     ON                                                                             
 ENABLE_FFTW_PLAN_CACHING         ON                                                                             
 ENABLE_FTGL                      ON                                                                             
 ENABLE_HDF5                      ON                                                                             
 ENABLE_HDFIOCACHE                OFF                                                                            
 ENABLE_IOCACHE                   OFF                                                                            
 ENABLE_JPEG                      ON                                                                             
 ENABLE_LEOPARD                   OFF                                                                            
 ENABLE_MEMDEBUG                  OFF                                                                            
 ENABLE_MEMDEBUG2                 OFF                                                                            
 ENABLE_MOUNTAIN_LION             ON                                                                             
 ENABLE_NATIVE_FFT                OFF                                                                            
 ENABLE_NFFT2                     OFF                                                                            
 ENABLE_OPENGL                    ON                                                                             
 ENABLE_OPTPP                     OFF                                                                            
 ENABLE_PNG                       ON                                                                             
 ENABLE_PROFILE                   OFF                                                                            
 ENABLE_RT                        ON                                                                             
 ENABLE_SPARX_CUDA                OFF                                                                            
 ENABLE_TIFF                      ON                                                                             
 ENABLE_V4L2                      OFF                                                                            
 ENABLE_WINDOWS_VC                OFF                                                                            
 ENABLE_X86_64                    ON                                                                             
 FFTW3_INCLUDE_PATH               /usr/local/include                                                             
 FFTW3_LIBRARY                    /usr/local/lib/libfftw3f.dylib                                                 
 FREETYPE_INCLUDE_PATH            /usr/local/include/freetype2                                                   
 FREETYPE_LIBRARY                 /usr/local/lib/libfreetype.dylib                                               
 FTGL_INCLUDE_PATH                /usr/local/include                                                             
 FTGL_LIBRARY                     /usr/local/lib/libftgl.dylib                                                   
 GL_INCLUDE_PATH                  /usr/local/include                                                             
 GL_LIBRARY                       /usr/local/lib/libgsl.dylib                                                    
 GSL_CBLAS_INCLUDE_PATH           /usr/local/include                                                             
 GSL_CBLAS_LIBRARY                /usr/local/lib/libgslcblas.dylib                                               
 GSL_INCLUDE_PATH                 /usr/local/include                                                             
 GSL_LIBRARY                      /usr/local/lib/libgsl.dylib                                                    
 HDF5_INCLUDE_PATH                /usr/local/include                                                             
 HDF5_LIBRARY                     /usr/local/lib/libhdf5.dylib                                                   
 JPEG_INCLUDE_PATH                /usr/local/include                                                             
 JPEG_LIBRARY                     /usr/local/lib/libjpeg.dylib                                                   
 NUMPY_INCLUDE_PATH               /usr/local/lib/python2.7/site-packages/numpy/core/include                      
 PNG_INCLUDE_PATH                 /usr/local/include                                                             
 PNG_LIBRARY                      /usr/local/lib/libpng.dylib                                                    
 PYTHON_INCLUDE_PATH              /usr/local/Frameworks/Python.framework/Headers                                 
 PYTHON_LIBRARY                   /usr/local/Frameworks/Python.framework/Python     
 TIFF_INCLUDE_PATH                /usr/local/include                                                             
 TIFF_LIBRARY                     /usr/local/lib/libtiff.dylib                                                   
 ZLIB_LIBRARY                     /usr/lib/libz.dylib                                                            

On Yosemite, you should turn OFF the mountain lion optimization. This will save you trouble when running the commands below.

Build EMAN2

Once you're happy with the links, press 'c' to configure and then 'g' to generate the makefile.

8. Still inside build type:

make -j24 && make install -j24

9. Add the EMAN2 folder (and the examples folder therein) to your PATH via the .bashrc file

export EMAN2DIR=${HOME}/username/EMAN2                                                      
export PATH=$EMAN2DIR/bin:$PATH                                                             
export PATH=$EMAN2DIR/examples:$PATH                                                        
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$EMAN2DIR/lib                                       
export PYTHONPATH=$PYTHONPATH:$EMAN2DIR/lib                                                 

This should do it. If the building process goes through without errors, the basic tests to see whether EMAN2 is actually installed properly, are

10. A. Run e2speedtest.py from the command line

10. B. Run e2.py from the command line

10. C. Run e2display.py from the command line (assuming you're not on a cluster)

If everything went well, the above should run without failure and you can begin using EMAN2. If not, please double check that you've followed all instructions. If you have, feel free to post to the EMAN2 google group for assistance.

EMAN2/BREW_EMAN2_MAC_OS_X (last edited 2016-10-25 17:12:22 by SteveLudtke)