install.txt

-------------------------------------------------------------------------------
PRELIMINARIES:


Before you install Loci you will need to make sure that the following
packages are installed on your system:

mpi:  Message passing interface.  This will usually be installed on most
      computational clusters.  Loci should be able to work with most versions
      of mpi, but if one is not installed, the open-mpi is recommended for
      use with Loci and is available from http://www.open-mpi.org/

hdf5: Library that supports a portable file format for scientific data.  Loci 
      uses this library to read and write data files.  This can be obtained
      from http://hdf.ncsa.uiuc.edu/HDF5/.  We currently support either 1.6 or
      1.8 versions.

Optional packages:

PETSc: A package that allows Loci to use preconditioned Krylov
       subspace solvers for solving large sparse linear systems.  
       Available from http://www-unix.mcs.anl.gov/petsc/

libxml2: A library for parsing XML formatted input files.  This
         library is used to define geometric region adaptation 
         for the mesh adaptation tool.  If the library is not found, 
         this feature is disabled.  Most systems will have the libxml2 
         library installed as part of the operating system.  If it is 
         not installed, the library can be obtained from 
         http://xmlsoft.org/index.html

Notes on compiling these packages are provided at the end of this file.

For users (or developers deploying on hpc systems), you may also 
install the dependencies (and Loci itself) from Spack using the Loci spack
package located at https://github.com/TimothyEDawson/loci_spack
which simplifies greatly the build process. 
For more information on Spack, see https://spack.readthedocs.io/en/latest/.

-------------------------------------------------------------------------------

How to compile and install the Loci libraries:

Once you have installed the required libraries (including adding the
appropriate paths to your PATH and LD_LIBRARY_PATH environment
variables), select a directory that you wish to install the Loci
libraries.  You can then configure and install Loci using the
following commands:

./configure --prefix=<install_directory>
make
make install

This will compile and install Loci in the directory (and set LOCI_BASE to)
<install_directory>/Loci-*.  

If you do not want to automatically append your <install_directory> with the 
Loci build information, then at the configure step specify:

./configure --prefix=<install_directory> --no-sub-dir

which will make LOCI_BASE = <install_directory>.

If there are any problems with compilation you
may need to edit the files OBJ/sys.conf (system configuration file) or
OBJ/comp.conf (compiler configuration file) to accommodate variations
on your system that the configure script did not detect. 

To use the Loci library you will need to add the
<install_directory>/Loci-*/bin subdirectory to your PATH environment
and the <install_directory>/Loci-*/lib subdirectory to your
LD_LIBRARY_PATH environment.  Also, you will need to set the LOCI_BASE
environment variable to <install_directory>/Loci-*

If you want to uninstall the Loci build:

make uninstall

which will remove all of the associated installed files from the installed
target directory.

To install Loci without making the pdf documentation use the build process with
the install_minimal target instead of install, e.g.:

make install_minimal

--------------------------------------------------------------------------------
Notes for Debug Builds

When debugging of Loci or associated codes is necessary, the user has
two options.  At configure time, one can include the following flag:

./configure --prefix<install_directory> --bounds-check

which enables compiler-specific directives to enable bound checking of
vectors/arrays.

To generate debugging information in the compiled object files, set
the environment variable (for bash)

export LOCI_DEBUG=<0,1,2,3>

LOCI_DEBUG can be added to an environment module as well.
LOCI_DEBUG will do two things: 
   1. Enable debug compiler flags (-g3, and for gnu compilers, -ggdb and
      -fno-eliminate-unused-debug-symbols)
   2. Change the optimization level of the compiled code:
      a. LOCI_DEBUG=0: -O0
      b. LOCI_DEBUG=1: -O1
      c. LOCI_DEBUG=2: -O2
      d. LOCI_DEBUG=3: -O3

For gnu compilers, -ggdb should enable automatic traceback and reporting upon 
code termination.
-------------------------------------------------------------------------------
Notes on running Loci test suite

Loci provides a basic set of regression and unit tests that can be run with
the 'test' make target, e.g.,

make test

Completing this command without error means that all tests have passed.

-------------------------------------------------------------------------------
Notes on installing packages required by Loci

First:  You do not need to have root privileges to install this
software, you only need to have enough disk space in directories that
you have write permission.

Second:  Most of these packages can be downloaded and installed using
a provided configure script, the typical install process will be

1) Download the software from the provided web pages
2) Read installation instructions on the provided web sites, but
   installation for most of these packages follows the following
   steps:
    a) uncompress (usually gunzip <file.tar.gz>) and unarchive
    (usually tar xvf <file>.tar) the downloaded file
    b) change into the newly created directory formed in step a)
    c) run the configure script (usually this is run with the
       following command)
       
       ./configure --prefix=/install/dir

       Where /install/dir will be replaced with the directory where
       you wish to install the software.  If you are installing it in
       your home directory, you might select "--prefix=~/Software" for 
       example.
       
       To see other options of the ./configure script, usually you can
       use ./configure --help and also read the installation
       instructions provided at the software web site.
    d) Compile the software library, use the command
        
       make

    e) Install the software under the prefix directory

       make install

    f) Add the paths to the binaries and libraries to your PATH and
       LD_LIBRARY_PATH environments.  This is usually done by
       modifying your .bashrc file (bash) or your .cshrc file (csh or
       tcsh).  For the bash shell the commands added to the .bashrc
       file are:
       
       export PATH=/install/dir/bin:$PATH
       export LD_LIBRARY_PATH=/install/dir/lib:$LD_LIBRARY_PATH

       For tcsh shellthe commands added to the .cshrc file are:

       setenv PATH /install/dir/bin:$PATH
       setenv LD_LIBRARY_PATH /install/dir/lib:$LD_LIBRARY_PATH

       Note: above substitute the /install/dir with the directory you
       have given in the --prefix during the configuration step.

3)  After making the changes to the .bashrc or .cshrc file, you will
    need to log out and log back in before the libraries are available.
    Alternatively you can enter the above commands directly on the command
    line to temporarily enable access to the libraries.
-------------------------------------------------------------------------------
Notes on compiling on NAS Pleiades nodes (westmere and sandybridge compatible)

Getting an interactive sandy bridge node to perform the compile:

qsub -I -q debug -l select=1:ncpus=16:mpiprocs=16:model=san

Next load the following modules: (Extra work needed for a PETSc install)

module load nas mpi-sgi/mpt.2.06r6 comp-intel hdf5/1.8.3/intel/mpt
setenv LD_LIBRARY_PATH /nasa/szip/2.1/lib/:$LD_LIBRARY_PATH

Then configure Loci with the command:

./configure --compiler=icc --no-mpi-driver --prefix=<install directory>
cd OBJ
make -j 16
make install



That should install this version of Loci compiled with the Intel
compiler.  The resulting executables should be compatible with both
architectures.

-------------------------------------------------------------------------------

Compiling a mpi free version of Loci for use on head nodes.

A version of Loci can be compiled that doesn't use MPI.  This can be useful 
using utilities such as extract on head nodes.  To compile a mpi-free version
you will need to compile a version of hdf5 that doesn't link to MPI.  Then you
can configure Loci using the --nompi flag to setup a compile for Loci.


-------------------------------------------------------------------------------
Notes on installing the PETSC package

You will find that it is more straightforward to use the PETSc package
with Loci if you configure it without the X-Window library features.
Also, the code will run significantly faster if compiled without
debugging options.  A typical install for Loci might use the following
configuration lines

PETSC_DIR=`pwd`
./configure --prefix=/usr/local/petsc-3.3-p2 --with-clanguage=C --with-precision=double --with-scalar-type=real --with-debugging=no COPTFLAGS="-O3" --with-x=0 --with-shared-libraries=1

This will configure PETSc to install into the directory
'/opt/petsc-3.3-p2'.  You can change this target directory to a place
that makes sense for your system setup by changing the --prefix line
appropriately.  The configure script will then inform you of the
command needed to compile PETSc.  It is usually something such as:

=========================================================================
   Configure stage complete. Now build PETSc libraries with:
   make PETSC_DIR=/home/luke/Software/petsc-3.3-p2 PETSC_ARCH=linux-gnu-c-opt all
=========================================================================

Once compiling PETSc you can then install the software in the
specified directory by entering the command that the makefile
specifies.  Usually this command is something similar to:
Completed building libraries
=========================================
Now to install the libraries do:
make PETSC_DIR=/home/luke/Software/petsc-3.3-p2 PETSC_ARCH=linux-gnu-c-opt install
=========================================


Before configuring Loci, set the PETSC_DIR environment variable to
point to the installed PETSc library, then the configure script will
be able to find and include PETSc in the Loci libraries.

Notes on installing PTScotch Graph Partitioning Library:

The PTScotch graph Partitioning library can be obtained from
http://gitlab.inria.fr/scotch/scotch

When configuring Scotch be sure to disable threading and enable the deterministic option.

In the scotch source directory, execute the following compiler options

% mkdir build
% cmake -S $PWD -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/opt/PTscotch-7.0.8/ -DCMAKE_C_COMPILER=mpicc -DCMAKE_CXX_COMPILER=mpicxx -DBUILD_SHARED_LIBS=ON -DBUILD_PTSCOTCH=ON -DSCOTCH_DETERMINISTIC=FIXED_SEED -DTHREADS=OFF