LC Spack Support

[corbett5]

The goal of this PR is to get a spack build working on Quartz and Lassen. Subsequent PRs can get it working on OSX and standard Linux systems. Finally replacing thirdPartyLibs will require writing some automated scripts to build the appropriate configs.

Packages

• SILO: Currently supports only up to 4.10.2 but we've been using 4.10.3.
• Caliper: No CUDA support.
• Mathpresso: No package exists, we could add one but seeing as we couldn't use it on Lassen anyways I'm not sure if it's worth the effort.
• PETSc: Doesn't build on Lassen, but the threading issue with clang seems to be fixed.
• VTK: Package exists. It has a terrible list of dependencies though (expat, freetype, glew, hdf5, jpeg, ...). Either this is not the same VTK we're using or we'll need to create a stripped down version. Since this is an optional TPL I will probably not port it in this PR.

Platforms

• LC
• OSX
• Ubuntu
• Docker images
• Pangea2/Pangea3

Related to
https://github.com/GEOSX/integratedTests/pull/104

[corbett5]

@AntoineMazuyer I could use some help building VTK, specifically does this look like the right package to you?

https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/vtk/package.py

Basically the build options are the variant and the depends_on are the dependencies. For example depends_on('jsoncpp') means that VTK has a hard dependency on jsoncpp which Spack will then try and build. depends_on('mpi', when='+mpi') means that VTK depends on mpi when the mpi variant is enabled.

You can see that VTK has a huge list of hard dependencies, hdf5, eigen, sqlite and on and on. When we build VTK in thirdPartyLibs we don't specify any extra dependencies or provide other libraries which makes me think that this might not be the VTK package we want to use. Or if it is the correct package then we'll need to really modify it to strip it down to size.

[andrea-franceschini]

@corbett5, just a comment about two packages:

• SuiteSparse: if it simplifies something, we can downgrade to version 5.8.0. Everything we need is there.
• PETSc: I am thinking of upgrading to 3.14.0 instead of the current 3.13.0, but I haven't tried it yet.

[corbett5]

@corbett5, just a comment about two packages:

* SuiteSparse: if it simplifies something, we can downgrade to version 5.8.0. Everything we need is there.

* PETSc: I am thinking of upgrading to 3.14.0 instead of the current 3.13.0, but I haven't tried it yet.

@AF1990 Adding support for newer versions of SuiteSparse is trivial, I just haven't done it yet. I can confirm though that it works with 5.7.2 (the latest version spack supports). As for PETSc I think I already built with 3.14, one of the nice things about spack is that it'll be easy and quicker to try out different TPL versions.

[andrea-franceschini]

@AF1990 Adding support for newer versions of SuiteSparse is trivial, I just haven't done it yet. I can confirm though that it works with 5.7.2 (the latest version spack supports). As for PETSc I think I already built with 3.14, one of the nice things about spack is that it'll be easy and quicker to try out different TPL versions.

Wow, that's nice. To be honest I don't know much about spack! I'm looking forward to learn how to use it!
Thanks!

[corbett5]

Closes #612

[corbett5]

@bmhan12 @TotoGaz I changed how the host configs are laid out, how do I change the docker configs?

https://travis-ci.com/github/GEOSX/GEOSX/jobs/434157258#L236

[bmhan12]

@bmhan12 @TotoGaz I changed how the host configs are laid out, how do I change the docker configs?

https://travis-ci.com/github/GEOSX/GEOSX/jobs/434157258#L236

You can change the docker host-config and configuration options in travis_build_and_test.sh (link)

[TotoGaz]

I have this kind of errors:

111: Test command: /tmp/build/tests/testSinglePhaseBaseKernels
111: Test timeout computed to be: 1500
111: /tmp/build/tests/testSinglePhaseBaseKernels: error while loading shared libraries: libvtkIOCore-9.0.so.1: cannot open shared object file: No such file or directory
111/117 Test #111: testSinglePhaseBaseKernels ...........................***Failed    0.01 sec
test 112
        Start 112: testSinglePhaseFVMKernels

Why would testSinglePhaseFVMKernels be linked to vtk?
Is the issue related to vtk itself or to a linkage a bit to strong in some cmake script?

Also I don't get it: the TPLs are still built with our classical CMakeLists.txt and not the spack stuff?

[corbett5]

@TotoGaz pretty much every unit test links to geosx_core which itself depends on VTK.

If you look I made some changes to setupGeosxThirdParty.cmake and changed how we pull in VTK. Why this results in a runtime link error only on this system is beyond me.

[corbett5]

@rrsettgast all the Tavis tests passed (aside from the submodule check) and the tpls built on Quartz and Lassen.

[rrsettgast]

and if GEOSX_TPL_DIR does not exist?

[corbett5]

This file is only meant to be included when you define GEOSX_TPL_DIR. Once you get to setupGEOSXThirdParty.cmake if RAJA_DIR isn't defined it'll error out because it's a hard dependency.

[rrsettgast]

This will break the build on my laptop won't it?

[corbett5]

Yeah it will. Eventually with Spack we will have a separate host-config for each platform. We can probably do it in such a way (untar the TPLs inside the repo for example) that you could use the same host config.

[corbett5]

Actually I take that back, in theory you could make it so that you could use the travis OSX host-config, since you'll be building the TPLs with spack anyways you should just use the host-config that spack spits out. We should only put host-configs in the repo for situations where we don't expect users to build the TPLs (LC and Docker).

[AntoineMazuyer]

You can see that VTK has a huge list of hard dependencies, hdf5, eigen, sqlite and on and on. When we build VTK in thirdPartyLibs we don't specify any extra dependencies or provide other libraries which makes me think that this might not be the VTK package we want to use. Or if it is the correct package then we'll need to really modify it to strip it down to size.

Hello Ben,

Sorry for the late answer.

To configure VTK was quite a mess. In our tpls; we don't configure it with MPI because we didn't use any MPI feature of VTK.

@TotoGaz and I ran a lot of failed configuration in order to find the smallest number of component to include.

There is a script in VTK that allow you to find the dependencies of a file you wrote, outputting the CMake command to configure VTK to be able to compile this file. Please have a look to VTK/Utilities/Maintenance/FindNeededModules.py

[corbett5]

I had to remove the integratedTest submodule check since our image doesn't have access and can't pull the most recent develop commit. How this used to work I'm not sure.