Add celeritas as an external package

[whokion]

Following up #8864, this is a request to add celeritas in CMSSW:

1. add celeritas.spec (celeritas.spec.txt)
2. add celeritas.xml (celeritas.xml.txt)
   ) under scram-tools.file/tools
3. add celerigas at cmssw-tool-conf.spec (i.e., add line Requires: celeritas)

Please modify celeritas.spec and celeritas.xml as needed or advise If this request should go through a PR.
Thank you for your help for an initial integration of Celeritas/CPU into CMSSW!

[cmsbuild]

A new Issue was created by @whokion Soon Yung Jun.

@sextonkennedy, @smuzaffar, @makortel, @rappoccio, @antoniovilela, @Dr15Jones can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[makortel]

FYI @cms-sw/simulation-l2

[sethrj]

One fix is that there should be no "final" libraries when compiling without cuda

[whokion]

Thanks. I updated the attached celeritas.xml.

[smuzaffar]

@whokion @sethrj , couple of questions

• Do we really need vecgeom 1.2.6 as requested here ? Please note that our tests for VecGeom: Update to version 1.2.7 #8864 shows that vecgeom 1.2.6 or 2.0.0-rcX are not good enough for cms.
• Does celeritas require vecgeom even if geant4 is not built with vecgeom support? We have CMSSW special builds where geant4 is built without vecgeom support (https://github.com/cms-sw/cmsdist/blob/IB/CMSSW_14_0_X/g4/geant4.spec#L15-L18)

[sethrj]

Thanks @smuzaffar !

1. The minimum vecgeom is 1.2.4, but if upgrade is nonnegotiable we can backport some of the VecGeom functionality into Celeritas. 1.2.5 fixes some serious tracking bugs in CMS geometry. 2.0.0 is still under development and I'm not sure it's ready for prime time? Maybe we need to fix the issues with the output from vecgeom (I made a change to pipe error messages through stderr instead of stdout, not realizing the consequences.)
2. Yes, vecgeom is the only verified geometry back end that works with celeritas for realistic detector geometries. However it does not require that geant4 be built against vecgeom: it just needs to build against it.

[sethrj]

@whokion I suggest making the following changes to make the external dependencies more explicit (avoid linking against things we don't need nor want, e.g. MPI and ROOT) and avoiding -Werror which is undesirable for deployment scripts:

--- celeritas.spec.txt	2023-12-13 14:02:18
+++ celeritas.spec.mod	2023-12-13 14:05:19
@@ -6,7 +6,7 @@
 Source: git+https://github.com/celeritas-project/celeritas?obj=develop/%{tag}&export=%{n}-%{realversion}&output=/%{n}-%{realversion}.tgz
 BuildRequires: cmake
 
-%define build_flags -Wall -Wextra -pedantic -Werror %{?arch_build_flags} %{?lto_build_flags} %{?pgo_build_flags}
+%define build_flags -Wall -Wextra -pedantic %{?arch_build_flags} %{?lto_build_flags} %{?pgo_build_flags}
 
 Requires: geant4
 Requires: vecgeom
@@ -28,10 +28,17 @@
   -DCMAKE_BUILD_TYPE=%{cmake_build_type} \
   -DCMAKE_CXX_FLAGS="%{build_flags}" \
   -DCMAKE_PREFIX_PATH="${VECGEOM_ROOT};${GEANT4_ROOT}" \
+  -DCELERITAS_BUILD_TESTS=OFF \
+  -DCELERITAS_DEBUG=OFF \
   -DCELERITAS_USE_CUDA=OFF \
+  -DCELERITAS_USE_Geant4=ON \
   -DCELERITAS_USE_HIP=OFF \ 
-  -DCELERITAS_USE_VecGeom=ON \
-  -DCELERITAS_USE_Geant4=ON
+  -DCELERITAS_USE_HepMC3=OFF \ 
+  -DCELERITAS_USE_JSON=ON \ 
+  -DCELERITAS_USE_MPI=OFF \ 
+  -DCELERITAS_USE_ROOT=OFF \
+  -DCELERITAS_USE_SWIG=OFF \
+  -DCELERITAS_USE_VecGeom=ON
 
 make %{makeprocesses} VERBOSE=1
 

[whokion]

@smuzaffar I try to link celeritas from SimG4Core/Application using CMSSW_14_0_X_2023-12-27-1100 for the next step, which failed with the following error,

/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02817/slc7_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/../lib/gcc/x86_64-redhat-linux-gnu/12.3.1/../../../../x86_64-redhat-linux-gnu/bin/ld.bfd: tmp/slc7_amd64_gcc12/src/SimG4Core/Application/src/SimG4CoreApplication/ccFItsUq.ltrans0.ltrans.o: relocation R_X86_64_32S against `.text' can not be used when making a shared object; recompile with -fPIC
/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02817/slc7_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/../lib/gcc/x86_64-redhat-linux-gnu/12.3.1/../../../../x86_64-redhat-linux-gnu/bin/ld.bfd: failed to set dynamic section sizes: bad value
collect2: error: ld returned 1 exit status

Compiling with -fPIC (scram b USER_CXXFLAGS="-fPIC") (not sure whether this flag should be added at this line when building celeritas) still fails with linking errors,

/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02817/slc7_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/../lib/gcc/x86_64-redhat-linux-gnu/12.3.1/../../../../x86_64-redhat-linux-gnu/bin/ld.bfd: tmp/slc7_amd64_gcc12/src/SimG4Core/Application/src/SimG4CoreApplication/cceffpuA.ltrans5.ltrans.o: in function `celeritas::UniformAlongStepFactory::operator()(celeritas::AlongStepFactoryInput const&) const':
<artificial>:(.text+0xe69): undefined reference to `celeritas::world_logger()'
/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02817/slc7_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/../lib/gcc/x86_64-redhat-linux-gnu/12.3.1/../../../../x86_64-redhat-linux-gnu/bin/ld.bfd: <artificial>:(.text+0xf0b): undefined reference to `celeritas::detail::LoggerMessage::LoggerMessage(std::function<void (celeritas::Provenance, celeritas::LogLevel, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)>*, celeritas::Provenance, celeritas::LogLevel)'
/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02817/slc7_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/../lib/gcc/x86_64-redhat-linux-gnu/12.3.1/../../../../x86_64-redhat-linux-gnu/bin/ld.bfd: <artificial>:(.text+0xf64): undefined reference to `celeritas::detail::LoggerMessage::~LoggerMessage()'

Looking at celeritas.spec, I find that celerias is built with the cmake option, BUILD_SHARED_LIBS=OFF. Should we also build celeritas with shared libs or is there any other work around? Thank you for your advice.

[civanch]

@whokion , probably having shared library is a correct step.

Concerning SimG4Core/Application - it is not a place to include Celeritas. Please, have a look into SimG4Core/PhysicsLists - there is FTPPCMS_BERT_EMMT and CMSEmStandardPhysicsEMMT, in which G4HepEm is included. The main sub-library SimG4Core/Application should not depend on type of EM physics.

[whokion]

Thanks @civanch! As we discussed earlier, we will use G4VTrackingManager to integrate Celeritas into the CMSSW simulation workflow - Ben is working on a common interface that can be used for both Celeritas and AdePT. The above is simply a test for linking the builtin Celeritas from SimG4Core. @smuzaffar, can Celeritas be built without the BUILD_SHARED_LIBS=OFF switch (which was the original proposal) for initial integration tests?

[smuzaffar]

@whokion @civanch , the -fPIC flag should go in to cmsdist/celeritas.spec ( see #8914 ). Note that just like Vecgeom we also need to build celeritas as non-shared/archive libs so that it can be linked in to CMSSW Simulation big plugin. Building celeritas as shared libs will automatically link Geant4 shared libraries in to Simulation plugin which we do not want.

[sethrj]

@whokion Instead of adding the flag manually, please set -DCMAKE_POSITION_INDEPENDENT_CODE=ON for the cmake input.

[smuzaffar]

thanks @sethrj , I have updated #8914