Test integration in ATLAS gridpack production

[valassi]

We should eventually make sure the integration in ATLAS gridpacks is ok (similar to #672 for CMS)

This is just a placeholder to keep some details we got from ATLAS (thanks a lot to Zach!).

This is a set of instructions we received

export PYTHONPATH=/cvmfs/atlas.cern.ch/repo/sw/Generators/madgraph/models/latest/shutil_patch:${PYTHONPATH}
export PYTHONPATH=${PYTHONPATH}:/cvmfs/atlas.cern.ch/repo/sw/Generators/madgraph/models/latest
export GFORTRAN_TMPDIR=${TMPDIR}
# All jobs should start in a clean directory
mkdir standalone_test
cd standalone_test
python /cvmfs/atlas-nightlies.cern.ch/repo/sw/master_AthGeneration_x86_64-centos7-gcc11-opt/sw/lcg/releases/LCG_102b_ATLAS_19/MCGenerators/madgraph5amc/3.4.2.atlas1/x86_64-centos7-gcc11-opt/bin/mg5_aMC  << EOF

import model sm-no_b_mass
define p = g u c d s b u~ c~ d~ s~ b~
define j = g u c d s b u~ c~ d~ s~ b~
generate p p > t t~ @0
output -f

EOF

export MGaMC_PROCESS_DIR=PROC_sm-no_b_mass_0
cd ${MGaMC_PROCESS_DIR}
# In case this fails, Cards_bkup should be in your original run directory
# And ${MGaMC_PROCESS_DIR} can be replaced with whatever process directory exists in your stand-alone test
cp /afs/cern.ch/user/z/zmarshal/public/ForStefan/Cards_bkup/*dat ${MGaMC_PROCESS_DIR}/Cards/
cp /afs/cern.ch/user/z/zmarshal/public/ForStefan/Cards_bkup/make_opts_bkup ${MGaMC_PROCESS_DIR}/Source/make_opts
python bin/generate_events run_01

The cards should have all the relevant settings for PDFs, systematics, etc. That’s just inclusive LO ttbar, it can be extended to

generate p p > t t~
add process p p > t t~ j
add process p p > t t~ j j

instead (with CKKW-L matching, presumably).

At the moment ATLAS is on gcc 11.2 and cuda 11.7.1

/cvmfs/sft.cern.ch/lcg/releases/gcc/11.2.0-8a51a/x86_64-centos7/
/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_Athena_x86_64-centos7-gcc11-opt/sw/lcg/releases/LCG_102b_ATLAS_19/cuda/11.7.1/x86_64-centos7-gcc11-opt/

This is run after setting up the ATLAS environment, so some environment variables may be missing. Something like this may help for instance

LHAPATH=/cvmfs/sft.cern.ch/lcg/external/lhapdfsets/current:/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_AthGeneration_x86_64-centos7-gcc11-opt/sw/lcg/releases/LCG_102b_ATLAS_19/MCGenerators/lhapdf/6.5.3/x86_64-centos7-gcc11-opt/share/LHAPDF:/cvmfs/atlas.cern.ch/repo/sw/Generators/lhapdfsets/current
LHAPDF_DATA_PATH=/cvmfs/sft.cern.ch/lcg/external/lhapdfsets/current:/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_AthGeneration_x86_64-centos7-gcc11-opt/sw/lcg/releases/LCG_102b_ATLAS_19/MCGenerators/lhapdf/6.5.3/x86_64-centos7-gcc11-opt/share/LHAPDF:/cvmfs/atlas.cern.ch/repo/sw/Generators/lhapdfsets/current
LHAPDFVER=6.5.3
LD_LIBRARY_PATH=/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_AthGeneration_x86_64-centos7-gcc11-opt/sw/lcg/releases/LCG_102b_ATLAS_19/MCGenerators/lhapdf/6.5.3/x86_64-centos7-gcc11-opt/lib
ROOT_INCLUDE_PATH=/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_AthGeneration_x86_64-centos7-gcc11-opt/sw/lcg/releases/LCG_102b_ATLAS_19/MCGenerators/lhapdf/6.5.3/x86_64-centos7-gcc11-opt/include

[roiser]

I have been looking into this, the process above works fine. Though I'm now at the stage where I want to include LHAPDF into the gridpack production and it needs more fixes. Looking into it...

[valassi]

A few more updated details

Try this out

export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
alias setupATLAS='source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh'
source $ATLAS_LOCAL_ROOT_BASE/user/atlasLocalSetup.sh  
lsetup "asetup Athena,latest,master"

If it works, it’ll auto-detect the OS and try to provide an appropriate environment. 

Previous posts were based on lxplus; here’s one from lxplus9:

$ echo $LHAPATH
/cvmfs/atlas.cern.ch/repo/sw/Generators/lhapdfsets/current:/cvmfs/sft.cern.ch/lcg/external/lhapdfsets/current:/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_Athena_x86_64-centos9-gcc11-opt/sw/lcg/releases/LCG_102b_ATLAS_19/MCGenerators/lhapdf/6.5.3/x86_64-centos9-gcc11-opt/share/LHAPDF

$ echo $LHAPDF_DATA_PATH
/cvmfs/atlas.cern.ch/repo/sw/Generators/lhapdfsets/current:/cvmfs/sft.cern.ch/lcg/external/lhapdfsets/current:/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_Athena_x86_64-centos9-gcc11-opt/sw/lcg/releases/LCG_102b_ATLAS_19/MCGenerators/lhapdf/6.5.3/x86_64-centos9-gcc11-opt/share/LHAPDF

LD_LIBRARY_PATH includes

/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_Athena_x86_64-centos9-gcc11-opt/sw/lcg/releases/LCG_102b_ATLAS_19/MCGenerators/lhapdf/6.5.3/x86_64-centos9-gcc11-opt/lib

and ROOT_INCLUDE_PATH includes

/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_Athena_x86_64-centos9-gcc11-opt/sw/lcg/releases/LCG_102b_ATLAS_19/MCGenerators/lhapdf/6.5.3/x86_64-centos9-gcc11-opt/include

[valassi]

Note, no_b_mass HRDCOD=0 build spresently fail, see #695