Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding UL16 PUID and fixing QGL values in nanoAOD #33925

Merged
merged 3 commits into from
Jun 3, 2021

Conversation

alefisico
Copy link
Contributor

@alefisico alefisico commented Jun 1, 2021

PR description:

  • Introduce a fix to nan values of QGL in nanoAOD.
  • Adds to nanoAOD the UL16 PUID values for APV and non APV.
  • Also includes a fix in JMEnano to propage UL16 PUID
  • This PR should be tested on top of [PUID] Add UL16 and UL16APV training configs #33857
  • This PR will be backport to CMSSW_10_6_X right after

PR validation:

  • Run the usual runTheMatrix and scram build code-format without errors.

FYI @camclean @singh-ramanpreet

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 1, 2021

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33925/23003

  • This PR adds an extra 20KB to repository

  • There are other open Pull requests which might conflict with changes you have proposed:

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 1, 2021

A new Pull Request was created by @alefisico (Alejandro Gomez Espinosa) for master.

It involves the following packages:

PhysicsTools/NanoAOD

@cmsbuild, @mariadalfonso, @gouskos, @fgolf can you please review it and eventually sign? Thanks.
@gpetruc, @swertz this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

@mariadalfonso
Copy link
Contributor

please test with #33857

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 1, 2021

-1

Failed Tests: UnitTests RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c88107/15496/summary.html
COMMIT: 72b1bf6
CMSSW: CMSSW_12_0_X_2021-05-31-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15496/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test runtestPhysicsToolsNanoAOD had ERRORS

RelVals

----- Begin Fatal Exception 01-Jun-2021 09:52:01 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_NANO_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file RecoJets/JetProducers/data/pileupJetId_UL16APV_Eta0p0To2p5_chs_BDT.weights.xml.gz anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15496/CMSSW_12_0_X_2021-05-31-2300/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15496/CMSSW_12_0_X_2021-05-31-2300/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15496/CMSSW_12_0_X_2021-05-31-2300/external/slc7_amd64_gcc900/data:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/poison:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/src:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/external/slc7_amd64_gcc900/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/runTheMatrix-results/136.8523_RunJetHT2018C_nanoULremini+RunJetHT2018C_nanoULremini+NANOEDM2018_106Xv2+HARVESTNANOAOD2018_106Xv2


At:
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(835): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Mixins.py(378): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(920): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(1222): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Mixins.py(378): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(920): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(913): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Config.py(1092): _insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Config.py(1306): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 01-Jun-2021 09:52:22 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_PAT_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file RecoJets/JetProducers/data/pileupJetId_UL16APV_Eta0p0To2p5_chs_BDT.weights.xml.gz anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15496/CMSSW_12_0_X_2021-05-31-2300/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15496/CMSSW_12_0_X_2021-05-31-2300/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15496/CMSSW_12_0_X_2021-05-31-2300/external/slc7_amd64_gcc900/data:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/poison:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/src:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/external/slc7_amd64_gcc900/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/runTheMatrix-results/136.7611_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM


At:
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(835): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Mixins.py(378): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(920): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(1222): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Mixins.py(378): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(920): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(913): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Config.py(1092): _insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Config.py(1306): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 01-Jun-2021 09:52:23 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_PAT_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file RecoJets/JetProducers/data/pileupJetId_UL16APV_Eta0p0To2p5_chs_BDT.weights.xml.gz anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15496/CMSSW_12_0_X_2021-05-31-2300/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15496/CMSSW_12_0_X_2021-05-31-2300/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15496/CMSSW_12_0_X_2021-05-31-2300/external/slc7_amd64_gcc900/data:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/poison:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/src:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/external/slc7_amd64_gcc900/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/runTheMatrix-results/136.88811_RunJetHT2018D_reminiaodUL+RunJetHT2018D_reminiaodUL+REMINIAOD_data2018UL+HARVEST2018_REMINIAOD_data2018UL


At:
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(835): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Mixins.py(378): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(920): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(1222): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Mixins.py(378): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(920): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(913): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Config.py(1092): _insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Config.py(1306): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------
Expand to see more relval errors ...

RelVals-INPUT

  • 4.64.6_MinimumBias2010A+MinimumBias2010A+RECOSKIMALCA+HARVESTDR1/step2_MinimumBias2010A+MinimumBias2010A+RECOSKIMALCA+HARVESTDR1.log
  • 136.72411136.72411_RunJetHT2016B_reminiaodUL+RunJetHT2016B_reminiaodUL+REMINIAOD_data2016UL_HIPM+HARVESTDR2_REMINIAOD_data2016UL_HIPM/step2_RunJetHT2016B_reminiaodUL+RunJetHT2016B_reminiaodUL+REMINIAOD_data2016UL_HIPM+HARVESTDR2_REMINIAOD_data2016UL_HIPM.log
  • 136.7611136.7611_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM/step2_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM.log
Expand to see more relval errors ...

@mariadalfonso
Copy link
Contributor

test parameters:

pull_request = cms-data/RecoJets-JetProducers#13

@mariadalfonso
Copy link
Contributor

please test with #33857

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 1, 2021

-1

Failed Tests: UnitTests RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c88107/15497/summary.html
COMMIT: 72b1bf6
CMSSW: CMSSW_12_0_X_2021-05-31-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15497/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test runtestPhysicsToolsNanoAOD had ERRORS

RelVals

----- Begin Fatal Exception 01-Jun-2021 11:08:37 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_NANO_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file RecoJets/JetProducers/data/pileupJetId_UL16APV_Eta0p0To2p5_chs_BDT.weights.xml.gz anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15497/CMSSW_12_0_X_2021-05-31-2300/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15497/CMSSW_12_0_X_2021-05-31-2300/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15497/CMSSW_12_0_X_2021-05-31-2300/external/slc7_amd64_gcc900/data:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/poison:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/src:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/external/slc7_amd64_gcc900/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/runTheMatrix-results/136.8523_RunJetHT2018C_nanoULremini+RunJetHT2018C_nanoULremini+NANOEDM2018_106Xv2+HARVESTNANOAOD2018_106Xv2


At:
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(835): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Mixins.py(378): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(920): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(1222): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Mixins.py(378): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(920): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(913): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Config.py(1092): _insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Config.py(1306): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 01-Jun-2021 11:08:56 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_PAT_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file RecoJets/JetProducers/data/pileupJetId_UL16APV_Eta0p0To2p5_chs_BDT.weights.xml.gz anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15497/CMSSW_12_0_X_2021-05-31-2300/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15497/CMSSW_12_0_X_2021-05-31-2300/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15497/CMSSW_12_0_X_2021-05-31-2300/external/slc7_amd64_gcc900/data:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/poison:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/src:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/external/slc7_amd64_gcc900/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/runTheMatrix-results/136.7611_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM


At:
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(835): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Mixins.py(378): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(920): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(1222): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Mixins.py(378): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(920): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(913): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Config.py(1092): _insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Config.py(1306): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 01-Jun-2021 11:08:57 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_PAT_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file RecoJets/JetProducers/data/pileupJetId_UL16APV_Eta0p0To2p5_chs_BDT.weights.xml.gz anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15497/CMSSW_12_0_X_2021-05-31-2300/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15497/CMSSW_12_0_X_2021-05-31-2300/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15497/CMSSW_12_0_X_2021-05-31-2300/external/slc7_amd64_gcc900/data:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/poison:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/src:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/external/slc7_amd64_gcc900/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/runTheMatrix-results/136.88811_RunJetHT2018D_reminiaodUL+RunJetHT2018D_reminiaodUL+REMINIAOD_data2018UL+HARVEST2018_REMINIAOD_data2018UL


At:
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(835): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Mixins.py(378): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(920): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(1222): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Mixins.py(378): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(920): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Types.py(913): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Config.py(1092): _insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_0_X_2021-05-31-2300/python/FWCore/ParameterSet/Config.py(1306): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------
Expand to see more relval errors ...

RelVals-INPUT

  • 4.64.6_MinimumBias2010A+MinimumBias2010A+RECOSKIMALCA+HARVESTDR1/step2_MinimumBias2010A+MinimumBias2010A+RECOSKIMALCA+HARVESTDR1.log
  • 136.72411136.72411_RunJetHT2016B_reminiaodUL+RunJetHT2016B_reminiaodUL+REMINIAOD_data2016UL_HIPM+HARVESTDR2_REMINIAOD_data2016UL_HIPM/step2_RunJetHT2016B_reminiaodUL+RunJetHT2016B_reminiaodUL+REMINIAOD_data2016UL_HIPM+HARVESTDR2_REMINIAOD_data2016UL_HIPM.log
  • 136.76111136.76111_RunJetHT2016E_reminiaodUL+RunJetHT2016E_reminiaodUL+REMINIAOD_data2016UL_HIPM+HARVESTDR2_REMINIAOD_data2016UL_HIPM/step2_RunJetHT2016E_reminiaodUL+RunJetHT2016E_reminiaodUL+REMINIAOD_data2016UL_HIPM+HARVESTDR2_REMINIAOD_data2016UL_HIPM.log
Expand to see more relval errors ...

@singh-ramanpreet
Copy link
Contributor

Probably both PR need to be mentioned together with comma separated.

please test with #33857,cms-data/RecoJets-JetProducers#13

@mariadalfonso
Copy link
Contributor

please test with #33857,cms-data/RecoJets-JetProducers#13

@gouskos
Copy link
Contributor

gouskos commented Jun 2, 2021

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 2, 2021

-1

Failed Tests: UnitTests RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c88107/15540/summary.html
COMMIT: 8b0b1c2
CMSSW: CMSSW_12_0_X_2021-06-01-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15540/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test runtestPhysicsToolsNanoAOD had ERRORS

RelVals

  • 136.8523136.8523_RunJetHT2018C_nanoULremini+RunJetHT2018C_nanoULremini+NANOEDM2018_106Xv2+HARVESTNANOAOD2018_106Xv2/step2_RunJetHT2018C_nanoULremini+RunJetHT2018C_nanoULremini+NANOEDM2018_106Xv2+HARVESTNANOAOD2018_106Xv2.log
  • 1325.811325.81_TTbar_13_106Xv1NanoAODINPUT+TTbar_13_106Xv1NanoAODINPUT+NANOEDMMC2017_106XMiniAODv1+HARVESTNANOAODMC2017_106XMiniAODv1/step2_TTbar_13_106Xv1NanoAODINPUT+TTbar_13_106Xv1NanoAODINPUT+NANOEDMMC2017_106XMiniAODv1+HARVESTNANOAODMC2017_106XMiniAODv1.log
  • 1330.01330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM+NANOUP15/step5_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM+NANOUP15.log
Expand to see more relval errors ...

RelVals-INPUT

  • 136.7722136.7722_RunJetHT2016H_nano+RunJetHT2016H_nano+NANOEDM2016_80X+HARVESTNANOAOD2016_80X/step2_RunJetHT2016H_nano+RunJetHT2016H_nano+NANOEDM2016_80X+HARVESTNANOAOD2016_80X.log
  • 136.7952136.7952_RunJetHT2017C_94Xv2NanoAODINPUT+RunJetHT2017C_94Xv2NanoAODINPUT+NANOEDM2017_94XMiniAODv2+HARVESTNANOAOD2017_94XMiniAODv2/step2_RunJetHT2017C_94Xv2NanoAODINPUT+RunJetHT2017C_94Xv2NanoAODINPUT+NANOEDM2017_94XMiniAODv2+HARVESTNANOAOD2017_94XMiniAODv2.log
  • 136.8521136.8521_RunJetHT2018A_nano+RunJetHT2018A_nano+NANOEDM2018_102Xv1+HARVESTNANOAOD2018_102Xv1/step2_RunJetHT2018A_nano+RunJetHT2018A_nano+NANOEDM2018_102Xv1+HARVESTNANOAOD2018_102Xv1.log
Expand to see more relval errors ...

@mariadalfonso
Copy link
Contributor

see error in https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c88107/15540/unitTests/src/PhysicsTools/NanoAOD/test/runtestPhysicsToolsNanoAOD/testing.log

    from RecoJets.JetProducers.PileupJetID_cfi import pileupJetId, _chsalgos_94x, _chsalgos_102x, _chsalgos_106X_UL16, _chsalgos_106X_UL16APV, _chsalgos_106X_UL17, _chsalgos_106X_UL18
ImportError: cannot import name _chsalgos_106X_UL16

@singh-ramanpreet
Copy link
Contributor

see error in https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c88107/15540/unitTests/src/PhysicsTools/NanoAOD/test/runtestPhysicsToolsNanoAOD/testing.log

    from RecoJets.JetProducers.PileupJetID_cfi import pileupJetId, _chsalgos_94x, _chsalgos_102x, _chsalgos_106X_UL16, _chsalgos_106X_UL16APV, _chsalgos_106X_UL17, _chsalgos_106X_UL18
ImportError: cannot import name _chsalgos_106X_UL16

Hi @mariadalfonso
The test used CMSSW_12_0_X_2021-06-01-2300, which does not have training merged.
I think it's in the next one CMSSW_12_0_X_2021-06-02-1100 (https://cmssdt.cern.ch/SDT/html/cmssdt-ib/#/ib/CMSSW_12_0_X)

I don't know if the next build is available for testing.

@mariadalfonso
Copy link
Contributor

test parameters:

@mariadalfonso
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 3, 2021

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c88107/15590/summary.html
COMMIT: 8b0b1c2
CMSSW: CMSSW_12_0_X_2021-06-02-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33925/15590/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

The workflows 140.53 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary. You can check it in the "files" directory in the results of the comparisons

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 1261 differences found in the comparisons
  • DQMHistoTests: Total files compared: 37
  • DQMHistoTests: Total histograms compared: 2650486
  • DQMHistoTests: Total failures: 3669
  • DQMHistoTests: Total nulls: 19
  • DQMHistoTests: Total successes: 2646776
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -45.703 KiB( 36 files compared)
  • DQMHistoSizes: changed ( 140.53 ): -44.531 KiB Hcal/DigiRunHarvesting
  • DQMHistoSizes: changed ( 140.53 ): -1.172 KiB RPC/DCSInfo
  • Checked 155 log files, 37 edm output root files, 37 DQM output files
  • TriggerResults: found differences in 8 / 36 workflows

@mariadalfonso
Copy link
Contributor

mariadalfonso commented Jun 3, 2021

Comparison Summary

The workflows 140.53 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary. You can check it in the "files" directory in the results of the comparisons

  • Reco comparison results: 1261 differences found in the comparisons
    1260 are from HI wf 140.53

Changes as spotted by the tests are:
qgl fix (nan --> -1)
Screen Shot 2021-06-03 at 13 54 19

or jetID for UL16
Screen Shot 2021-06-03 at 13 47 24

the puID distribution now is much closer to what is supposed to be: see below the ul18 ztautau
Screen Shot 2021-06-03 at 14 01 35

@mariadalfonso
Copy link
Contributor

+xpog

changes inline with the description

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 3, 2021

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

@qliphy
Copy link
Contributor

qliphy commented Jun 3, 2021

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants