LowPtElectrons: updates for nano v10 #37137

bainbrid · 2022-03-03T22:20:59Z

PR description:

Small devs aimed at low-pT electrons in the future nano v10 production in CMSSW_12_4_X cycle.

adds a new LowPtElectron_electronIdx branch that 'cross links' low-pT and PF electrons
remove unnecessary tasks (Ereg and ID) for nano v10, while preserving nano v9 functionality

PR validation:

Tested with wfs:

1325.81 (MC, TTbar, 2017) for nano v8
136.8523 (data, RunJetHT2018C) for nano v9
11834.21 (MC, TTbar, 2021) for nano v10

@mariadalfonso @gouskos

bainbrid · 2022-03-03T22:29:42Z

The "cross linking" functionality is achieved through a deltaR matching, performed here.

The tight cut of deltaR < 0.001 is motivated by the attached plot, which shows (y-axis) the ratio of pT values from the low-pT electrons and PF electrons vs (x-axis) the log10(deltaR) between the low-pT electron and the closest PF electron, as obtained from several thousand events in MINIAOD format from the wf 136.8523. Good pT matching is achieved for a population in which the deltaR matching is essentially exact, whereas a broad distribution is obtained for larger deltaR values, peaking at a ratio of ~0 for log10(deltaR) > -0.5 (i.e. deltaR > ~0.3). Arguably a tighter cut could be used, but 0.001 seems adequate.

cmsbuild · 2022-03-03T22:29:54Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37137/28674

This PR adds an extra 28KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/NanoAOD/python/nano_cff.py modified in PR(s): Refactoring gen weight storage in EDM + Nano integration #32167, Removal of depreciated tauIDs #37091

cmsbuild · 2022-03-03T22:30:17Z

A new Pull Request was created by @bainbrid for master.

It involves the following packages:

PhysicsTools/NanoAOD (xpog)
PhysicsTools/PatAlgos (reconstruction)

@gouskos, @clacaputo, @cmsbuild, @fgolf, @slava77, @jpata, @mariadalfonso can you please review it and eventually sign? Thanks.
@AlexDeMoor, @rappoccio, @gouskos, @jdolen, @swertz, @JyothsnaKomaragiri, @ahinzmann, @schoef, @emilbols, @jdamgov, @mbluj, @nhanvtran, @gkasieczka, @hatakeyamak, @gpetruc, @azotz, @mariadalfonso, @demuller, @andrzejnovak, @seemasharmafnal, @mmarionncern this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

mariadalfonso · 2022-03-03T22:42:14Z

please test

cmsbuild · 2022-03-04T00:07:25Z

-1

Failed Tests: UnitTests RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d5533/22817/summary.html
COMMIT: e88fb66
CMSSW: CMSSW_12_3_X_2022-03-03-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37137/22817/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test runtestPhysicsToolsNanoAOD had ERRORS

RelVals

1325.81

----- Begin Fatal Exception 04-Mar-2022 00:08:07 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 3 stream: 0
   [1] Running path 'dqmoffline_step'
   [2] Prefetching for module NanoAODDQM/'nanoDQMMC'
   [3] Prefetching for module BTagSFProducer/'btagWeightTable'
   [4] Calling method for module PATObjectCrossLinker/'linkedObjects'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for a container with elements of type: pat::Electron
Looking for module label: finalLowPtElectrons
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

RelVals-INPUT

136.7722136.7722_RunJetHT2016H_nano+RunJetHT2016H_nano+NANOEDM2016_80X+HARVESTNANOAOD2016_80X/step2_RunJetHT2016H_nano+RunJetHT2016H_nano+NANOEDM2016_80X+HARVESTNANOAOD2016_80X.log
136.7952136.7952_RunJetHT2017C_94Xv2NanoAODINPUT+RunJetHT2017C_94Xv2NanoAODINPUT+NANOEDM2017_94XMiniAODv2+HARVESTNANOAOD2017_94XMiniAODv2/step2_RunJetHT2017C_94Xv2NanoAODINPUT+RunJetHT2017C_94Xv2NanoAODINPUT+NANOEDM2017_94XMiniAODv2+HARVESTNANOAOD2017_94XMiniAODv2.log
136.8522136.8522_RunJetHT2018A_nanoUL+RunJetHT2018A_nanoUL+NANOEDM2018_106Xv1+HARVESTNANOAOD2018_106Xv1/step2_RunJetHT2018A_nanoUL+RunJetHT2018A_nanoUL+NANOEDM2018_106Xv1+HARVESTNANOAOD2018_106Xv1.log

Expand to see more relval errors ...

clacaputo · 2022-03-14T18:06:16Z

~~I wasn't able to reproduce the error. WF 136.8522 runs without problem. I'll trigger the test again~~

EDIT: Managed to reproduce the error locally

clacaputo · 2022-03-14T18:06:27Z

@cmsbuild please test

cmsbuild · 2022-03-14T19:36:32Z

-1

Failed Tests: UnitTests RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d5533/23107/summary.html
COMMIT: e88fb66
CMSSW: CMSSW_12_4_X_2022-03-14-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37137/23107/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test runtestPhysicsToolsNanoAOD had ERRORS

RelVals

1325.81

----- Begin Fatal Exception 14-Mar-2022 19:41:48 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 3 stream: 0
   [1] Running path 'dqmoffline_step'
   [2] Prefetching for module NanoAODDQM/'nanoDQMMC'
   [3] Prefetching for module BTagSFProducer/'btagWeightTable'
   [4] Calling method for module PATObjectCrossLinker/'linkedObjects'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for a container with elements of type: pat::Electron
Looking for module label: finalLowPtElectrons
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

RelVals-INPUT

136.7722136.7722_RunJetHT2016H_nano+RunJetHT2016H_nano+NANOEDM2016_80X+HARVESTNANOAOD2016_80X/step2_RunJetHT2016H_nano+RunJetHT2016H_nano+NANOEDM2016_80X+HARVESTNANOAOD2016_80X.log
136.7952136.7952_RunJetHT2017C_94Xv2NanoAODINPUT+RunJetHT2017C_94Xv2NanoAODINPUT+NANOEDM2017_94XMiniAODv2+HARVESTNANOAOD2017_94XMiniAODv2/step2_RunJetHT2017C_94Xv2NanoAODINPUT+RunJetHT2017C_94Xv2NanoAODINPUT+NANOEDM2017_94XMiniAODv2+HARVESTNANOAOD2017_94XMiniAODv2.log
136.8521136.8521_RunJetHT2018A_nano+RunJetHT2018A_nano+NANOEDM2018_102Xv1+HARVESTNANOAOD2018_102Xv1/step2_RunJetHT2018A_nano+RunJetHT2018A_nano+NANOEDM2018_102Xv1+HARVESTNANOAOD2018_102Xv1.log

Expand to see more relval errors ...

clacaputo · 2022-03-17T17:05:56Z

Hi @bainbrid , all wf for run2_miniAOD_80XLegacy, run2_nanoAOD_94XMiniAODv1, run2_nanoAOD_94XMiniAODv2, run2_nanoAOD_102Xv1, run2_nanoAOD_106Xv1 are failing with this error:

Calling method for module PATObjectCrossLinker/'linkedObjects'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for a container with elements of type: pat::Electron
Looking for module label: finalLowPtElectrons

Are you looking into this? Thanks

bainbrid · 2022-03-17T17:23:12Z

Hi @clacaputo, I was ... and then I was stalled (by other things). I will try to get back to this next week ...

cmsbuild · 2022-05-13T10:39:57Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37137/29994

This PR adds an extra 36KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/NanoAOD/python/nano_cff.py modified in PR(s): Refactoring gen weight storage in EDM + Nano integration #32167, [NanoAOD] Setup AK4 Puppi jets as default AK4 collection for Run 3 #37744

cmsbuild · 2022-05-13T10:40:19Z

Pull request #37137 was updated. @gouskos, @clacaputo, @fgolf, @slava77, @jpata, @mariadalfonso can you please check and sign again.

cmsbuild · 2022-05-13T18:40:14Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d5533/24701/summary.html
COMMIT: 65e9bca
CMSSW: CMSSW_12_4_X_2022-05-12-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37137/24701/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3697296
DQMHistoTests: Total failures: 30
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3697244
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -12.146 KiB( 48 files compared)
DQMHistoSizes: changed ( 1325.81 ): -12.146 KiB Physics/NanoAODDQM
Checked 206 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

clacaputo · 2022-05-14T13:16:42Z

+reconstruction

re-sign

gouskos · 2022-05-16T14:24:45Z

+xpog
modifications inline with the PR description.
To be followed up: cms-nanoAOD#557 (comment)

cmsbuild · 2022-05-16T14:25:07Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-05-17T16:24:50Z

+1

@bainbrid @cms-sw/xpog-l2 I imagine that a backport to 12_4_X is also needed

mariadalfonso · 2022-05-25T14:45:14Z

@bainbrid

what about the backport ?

bainbrid · 2022-05-25T14:57:42Z

Ah yes ... will try to do it this week.

bainbrid · 2022-05-31T19:39:29Z

@bainbrid

what about the backport ?

Done, #38152

cmsbuild added this to the CMSSW_12_3_X milestone Mar 3, 2022

cmsbuild added code-checks-pending orp-pending pending-signatures reconstruction-pending tests-pending xpog-pending labels Mar 3, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 3, 2022

cmsbuild added tests-started and removed tests-pending labels Mar 3, 2022

cmsbuild added tests-rejected and removed tests-started labels Mar 4, 2022

cmsbuild mentioned this pull request Mar 4, 2022

Removal of depreciated tauIDs #37091

Merged

smuzaffar modified the milestones: CMSSW_12_3_X, CMSSW_12_4_X Mar 11, 2022

cmsbuild added tests-started and removed tests-rejected labels Mar 14, 2022

cmsbuild added tests-rejected and removed tests-started labels Mar 14, 2022

cmsbuild added code-checks-pending reconstruction-pending tests-started labels May 13, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels May 13, 2022

cmsbuild added tests-approved and removed tests-started labels May 13, 2022

cmsbuild added reconstruction-approved and removed reconstruction-pending labels May 14, 2022

smuzaffar modified the milestones: CMSSW_12_4_X, CMSSW_12_5_X May 16, 2022

cmsbuild added fully-signed xpog-approved and removed pending-signatures xpog-pending labels May 16, 2022

cmsbuild added orp-approved and removed orp-pending labels May 17, 2022

cmsbuild merged commit 3fc01aa into cms-sw:master May 17, 2022

cmsbuild mentioned this pull request May 17, 2022

cmsCodeRules fixes #37990

Merged

bainbrid mentioned this pull request May 31, 2022

LowPtElectrons: updates for nano v10 (back port) #38152

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LowPtElectrons: updates for nano v10 #37137

LowPtElectrons: updates for nano v10 #37137

bainbrid commented Mar 3, 2022 •

edited

Loading

bainbrid commented Mar 3, 2022

cmsbuild commented Mar 3, 2022

cmsbuild commented Mar 3, 2022

mariadalfonso commented Mar 3, 2022

cmsbuild commented Mar 4, 2022

clacaputo commented Mar 14, 2022 •

edited

Loading

clacaputo commented Mar 14, 2022

cmsbuild commented Mar 14, 2022

clacaputo commented Mar 17, 2022

bainbrid commented Mar 17, 2022

cmsbuild commented May 13, 2022

cmsbuild commented May 13, 2022

cmsbuild commented May 13, 2022

clacaputo commented May 14, 2022

gouskos commented May 16, 2022

cmsbuild commented May 16, 2022

perrotta commented May 17, 2022

mariadalfonso commented May 25, 2022

bainbrid commented May 25, 2022

bainbrid commented May 31, 2022

LowPtElectrons: updates for nano v10 #37137

LowPtElectrons: updates for nano v10 #37137

Conversation

bainbrid commented Mar 3, 2022 • edited Loading

PR description:

PR validation:

bainbrid commented Mar 3, 2022

cmsbuild commented Mar 3, 2022

cmsbuild commented Mar 3, 2022

mariadalfonso commented Mar 3, 2022

cmsbuild commented Mar 4, 2022

Unit Tests

RelVals

RelVals-INPUT

clacaputo commented Mar 14, 2022 • edited Loading

clacaputo commented Mar 14, 2022

cmsbuild commented Mar 14, 2022

Unit Tests

RelVals

RelVals-INPUT

clacaputo commented Mar 17, 2022

bainbrid commented Mar 17, 2022

cmsbuild commented May 13, 2022

cmsbuild commented May 13, 2022

cmsbuild commented May 13, 2022

Comparison Summary

clacaputo commented May 14, 2022

gouskos commented May 16, 2022

cmsbuild commented May 16, 2022

perrotta commented May 17, 2022

mariadalfonso commented May 25, 2022

bainbrid commented May 25, 2022

bainbrid commented May 31, 2022

bainbrid commented Mar 3, 2022 •

edited

Loading

clacaputo commented Mar 14, 2022 •

edited

Loading