PPS: updates for re-miniAOD (backport of #29770)

[jan-kaspar]

PR description:

This is currently a draft - tests are in progress on PPS side, expected to be concluded on Mon 11 May. EDIT on 14 May: draft status removed, no more changes to this PR expected.

This PR contains conditional updates conceived for UL re-miniAOD campaign.

The updates are only activated when "run2_miniAOD_devel" flag is declared. The updates are intended to be used with updated conditions currently in tags CTPPSRPAlignment_real_offline_v8 + PPSOpticalFunctions_offline_v7.

When the "run2_miniAOD_devel" flag is not declared (and the current conditions are used), no change is produced wrt. the UL re-reco state.

PR validation:

The plot below shows a comparison of tests with old conditions and without the devel flag:

• blue: before this PR
• red dashed: with this PR

There's no difference.

The plot below shows a comparison of tests with updated conditions and with the devel flag:

• blue: this PR
• red: PR PPS: updates for re-miniAOD #29770

There is no difference.

if this PR is a backport please specify the original PR and why you need to backport that PR:

This is a backport of #29770. It is needed for the UL re-miniAOD campaign.

[cmsbuild]

A new Pull Request was created by @jan-kaspar for CMSSW_10_6_X.

It involves the following packages:

PhysicsTools/PatAlgos
RecoCTPPS/Configuration
RecoCTPPS/PixelLocal
RecoCTPPS/ProtonReconstruction
RecoCTPPS/TotemRPLocal
Validation/CTPPS

@perrotta, @andrius-k, @kmaeshima, @schneiml, @cmsbuild, @jfernan2, @fioriNTU, @slava77, @santocch can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @forthommel, @hatakeyamak, @emilbols, @peruzzim, @seemasharmafnal, @mmarionncern, @ahinzmann, @smoortga, @jdolen, @ferencek, @jdamgov, @nhanvtran, @gkasieczka, @schoef, @andrzejnovak, @clelange, @riga, @JyothsnaKomaragiri, @mverzett, @gpetruc, @mariadalfonso this is something you requested to watch as well.
@silviodonato, @dpiparo you are the release manager for this.

cms-bot commands are listed here

• Backported from PPS: updates for re-miniAOD #29770

[slava77]

@cmsbuild please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/6182/console Started: 2020/05/08 15:03

[cmsbuild]

-1

Tested at: e5a7b68

CMSSW: CMSSW_10_6_X_2020-05-08-1100
SCRAM_ARCH: slc7_amd64_gcc700
You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-be920d/6182/summary.html

I found follow errors while testing this PR

Failed tests: AddOn

• AddOn:

I found errors in the following addon tests:

cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_2016 : FAILED - time: date Fri May 8 16:20:10 2020-date Fri May 8 16:16:08 2020 s - exit: 17920

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-be920d/6182/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 33
• DQMHistoTests: Total histograms compared: 3212324
• DQMHistoTests: Total failures: 3
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3211987
• DQMHistoTests: Total skipped: 334
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 32 files compared)
• Checked 137 log files, 14 edm output root files, 33 DQM output files

[slava77]

backport of #29770

[cmsbuild]

+1
Tested at: 8b2d732
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-be920d/6502/summary.html
CMSSW: CMSSW_10_6_X_2020-05-21-2300
SCRAM_ARCH: slc7_amd64_gcc700

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-be920d/6502/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 33
• DQMHistoTests: Total histograms compared: 3212324
• DQMHistoTests: Total failures: 4
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3211986
• DQMHistoTests: Total skipped: 334
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 32 files compared)
• Checked 137 log files, 14 edm output root files, 33 DQM output files

[jfernan2]

+1

[perrotta]

+1

• PPS time and proton reconstrunction backported from the master, but those updates are only accessed with the run2_miniAOD_devel modifier.
• Validation plots verify that with that modifier the outputs are the same as in the master
• Jenkins tests show no differences for standard workflows, as it should

[santocch]

+1

[silviodonato]

@jan-kaspar : there are some changes that were not included in #29770. Could you make a PR to include these changes in master? I'm referring, for instance, to the conversion from Sequence of Task of PPS sequence in Configuration/StandardSequences/python/Reconstruction_cff.py.

[jan-kaspar]

@silviodonato I am confused. The Sequence-to-Task changes in this PR were done following the code in master. Taking the example you point out, the PPS list in the copyAndExclude call should be identical in this PR:
https://github.com/cms-sw/cmssw/pull/29771/files#diff-bc1d3eddbea8b75d590d62e72fee751eL93
and in master
https://github.com/cms-sw/cmssw/blob/master/Configuration/StandardSequences/python/Reconstruction_cff.py#L96
Do I miss anything?

[silviodonato]

+operations
@jan-kaspar I was looking at the file changed by the PRs. Ok, so you are backporting also part of #28678

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_10_6_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_11_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo (and backports should be raised in the release meeting by the corresponding L2)

[silviodonato]

+1