features for bParking UL rereco and miniAOD in 10_6_X (10_6_26?)

[slava77]

This issue is to collect PRs/features for integration towards b-parking dataset rereco and miniAOD in 10_6_X

Deadline for integration is June 7, 2021 (PRs should be merged by this date)

RECO/AOD part:

• Add CSC/DT rechits to AOD[backport] #34066 Add CSC/DT rechits to AOD
• MC-specific:
  • [Backport] Track-gen matching passing through TrackingParticle #33996 Track-gen matching passing through TrackingParticle (merged June 13)
  • THIS IS A PATCH TO CMSSW_10_2_16_UL keep future products from UL DIGI step #34083 keep future products from UL DIGI step (merged June 10/11); the release https://github.com/cms-sw/cmssw/releases/CMSSW_10_2_16_UL2 is available on June 11.

miniAOD:

• LowPtElectrons: final energy regression and ID (back port of 32391) #33589 lowPt electrons ID and regression (merged June 6)
• Backport of #33776: Adding the correction structure to all electrons #33878: Adding the correction structure to all electrons (merged June 2)
• [Backport] Making miniAOD low pt tracks threshold tunable for 10_6_X (#33777) #33877 Making miniAOD low pt tracks threshold tunable (merged June 1)
• [Backport] Adding offlineVerticesWithBS to miniAOD to 10_6_X #33922 Adding offlineVerticesWithBS to miniAOD (merged June 4)
• [Backport] Adding Xi-tracks to lostTracks to 10_6_X #33962 Adding Xi-compatible tracks to lostTracks collection (merged June 4)
• Backport (of #33966) to 106X: Adding necessary electron structures for low pT part of slimmedElectron collection #33977 Adding necessary electron structures for low pT part of slimmedElectron collection (merged June 7)
• LowPtElectrons: NanoAOD integration (back port of #33817) #33992 "LowPtElectrons: NanoAOD integration", eventually does not modify miniAOD [could fill conversion info, dxy and dz to slimmedLowPtElectrons] (merged June 8)
• LowPtElectrons: embed extra info in pat::Electron for BParking UL #34023 LowPtElectrons: embed extra info in pat::Electron for BParking UL (merged June 9)
• Backport of Muon L1 prefire weights #33759 "Add Muon L1 to prefire weight" eventually does not modify miniAOD (merged June 11)
• [backport 106X] Usage of displaced collections in the B-parking data  #34008 displaced collections in the B-parking data (merged June 9)
• (PR is needed for ) todo: create a miniAOD wf for bParking UL data in 12X and 106X; review use of run2_miniAOD_UL in connection with bParking #33665 to define test workflows
  • manual setup for MC was tested as in Track-gen matching passing through TrackingParticle #33774 (comment) (now the HLT will use CMSSW_10_2_16_UL2)
  • this should not block the release

@cms-sw/xpog-l2
please check the list and clarify if more is needed/planned

[slava77]

assign reconstruction,xpog

[cmsbuild]

New categories assigned: xpog,reconstruction

@slava77,@fgolf,@mariadalfonso,@gouskos,@perrotta,@jpata you have been requested to review this Pull request/Issue and eventually sign? Thanks

[cmsbuild]

A new Issue was created by @slava77 Slava Krutelyov.

@Dr15Jones, @dpiparo, @silviodonato, @smuzaffar, @makortel, @qliphy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[slava77]

@rappoccio (tagging for PPD, I don't have Andreas' git at hand)

[slava77]

from https://hypernews.cern.ch/HyperNews/CMS/get/recoDevelopment/1716/1.html
code introduced in #33774 is discussed (potentially) as an addition in the MC dataset processing.
@mtosi @vmariani

[slava77]

updates in the issue description/list from earlier today:

• New:
  • [Backport] Adding offlineVerticesWithBS to miniAOD to 10_6_X #33922 "Adding offlineVerticesWithBS to miniAOD" was added to the list (master version Adding offlineVerticesWithBS to miniAOD #33778 was merged May 28)
• Status update:
  • [Backport] Making miniAOD low pt tracks threshold tunable for 10_6_X (#33777) #33877 Making miniAOD low pt tracks threshold tunable (merged June 1, 2021)
  • notes for other PRs were extended to mention the master PR (where it was not already mentioned)

[mariadalfonso]

There is the idea to keep the muonHits in AOD or a reduced collection in miniAOD. The idea is to enable some EXO analysis on the B-parking dataset. It was discussed at PC-coord may26 and PPD-coord june2
Marco Pieri will follow up with proponents

[slava77]

There is the idea to keep the muonHits in AOD or a reduced collection in miniAOD. The idea is to enable some EXO analysis on the B-parking dataset. It was discussed at PC-coord may26 and PPD-coord june2
Marco Pieri will follow up with proponents

isn't this enabled by default in AOD?
or is it about something other than what was added in #31350 in 10_6_17?

[slava77]

There is the idea to keep the muonHits in AOD or a reduced collection in miniAOD. The idea is to enable some EXO analysis on the B-parking dataset. It was discussed at PC-coord may26 and PPD-coord june2
Marco Pieri will follow up with proponents

isn't this enabled by default in AOD?
or is it about something other than what was added in #31350 in 10_6_17?

based on the discussion in the PPD today, this is different, to include all muon rechits

• Muon system acts like a sampling calorimeter
• LLP decays produce a shower of particles
• RecHits used to form clusters and identify a signal

IIUC, there is no plan to keep AOD on disk for this processing. Are the analyses targeting this prepared to use miniAOD for parts other than muon hits?

[rappoccio]

Correct, we don't have any plans to keep this on disk, but if we do a future miniaod (v3) later, it would be available.

[AdrianoDee]

From the low pt tracks point of view, we should be fine. Thanks a lot for the (super fast) effort done for reviewing the code!

[slava77]

current status of the list of features

I added two PRs which are described as targeting nanoAOD (v9), but they also include modifications in the miniAOD products. Both a backport of #33817 and #33759 will need to appropriately disable the updates in miniAOD except for bParking | run2_miniAOD_devel to follow the no-change policy for the existing old miniAOD production(s).

[slava77]

I updated the list with the following:

RECO/AOD part:

• Add CSC/DT rechits to AOD[backport] #34066 Add CSC/DT rechits to AOD
• MC-specific:
  • [Backport] Track-gen matching passing through TrackingParticle #33996 Track-gen matching passing through TrackingParticle
  • NEED A PATCH TO CMSSW_10_2_16_UL with keep statements from [Backport] Track-gen matching passing through TrackingParticle #33996 in SimTracker/Configuration/python/SimTracker_EventContent_cff.py

[slava77]

* NEED A PATCH TO CMSSW_10_2_16_UL with keep statements from #33996  in SimTracker/Configuration/python/SimTracker_EventContent_cff.py

the above came from the discussion in #33996 (comment)

@silviodonato @qliphy
what is your preference to make this update?
There is no branch for patches for CMSSW_10_2_16 ( = CMSSW_10_2_16_UL). Should the update be made to CMSSW_10_2_X? (The difference is more than a few files https://github.com/cms-sw/cmssw/compare/CMSSW_10_2_16_UL..CMSSW_10_2_X)

[qliphy]

probably we can make CMSSW_10_2_26_patch2 ? @silviodonato may comment more.

[silviodonato]

@slava77 once we have a backport of #33996 to CMSSW_10_2_X, we can build a special CMSSW_10_2_16_UL2 that is CMSSW_10_2_16_UL + backport of #33996.
The file differences should be harmless, but I think it is more safe to avoid the big jump CMSSW_10_2_16_UL -> CMSSW_10_2_27_UL only for b-parking UL.

@smuzaffar what is the best way to build a CMSSW_10_2_16_UL2 from the technical point of view?
Do we need a new branch CMSSW_10_2_16_UL_patchX or can we directly cherry-pick the commits that we want to add?

[smuzaffar]

@silviodonato , if CMSSW_10_2_16_UL2 should be CMSSW_10_2_16_UL + #33996 then we need a cmssw tag or branch with CMSSW_10_2_16_UL + #33996 which you can use as CMSSW_COMMIT in the build issue.

[smuzaffar]

and you also should use

CMSDIST_COMMIT=REL/CMSSW_10_2_16_UL/slc7_amd64_gcc700
PRODUCTION_ARCHITECTURE:slc7_amd64_gcc700

in order to not get all the external merged in IB/CMSSW_10_2_X/gcc700 branch

[slava77]

@silviodonato , if CMSSW_10_2_16_UL2 should be CMSSW_10_2_16_UL + #33996 then we need a cmssw tag or branch with CMSSW_10_2_16_UL + #33996 which you can use as CMSSW_COMMIT in the build issue.

I'm not familiar with this procedure, in the past we used PRs to make updates.

So, somebody will

• checkout CMSSW_10_2_16_UL in "from-CMSSW_10_2_16_UL" branch
• add a commit on it
• push it to the personal remote
• ... what's next?

[silviodonato]

I would say - first of all - to merge the 102X backport to the standard CMSSW_10_2_X.

Then, I've just created the branch CMSSW_10_2_16_UL_patchX, so you can make a PR towards CMSSW_10_2_16_UL_patchX and then we'll build the release

[qliphy]

@smuzaffar @silviodonato
Please check whether this is ok:
#34089

[slava77]

@smuzaffar @silviodonato
Please check whether this is ok:
#34089

the tag looks correct:
CMSSW_10_2_16_UL...CMSSW_10_2_16_UL2

[qliphy]

superseded by #34090 thanks to @smuzaffar

[qliphy]

CMSSW_10_2_16_UL2 is now ready
https://github.com/cms-sw/cmssw/releases/CMSSW_10_2_16_UL2

[slava77]

Current status is now down to one PR:

• [Backport] Track-gen matching passing through TrackingParticle #33996 "Track-gen matching passing through TrackingParticle" still needs the master version Track-gen matching passing through TrackingParticle #33774 to get through, which is missing pdmv and upgrade signatures

The other incomplete item is " to define test workflows" could probably be done now outside of the cmssw source.

[slava77]

it looks like all PRs are now merged

[slava77]

+reconstruction

the production processing is well in progress .
I guess this issue can be closed and if some bug fixes are needed, they can be tracked in a separate issue

[slava77]

@cms-sw/pdmv-l2

this is connected with the MC setup discussion earlier today.
When I closed the issue I thought that the MC production was done/in progress as well.