Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Saving generator protons from pile-up for standard mixing and premixing #20172

Merged

Conversation

antoniovilela
Copy link
Contributor

As agreed with CT-PPS simulation conveners, this is a follow-up of the previous request: #18347

With respect to the above PR (not merged) it has updates to the new GenPUProtonProducer and to DataMixingPileupCopy in the DataMixingModule and includes the new 'genPUProtons' sequence for mixing and premixing workflows.

@fabferro , @forthommel , @perrozzi , @mdhildreth

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @antoniovilela for master.

It involves the following packages:

Configuration/StandardSequences
PhysicsTools/HepMCCandAlgos
PhysicsTools/PatAlgos
SimGeneral/Configuration
SimGeneral/DataMixingModule
SimGeneral/MixingModule

@perrotta, @civanch, @mdhildreth, @monttj, @cmsbuild, @franzoni, @slava77, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @TaiSakuma, @gouskos, @felicepantaleo, @rappoccio, @Martin-Grunewald, @ahinzmann, @seemasharmafnal, @mmarionncern, @imarches, @makortel, @acaudron, @jdolen, @ferencek, @GiacomoSguazzoni, @rovere, @VinInn, @nhanvtran, @gkasieczka, @schoef, @ebrondol, @dgulhan, @JyothsnaKomaragiri, @mverzett, @gpetruc, @mariadalfonso, @pvmulder this is something you requested to watch as well.
@davidlange6, @slava77 you are the release manager for this.

cms-bot commands are listed here

@perrotta
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 15, 2017

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/22303/console Started: 2017/08/15 14:40

@slava77
Copy link
Contributor

slava77 commented Aug 15, 2017

@antoniovilela
what is the event size increase from this (on disk) vs PU?
most interestingly, for miniAOD

@@ -90,6 +90,8 @@
'keep GenLumiInfoHeader_generator_*_*',
'keep GenLumiInfoProduct_*_*_*',
'keep GenEventInfoProduct_generator_*_*',
'keep recoGenParticles_genPUProtons_*_*',
'keep recoGenParticles_*_genPUProtons_*',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why there are two collections?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gpetruc, @slava77
There will be only one collection saved, but there are two possible workflows:
i) Standard mixing, where the collection is saved from the 'genPUProtons' module in the DIGI sequence:
Type Module Label Process
vectorreco::GenParticle "genPUProtons" "" "HLT"

ii) Pre-mixing, where the collection is saved in the premixed sample and later copied by the DataMixingModule ('mixData'), with label 'genPUProtons':
Type Module Label Process
vectorreco::GenParticle "mixData" "genPUProtons" "DIGI2RAW"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these can/should be changed to be an EDAlias so that the only saved branches are recoGenParticles_genPUProtons

See examples in SimGeneral/MixingModule/python/aliases_PreMix_cfi.py
it will be something like this (perhaps for both mix and mixData)

genPUProtons = cms.EDAlias(
    mixData = cms.VPSet(cms.PSet(
        type = cms.string('recoGenParticles')
    ))
)

I tried this in 250202.17 (premix) and it worked:

  • default: recoGenParticles_mixData_genPUProtons_HLT.
  • with the modification (and keep of only recoGenParticles_genPUProtons__*) : recoGenParticles_genPUProtons_genPUProtons_HLT

@cmsbuild
Copy link
Contributor

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/PR-20172/126

Code check has found code style and quality issues which could be resolved by applying a patch in https://cmssdt.cern.ch/SDT/code-checks/PR-20172/126/git-diff.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/PR-20172/126/git-diff.patch | patch -p1

In future, you can run scram build code-checks to apply code checks

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@antoniovilela
Copy link
Contributor Author

@slava77
Below is an example with the "2016_25ns_Moriond17MC_PoissonOOTPU" pile-up scenario: O(1kB) uncompressed and a few hundred bytes compressed.
It scales with the number of mixed pile-up events. We are saving only particles from the in-time bunch crossing.

Branch Name | Average Uncompressed Size (Bytes/Event) | Average Compressed Size (Bytes/Event)
recoGenParticles_genPUProtons__HLT. 703.33 321.8

@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/PR-20172/901

@cmsbuild
Copy link
Contributor

Pull request #20172 was updated. @perrotta, @civanch, @mdhildreth, @monttj, @cmsbuild, @franzoni, @slava77, @davidlange6 can you please check and sign again.

@perrotta
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 22, 2017

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/23160/console Started: 2017/09/22 12:12

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20172/23160/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 26
  • DQMHistoTests: Total histograms compared: 2661038
  • DQMHistoTests: Total failures: 222
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2660627
  • DQMHistoTests: Total skipped: 189
  • DQMHistoTests: Total Missing objects: 0
  • Checked 107 log files, 15 edm output root files, 26 DQM output files

@slava77
Copy link
Contributor

slava77 commented Sep 22, 2017

+1

for #20172 39621ab

@davidlange6
Copy link
Contributor

merge

@cmsbuild cmsbuild merged commit 630acaf into cms-sw:master Sep 24, 2017
@forthommel forthommel deleted the 93X-GenPUProtonProducer-Premixing-StandardSequences branch November 11, 2021 09:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.