Add feature to save generator protons from pile-up for standard mixing and premixing #18347

antoniovilela · 2017-04-12T22:56:08Z

Add feature to save generator protons from pile-up, related to simulation/analysis with proton detectors.

In PhysicsTools/HepMCCandAlgos, add GenPUProtonProducer and customisation functions to run with cmsDriver;
In SimGeneral/DataMixingModule, update DataMixingModule with optional list of GenParticle collections from premixing sample, to save in the main event. Also add customisation function to run with cmsDriver.

cmsbuild · 2017-04-12T22:56:27Z

A new Pull Request was created by @antoniovilela for master.

It involves the following packages:

PhysicsTools/HepMCCandAlgos
SimGeneral/DataMixingModule

@cmsbuild, @civanch, @monttj, @mdhildreth, @davidlange6 can you please review it and eventually sign? Thanks.
@Muzaffar, @davidlange6, @smuzaffar you are the release manager for this.

cms-bot commands are listed here

mdhildreth · 2017-04-15T03:19:07Z

please test

cmsbuild · 2017-04-15T03:19:24Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/19189/console Started: 2017/04/15 05:21

mdhildreth · 2017-04-15T03:27:13Z

PhysicsTools/HepMCCandAlgos/plugins/GenPUProtonProducer.cc

+   size_t idx_particle = 0;
+   idx_mix = 0;
+   // Fill collection
+   for( mixHepMC_itr = cfhepmcprod->begin() ; mixHepMC_itr != cfhepmcprod->end() ; ++mixHepMC_itr, ++idx_mix ){


Why do you need two loops over the genParticles? It looks like you could just push back the protons onto the particle vector without needing to know how big it will eventually be.

@mdhildreth Indeed the first loop in the HepMC products could be suppressed. It is reminiscent of the initial structure used as a basis for this module. I added below [1] some time report lines when running the premixing sample with these modules with few events in lxplus. The cmsDriver command is also shown below [2]. The 'genPUProtons*' modules added each around 1.5 ms. Of course the module that dominates the processing time is the mixing module (~30 s in this case).
Please let me know if I should update the module and prepare a new request or a fix could be added later.
Thanks.
[1]
TimeReport ---------- Event Summary ---[sec]----
TimeReport event loop CPU/event = 30.289363
TimeReport event loop Real/event = 52.353183
TimeReport sum Streams Real/event = 49.928158
TimeReport efficiency CPU/Real/thread = 0.578558

TimeReport ---------- Modules in Path: digitisation_step ---[Real sec]----
TimeReport per event per visit Name
TimeReport 0.000000 0.000000 generator
TimeReport 19.516880 19.516880 cpuSpender
TimeReport 0.000031 0.000031 generatorSmeared
TimeReport 0.000584 0.000584 genParticles
TimeReport 0.000225 0.000225 genParticlesForJets
TimeReport 0.000009 0.000009 genParticlesForJetsNoNu
TimeReport 0.000539 0.000539 ak4GenJets
TimeReport 0.000018 0.000018 ak8GenJets
TimeReport 0.000008 0.000008 ak4GenJetsNoNu
TimeReport 0.000006 0.000006 ak8GenJetsNoNu
TimeReport 0.000007 0.000007 genCandidatesForMET
TimeReport 0.000021 0.000021 genParticlesForMETAllVisible
TimeReport 0.000141 0.000141 genMetCalo
TimeReport 0.000006 0.000006 genMetTrue
TimeReport 0.000000 0.000000 randomEngineStateProducer
TimeReport 29.127603 29.127603 mix
TimeReport 0.000000 0.000000 generatorSmeared
TimeReport 0.055196 0.055196 simEcalTriggerPrimitiveDigis
TimeReport 0.020653 0.020653 simEcalDigis
TimeReport 0.000828 0.000828 simHcalDigis
TimeReport 0.142470 0.142470 simMuonCSCDigis
TimeReport 0.000838 0.000838 simMuonDTDigis
TimeReport 0.013858 0.013858 simMuonRPCDigis
TimeReport 0.000217 0.000217 addPileupInfo
TimeReport 0.001565 0.001565 genPUProtonsBx0
TimeReport 0.001450 0.001450 genPUProtonsBxm1
TimeReport 0.001455 0.001455 genPUProtonsBxp1

[2]
cmsDriver.py Configuration/Generator/python/SingleNuE10_cfi.py --fileout file:file_out_premixing.root --pileup_input dbs:/RelValProdMinBias_13/CMSSW_9_0_0-90X_mcRun2_asymptotic_v5-v1/GEN-SIM --mc --eventcontent PREMIX --pileup 2016_25ns_Moriond17MC_PoissonOOTPU --datatier GEN-SIM-DIGI-RAW --conditions 90X_mcRun2_asymptotic_v5 --step GEN,SIM,DIGIPREMIX,L1,DIGI2RAW --era Run2_2016 -n 30 --customise PhysicsTools/HepMCCandAlgos/addGenPUProtons.customiseGenPUProtons,PhysicsTools/HepMCCandAlgos/addGenPUProtonsEventContent.customiseGenPUProtonsEventContent

mdhildreth · 2017-04-15T03:28:03Z

@antoniovilela please see comments in code

cmsbuild · 2017-04-15T04:27:13Z

+1
Tested at: 5f96204
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18347/19189/summary.html

cmsbuild · 2017-04-15T04:27:16Z

Comparison job queued.

cmsbuild · 2017-04-15T06:21:54Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18347/19189/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 1630 differences found in the comparisons
DQMHistoTests: Total files compared: 23
DQMHistoTests: Total histograms compared: 1819408
DQMHistoTests: Total failures: 38977
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 1780258
DQMHistoTests: Total skipped: 173
DQMHistoTests: Total Missing objects: 0
Checked 94 log files, 14 edm output root files, 23 DQM output files

mdhildreth · 2017-04-25T14:47:06Z

+1

davidlange6 · 2017-04-26T17:37:35Z

Hi @antoniovilela @mdhildreth
what is the data volume impact of this ?(eg, if enabled on some standard events)

antoniovilela · 2017-04-26T18:27:19Z

Hi @davidlange6, @mdhildreth,

Here is a size estimate per event (from a test with 50 events and a default Minimum Bias sample):

Branch Name | Average Uncompressed Size (Bytes/Event) | Average Compressed Size (Bytes/Event)
recoGenParticles_mixData_genPUProtonsBxp1_HLT. 970.82 194.04
recoGenParticles_mixData_genPUProtonsBxm1_HLT. 951.92 191.88
recoGenParticles_mixData_genPUProtonsBx0_HLT. 872.22 179.48

This was run with the 2016 PU configuration. In this case it gives less than 600 bytes (compressed) per event.
Thanks.

davidlange6 · 2017-05-01T15:57:22Z

Thanks. To avoid introducing additional complexity, I would propose we just keep this information by default. (As it's <0.1%)

perrozzi · 2017-08-15T09:27:04Z

Hi, this PR is stuck since long time, probably without a real reason.
Can you please finalize the review?
This PR is very important to integrate the forward proton information for the TOTEM simulation.
Thanks
Luca

antoniovilela · 2017-08-15T10:26:07Z

As agreed with CT-PPS simulation conveners, a new PR was created: #20172

antoniovilela added 2 commits April 11, 2017 17:11

GenPUProtonProducer and updates to DataMixingModule

a7799e2

GenPUProtonProducer and updates to DataMixingModule

5f96204

cmsbuild added this to the Next CMSSW_9_1_X milestone Apr 12, 2017

cmsbuild added analysis-pending comparison-pending orp-pending pending-signatures simulation-pending tests-pending labels Apr 12, 2017

cmsbuild added tests-started and removed tests-pending labels Apr 15, 2017

mdhildreth reviewed Apr 15, 2017

View reviewed changes

cmsbuild added tests-approved and removed tests-started labels Apr 15, 2017

cmsbuild added comparison-available and removed comparison-pending labels Apr 15, 2017

cmsbuild added simulation-approved and removed simulation-pending labels Apr 25, 2017

smuzaffar added this to the CMSSW_9_2_X milestone May 4, 2017

smuzaffar removed this from the CMSSW_9_1_X milestone May 4, 2017

smuzaffar modified the milestones: CMSSW_9_3_X, CMSSW_9_2_X Jun 26, 2017

antoniovilela mentioned this pull request Aug 15, 2017

Saving generator protons from pile-up for standard mixing and premixing #20172

Merged

smuzaffar added the code-checks-pending label Aug 22, 2017

smuzaffar modified the milestones: CMSSW_9_4_X, CMSSW_9_3_X Sep 6, 2017

davidlange6 closed this Sep 18, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add feature to save generator protons from pile-up for standard mixing and premixing #18347

Add feature to save generator protons from pile-up for standard mixing and premixing #18347

antoniovilela commented Apr 12, 2017

cmsbuild commented Apr 12, 2017 •

edited by smuzaffar

Loading

mdhildreth commented Apr 15, 2017

cmsbuild commented Apr 15, 2017 •

edited

Loading

mdhildreth Apr 15, 2017

antoniovilela Apr 17, 2017

mdhildreth commented Apr 15, 2017

cmsbuild commented Apr 15, 2017

cmsbuild commented Apr 15, 2017

cmsbuild commented Apr 15, 2017

mdhildreth commented Apr 25, 2017

davidlange6 commented Apr 26, 2017

antoniovilela commented Apr 26, 2017

davidlange6 commented May 1, 2017

perrozzi commented Aug 15, 2017

antoniovilela commented Aug 15, 2017

Add feature to save generator protons from pile-up for standard mixing and premixing #18347

Add feature to save generator protons from pile-up for standard mixing and premixing #18347

Conversation

antoniovilela commented Apr 12, 2017

cmsbuild commented Apr 12, 2017 • edited by smuzaffar Loading

mdhildreth commented Apr 15, 2017

cmsbuild commented Apr 15, 2017 • edited Loading

mdhildreth Apr 15, 2017

Choose a reason for hiding this comment

antoniovilela Apr 17, 2017

Choose a reason for hiding this comment

mdhildreth commented Apr 15, 2017

cmsbuild commented Apr 15, 2017

cmsbuild commented Apr 15, 2017

cmsbuild commented Apr 15, 2017

mdhildreth commented Apr 25, 2017

davidlange6 commented Apr 26, 2017

antoniovilela commented Apr 26, 2017

davidlange6 commented May 1, 2017

perrozzi commented Aug 15, 2017

antoniovilela commented Aug 15, 2017

cmsbuild commented Apr 12, 2017 •

edited by smuzaffar

Loading

cmsbuild commented Apr 15, 2017 •

edited

Loading