Fix memory leak in GenParticles2HepMCConverter #46203

hqucms · 2024-10-02T19:15:44Z

PR description:

When checking the NANO matrix tests I noticed some MC workflows show very high memory usage (~6GB). This was tracked down to a memory leak due to undeleted raw pointer in GenParticles2HepMCConverter. This PR fixes that by switching to unique_ptr.

PR validation:

Tested with NANO matrix test and see much lower and stable memory usage after the fix.

cmsbuild · 2024-10-02T19:16:06Z

cms-bot internal usage

hqucms · 2024-10-02T19:16:39Z

enable nano

hqucms · 2024-10-02T19:16:43Z

please test

cmsbuild · 2024-10-02T19:18:21Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46203/42013

cmsbuild · 2024-10-02T19:18:45Z

A new Pull Request was created by @hqucms for master.

It involves the following packages:

GeneratorInterface/RivetInterface (generators)

@bbilin, @lviliani, @menglu21, @mkirsano can you please review it and eventually sign? Thanks.
@alberto-sanchez, @mkirsano this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2024-10-02T22:53:10Z

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2e6197/41901/summary.html
COMMIT: 1f38bfa
CMSSW: CMSSW_14_2_X_2024-10-02-1100/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46203/41901/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 2 lines to the logs
Reco comparison results: 29 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3331342
DQMHistoTests: Total failures: 2137
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3329185
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
Checked 193 log files, 163 edm output root files, 44 DQM output files
TriggerResults: no differences found

NANO Comparison Summary

Summary:

You potentially removed 1131 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 21
DQMHistoTests: Total histograms compared: 55056
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 55056
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 20 files compared)
Checked 102 log files, 58 edm output root files, 21 DQM output files
TriggerResults: no differences found

Nano size comparison Summary:

Sample	kb/ev	ref kb/ev	diff kb/ev	ev/s/thd	ref ev/s/thd	diff rate	mem/thd	ref mem/thd
2500.001	2.941	2.941	0.000 ( +0.0% )	3.04	3.04	-0.1%	2.315	6.098
2500.002	3.050	3.050	0.000 ( +0.0% )	2.70	2.74	-1.6%	2.342	6.442
2500.003	2.995	2.995	0.000 ( +0.0% )	2.84	2.84	-0.0%	2.356	6.400
2500.011	1.534	1.534	0.000 ( +0.0% )	4.51	4.67	-3.4%	2.424	2.167
2500.012	2.032	2.032	0.000 ( +0.0% )	2.84	2.86	-0.5%	2.229	2.199
2500.013	1.873	1.873	0.000 ( +0.0% )	4.02	4.04	-0.4%	2.527	2.221
2500.021	0.022	0.022	0.000 ( +0.0% )	0.93	0.93	+0.2%	2.359	2.362
2500.022	0.022	0.022	0.000 ( +0.0% )	0.91	0.90	+1.7%	2.353	2.353
2500.023	0.022	0.022	0.000 ( +0.0% )	0.88	0.87	+1.7%	2.218	2.225
2500.024	0.022	0.022	0.000 ( +0.0% )	0.67	0.66	+2.5%	2.445	2.454
2500.031	0.035	0.035	0.000 ( +0.0% )	0.84	0.82	+3.3%	2.415	2.424
2500.032	0.036	0.036	0.000 ( +0.0% )	0.85	0.84	+0.9%	2.381	2.384
2500.033	0.037	0.037	0.000 ( +0.0% )	0.75	0.75	+0.1%	2.471	2.468
2500.034	0.036	0.036	0.000 ( +0.0% )	0.77	0.76	+1.4%	2.445	2.444
2500.101	2.732	2.732	0.000 ( +0.0% )	8.44	8.50	-0.8%	2.514	6.359
2500.111	1.389	1.389	0.000 ( +0.0% )	18.68	18.60	+0.4%	2.236	2.244
2500.112	1.806	1.806	0.000 ( +0.0% )	14.56	14.97	-2.8%	2.314	2.315
2500.131	0.747	0.747	0.000 ( +0.0% )	17.22	17.48	-1.5%	1.322	1.522
2500.201	2.557	2.557	0.000 ( +0.0% )	7.25	7.39	-1.9%	2.072	6.240
2500.211	1.700	1.700	0.000 ( +0.0% )	17.00	17.15	-0.9%	2.282	2.284
2500.212	2.096	2.096	0.000 ( +0.0% )	13.78	13.50	+2.0%	2.365	2.370
2500.221	2.004	2.004	0.000 ( +0.0% )	7.39	7.56	-2.2%	2.001	2.459
2500.222	3.281	3.281	0.000 ( +0.0% )	7.21	7.39	-2.5%	2.085	2.543
2500.223	8.968	8.968	0.000 ( +0.0% )	2.53	2.57	-1.6%	2.113	2.576
2500.224	5.787	5.787	0.000 ( +0.0% )	0.59	0.59	-0.6%	2.106	2.603
2500.225	5.805	5.805	0.000 ( +0.0% )	0.57	0.57	-0.6%	2.125	2.624
2500.226	3.044	3.044	0.000 ( +0.0% )	7.15	7.19	-0.6%	2.083	2.540
2500.227	1.437	1.437	0.000 ( +0.0% )	11.24	11.29	-0.4%	1.445	1.434
2500.231	1.404	1.404	0.000 ( +0.0% )	13.37	13.61	-1.8%	2.185	2.189
2500.232	2.301	2.301	0.000 ( +0.0% )	13.35	13.51	-1.2%	2.281	2.281
2500.233	4.748	4.748	0.000 ( +0.0% )	4.07	4.14	-1.9%	2.305	2.300
2500.234	3.501	3.501	0.000 ( +0.0% )	0.75	0.76	-1.3%	2.076	2.313
2500.235	3.513	3.513	0.000 ( +0.0% )	0.73	0.74	-1.5%	2.097	2.330
2500.236	2.145	2.145	0.000 ( +0.0% )	13.32	13.42	-0.7%	2.276	2.266
2500.237	1.016	1.016	0.000 ( +0.0% )	16.31	16.48	-1.0%	1.467	1.472
2500.241	9.404	9.404	0.000 ( +0.0% )	3.38	3.30	+2.5%	1.948	1.954
2500.242	10.331	10.331	0.000 ( +0.0% )	0.83	0.84	-0.4%	1.741	1.738
2500.243	2.712	2.712	0.000 ( +0.0% )	7.59	8.03	-5.5%	1.083	1.082
2500.244	485.976	485.976	0.000 ( +0.0% )	0.53	0.53	-0.2%	1.689	1.679
2500.245	823.202	823.202	0.000 ( +0.0% )	0.70	0.71	-0.6%	1.669	1.688
2500.901	1.777	1.777	0.000 ( +0.0% )	19.94	20.42	-2.4%	1.425	1.852
2500.902	1.626	1.626	0.000 ( +0.0% )	20.32	19.75	+2.9%	1.327	1.775
2500.911	13.995	13.995	0.000 ( +0.0% )	3.63	2.99	+21.4%	1.094	1.100
2500.912	0.171	0.150	0.021 ( +14.1% )	1.08	1.30	-17.5%	0.981	0.984
2500.913	0.110	0.110	0.000 ( +0.0% )	1.02	1.03	-1.0%	0.979	0.984

hqucms · 2024-10-03T00:18:17Z

Maxmem profiling shows expected reduction:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisonsNANO/CMSSW_14_2_X_2024-10-02-1100+2e6197/64878/maxmem-comparison/

mseidel42 · 2024-10-03T07:36:15Z

Whoops, sorry for introducing this 🤦‍♂️ We should also have a backport to 14_1 so that it can be used for Rivet 4 development

hqucms · 2024-10-03T11:45:10Z

Whoops, sorry for introducing this 🤦‍♂️ We should also have a backport to 14_1 so that it can be used for Rivet 4 development

np :) I will do a backport after this is merged for master.

hqucms · 2024-10-04T11:35:56Z

Whoops, sorry for introducing this 🤦‍♂️ We should also have a backport to 14_1 so that it can be used for Rivet 4 development

@mseidel42 I opened the backport to 14_1_X in #46254.

cmsbuild · 2024-10-14T19:11:36Z

Pull request #46203 was updated. @bbilin, @cmsbuild, @lviliani, @menglu21, @mkirsano can you please check and sign again.

hqucms · 2024-10-14T19:13:21Z

please test

hqucms · 2024-10-14T19:17:36Z

assign xpog

cmsbuild · 2024-10-14T19:17:56Z

New categories assigned: xpog

@ftorrresd,@hqucms you have been requested to review this Pull request/Issue and eventually sign? Thanks

cmsbuild · 2024-10-14T22:38:10Z

+1

Size: This PR adds an extra 24KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2e6197/42176/summary.html
COMMIT: 19d6938
CMSSW: CMSSW_14_2_X_2024-10-14-1100/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46203/42176/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 2 lines to the logs
Reco comparison results: 55 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3331066
DQMHistoTests: Total failures: 3863
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3327183
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
Checked 193 log files, 163 edm output root files, 44 DQM output files
TriggerResults: no differences found

NANO Comparison Summary

Summary:

You potentially added 215 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 21
DQMHistoTests: Total histograms compared: 55028
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 55028
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 20 files compared)
Checked 102 log files, 58 edm output root files, 21 DQM output files
TriggerResults: no differences found

Nano size comparison Summary:

Sample	kb/ev	ref kb/ev	diff kb/ev	ev/s/thd	ref ev/s/thd	diff rate	mem/thd	ref mem/thd
2500.001	2.947	2.947	0.000 ( +0.0% )	3.02	3.10	-2.6%	2.320	6.055
2500.002	3.056	3.056	0.000 ( +0.0% )	2.71	2.76	-1.8%	2.762	6.411
2500.003	2.998	2.998	0.000 ( +0.0% )	2.84	2.89	-1.8%	2.746	6.487
2500.011	1.533	1.533	0.000 ( +0.0% )	4.61	4.80	-4.0%	2.422	2.421
2500.012	2.030	2.030	0.000 ( +0.0% )	2.85	2.90	-1.9%	2.613	2.605
2500.013	1.872	1.872	0.000 ( +0.0% )	4.02	4.10	-1.8%	2.516	2.514
2500.021	0.022	0.022	0.000 ( +0.0% )	0.91	0.98	-7.4%	2.384	2.381
2500.022	0.022	0.022	0.000 ( +0.0% )	0.87	0.93	-6.2%	2.374	2.376
2500.023	0.022	0.022	0.000 ( +0.0% )	0.87	0.93	-7.0%	2.253	2.244
2500.024	0.022	0.022	0.000 ( +0.0% )	0.66	0.71	-6.5%	2.469	2.458
2500.031	0.035	0.035	0.000 ( +0.0% )	0.81	0.86	-5.8%	2.439	2.451
2500.032	0.036	0.036	0.000 ( +0.0% )	0.81	0.88	-8.0%	2.414	2.412
2500.033	0.037	0.037	0.000 ( +0.0% )	0.73	0.80	-7.8%	2.487	2.492
2500.034	0.036	0.036	0.000 ( +0.0% )	0.76	0.80	-5.9%	2.472	2.468
2500.101	2.728	2.728	0.000 ( +0.0% )	8.43	9.00	-6.3%	2.510	6.339
2500.111	1.386	1.386	0.000 ( +0.0% )	19.52	20.28	-3.8%	2.210	2.232
2500.112	1.803	1.803	0.000 ( +0.0% )	15.09	15.29	-1.3%	2.285	2.313
2500.131	0.747	0.747	0.000 ( +0.0% )	17.26	18.38	-6.1%	1.508	1.474
2500.201	2.552	2.552	0.000 ( +0.0% )	7.24	7.58	-4.4%	2.075	5.610
2500.211	1.699	1.699	0.000 ( +0.0% )	17.14	18.01	-4.9%	2.279	2.281
2500.212	2.095	2.095	0.000 ( +0.0% )	13.58	14.29	-5.0%	2.362	2.361
2500.221	1.998	1.998	0.000 ( +0.0% )	7.46	7.76	-3.9%	2.003	2.444
2500.222	3.276	3.276	0.000 ( +0.0% )	7.27	7.62	-4.6%	2.081	2.527
2500.223	8.963	8.963	0.000 ( +0.0% )	2.56	2.66	-3.5%	2.110	2.409
2500.224	5.810	5.810	0.000 ( +0.0% )	0.58	0.60	-3.6%	2.107	2.130
2500.225	5.827	5.827	0.000 ( +0.0% )	0.57	0.59	-2.9%	2.124	2.146
2500.226	3.038	3.038	0.000 ( +0.0% )	7.35	7.60	-3.3%	2.082	2.411
2500.227	1.437	1.437	0.000 ( +0.0% )	11.07	11.88	-6.8%	1.431	1.434
2500.231	1.403	1.403	0.000 ( +0.0% )	13.59	13.54	+0.4%	2.181	2.180
2500.232	2.300	2.300	0.000 ( +0.0% )	13.40	14.03	-4.5%	2.272	2.275
2500.233	4.747	4.747	0.000 ( +0.0% )	4.09	4.26	-4.0%	2.293	2.279
2500.234	3.518	3.518	0.000 ( +0.0% )	0.74	0.77	-3.1%	2.073	2.070
2500.235	3.530	3.530	0.000 ( +0.0% )	0.72	0.75	-3.5%	2.098	2.088
2500.236	2.145	2.145	0.000 ( +0.0% )	13.37	13.72	-2.5%	2.272	2.268
2500.237	1.016	1.016	0.000 ( +0.0% )	16.22	17.05	-4.9%	1.463	1.459
2500.241	9.404	9.404	0.000 ( +0.0% )	3.42	3.74	-8.5%	1.916	1.951
2500.242	10.331	10.331	0.000 ( +0.0% )	0.80	0.91	-12.5%	1.703	1.735
2500.243	2.712	2.712	0.000 ( +0.0% )	8.09	8.39	-3.6%	1.077	1.077
2500.244	485.976	485.976	0.000 ( +0.0% )	0.53	0.56	-5.9%	1.674	1.692
2500.245	823.202	823.202	0.000 ( +0.0% )	0.70	0.75	-6.1%	1.663	1.664
2500.901	1.777	1.777	0.000 ( +0.0% )	19.92	21.53	-7.5%	1.418	1.846
2500.902	1.626	1.626	0.000 ( +0.0% )	20.19	21.05	-4.1%	1.323	1.773
2500.911	13.995	13.995	0.000 ( +0.0% )	3.09	2.92	+5.9%	1.090	1.095
2500.912	0.240	0.199	0.041 ( +20.3% )	1.11	1.77	-37.5%	0.975	0.974
2500.913	0.110	0.110	0.000 ( +0.0% )	1.14	1.17	-2.8%	0.976	0.972

hqucms · 2024-10-15T00:34:14Z

+1

hqucms · 2024-10-15T00:36:02Z

@cms-sw/generators-l2 Could you please review and sign? Thanks!

hqucms · 2024-10-17T20:32:20Z

ping @cms-sw/generators-l2

hqucms · 2024-10-17T20:33:01Z

type bugfix

bbilin · 2024-10-23T12:34:07Z

+1

cmsbuild · 2024-10-23T12:34:30Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @rappoccio, @sextonkennedy, @mandrenguyen, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)

mandrenguyen · 2024-10-23T15:45:22Z

+1

cmsbuild added this to the CMSSW_14_2_X milestone Oct 2, 2024

cmsbuild added generators-pending pending-signatures tests-pending orp-pending code-checks-pending labels Oct 2, 2024

cmsbuild added tests-started and removed tests-pending labels Oct 2, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 2, 2024

cmsbuild added tests-approved and removed tests-started labels Oct 2, 2024

hqucms mentioned this pull request Oct 4, 2024

[14_1_X] Fix memory leak in GenParticles2HepMCConverter #46254

Merged

hqucms added 2 commits October 14, 2024 21:05

Change pointer to stack var to avoid memory leak.

fcb891c

Use const pointer.

19d6938

hqucms force-pushed the dev/fix-mem-leak-hepmc-converter branch from 1f38bfa to 19d6938 Compare October 14, 2024 19:07

cmsbuild added tests-pending code-checks-pending and removed tests-approved code-checks-approved labels Oct 14, 2024

cmsbuild added the code-checks-approved label Oct 14, 2024

cmsbuild added tests-started and removed tests-pending labels Oct 14, 2024

cmsbuild added the xpog-pending label Oct 14, 2024

cmsbuild added tests-approved and removed tests-started labels Oct 14, 2024

cmsbuild added xpog-approved and removed xpog-pending labels Oct 15, 2024

cmsbuild added the bug-fix label Oct 17, 2024

cmsbuild added generators-approved fully-signed and removed generators-pending pending-signatures labels Oct 23, 2024

cmsbuild added orp-approved and removed orp-pending labels Oct 23, 2024

cmsbuild merged commit 854bda7 into cms-sw:master Oct 23, 2024
13 checks passed

cmsbuild mentioned this pull request Oct 24, 2024

Fix to L1 ZDC Emulation to fix Et Sum truncation. #46500

Merged

hqucms deleted the dev/fix-mem-leak-hepmc-converter branch December 17, 2024 07:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix memory leak in GenParticles2HepMCConverter #46203

Fix memory leak in GenParticles2HepMCConverter #46203

hqucms commented Oct 2, 2024

cmsbuild commented Oct 2, 2024 •

edited

Loading

hqucms commented Oct 2, 2024

hqucms commented Oct 2, 2024

cmsbuild commented Oct 2, 2024

cmsbuild commented Oct 2, 2024

cmsbuild commented Oct 2, 2024

hqucms commented Oct 3, 2024 •

edited

Loading

mseidel42 commented Oct 3, 2024

hqucms commented Oct 3, 2024

hqucms commented Oct 4, 2024

cmsbuild commented Oct 14, 2024

hqucms commented Oct 14, 2024

hqucms commented Oct 14, 2024

cmsbuild commented Oct 14, 2024

cmsbuild commented Oct 14, 2024

hqucms commented Oct 15, 2024

hqucms commented Oct 15, 2024

hqucms commented Oct 17, 2024

hqucms commented Oct 17, 2024

bbilin commented Oct 23, 2024

cmsbuild commented Oct 23, 2024

mandrenguyen commented Oct 23, 2024

Fix memory leak in GenParticles2HepMCConverter #46203

Fix memory leak in GenParticles2HepMCConverter #46203

Conversation

hqucms commented Oct 2, 2024

PR description:

PR validation:

cmsbuild commented Oct 2, 2024 • edited Loading

hqucms commented Oct 2, 2024

hqucms commented Oct 2, 2024

cmsbuild commented Oct 2, 2024

cmsbuild commented Oct 2, 2024

cmsbuild commented Oct 2, 2024

Comparison Summary

NANO Comparison Summary

hqucms commented Oct 3, 2024 • edited Loading

mseidel42 commented Oct 3, 2024

hqucms commented Oct 3, 2024

hqucms commented Oct 4, 2024

cmsbuild commented Oct 14, 2024

hqucms commented Oct 14, 2024

hqucms commented Oct 14, 2024

cmsbuild commented Oct 14, 2024

cmsbuild commented Oct 14, 2024

Comparison Summary

NANO Comparison Summary

hqucms commented Oct 15, 2024

hqucms commented Oct 15, 2024

hqucms commented Oct 17, 2024

hqucms commented Oct 17, 2024

bbilin commented Oct 23, 2024

cmsbuild commented Oct 23, 2024

mandrenguyen commented Oct 23, 2024

cmsbuild commented Oct 2, 2024 •

edited

Loading

hqucms commented Oct 3, 2024 •

edited

Loading