Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Local and Remote WeightUpdaters #2848

Open
wants to merge 3 commits into
base: gh/vmoens/115/base
Choose a base branch
from
Open

[Feature] Local and Remote WeightUpdaters #2848

wants to merge 3 commits into from

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Mar 12, 2025 •
edited
Loading

Stack from ghstack (oldest at bottom):

@vmoens
Update
032af80 
[ghstack-poisoned]
This was referenced Mar 11, 2025
Merged
Merged
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Mar 12, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2848

Note: Links to docs will display an error until the docs builds have been completed.

❌ 14 New Failures, 3 Unrelated Failures

As of commit 677e543 with merge base 9cd95d5 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens added a commit that referenced this pull request Mar 12, 2025
@vmoens
[Feature] Local and Remote WeightUpdaters
514ea14 
ghstack-source-id: a15a5a1f0d66d0eea474835481d938a17b39e264
Pull Request resolved: #2848
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 12, 2025
@vmoens
Update
6634af4 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Mar 12, 2025
@vmoens
[Feature] Local and Remote WeightUpdaters
9fd9dc5 
ghstack-source-id: 579385cf45f26caaa39a152520673aa5a94bbb0d
Pull Request resolved: #2848
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 12, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}10$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.6458s 0.5470s 1.8283 Ops/s 1.8381 Ops/s $\color{#d91a1a}-0.53\%$
test_transformed 1.1917s 1.0892s 0.9181 Ops/s 0.9397 Ops/s $\color{#d91a1a}-2.29\%$
test_serial 1.5577s 1.5534s 0.6438 Ops/s 0.6261 Ops/s $\color{#35bf28}+2.82\%$
test_parallel 1.4478s 1.3413s 0.7455 Ops/s 0.7650 Ops/s $\color{#d91a1a}-2.55\%$
test_step_mdp_speed[True-True-True-True-True] 0.1674ms 29.7950μs 33.5627 KOps/s 33.5555 KOps/s $\color{#35bf28}+0.02\%$
test_step_mdp_speed[True-True-True-True-False] 0.5645ms 17.9082μs 55.8405 KOps/s 56.9802 KOps/s $\color{#d91a1a}-2.00\%$
test_step_mdp_speed[True-True-True-False-True] 85.5060μs 16.9521μs 58.9896 KOps/s 59.3271 KOps/s $\color{#d91a1a}-0.57\%$
test_step_mdp_speed[True-True-True-False-False] 41.8080μs 10.0615μs 99.3886 KOps/s 100.9160 KOps/s $\color{#d91a1a}-1.51\%$
test_step_mdp_speed[True-True-False-True-True] 95.7320μs 31.7949μs 31.4516 KOps/s 31.0784 KOps/s $\color{#35bf28}+1.20\%$
test_step_mdp_speed[True-True-False-True-False] 80.0100μs 19.4276μs 51.4731 KOps/s 51.6207 KOps/s $\color{#d91a1a}-0.29\%$
test_step_mdp_speed[True-True-False-False-True] 53.5400μs 18.7595μs 53.3065 KOps/s 53.5019 KOps/s $\color{#d91a1a}-0.37\%$
test_step_mdp_speed[True-True-False-False-False] 76.1320μs 11.7236μs 85.2983 KOps/s 85.1427 KOps/s $\color{#35bf28}+0.18\%$
test_step_mdp_speed[True-False-True-True-True] 80.2800μs 33.8098μs 29.5772 KOps/s 29.6709 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[True-False-True-True-False] 66.0430μs 21.3798μs 46.7732 KOps/s 46.5599 KOps/s $\color{#35bf28}+0.46\%$
test_step_mdp_speed[True-False-True-False-True] 77.4040μs 18.9625μs 52.7357 KOps/s 53.2746 KOps/s $\color{#d91a1a}-1.01\%$
test_step_mdp_speed[True-False-True-False-False] 43.5710μs 11.6732μs 85.6664 KOps/s 85.1622 KOps/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[True-False-False-True-True] 98.1730μs 35.5808μs 28.1050 KOps/s 28.0300 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[True-False-False-True-False] 54.6420μs 22.9742μs 43.5272 KOps/s 43.4929 KOps/s $\color{#35bf28}+0.08\%$
test_step_mdp_speed[True-False-False-False-True] 86.2010μs 20.5144μs 48.7461 KOps/s 48.5201 KOps/s $\color{#35bf28}+0.47\%$
test_step_mdp_speed[True-False-False-False-False] 45.5650μs 13.3868μs 74.7003 KOps/s 74.0347 KOps/s $\color{#35bf28}+0.90\%$
test_step_mdp_speed[False-True-True-True-True] 93.7550μs 33.7925μs 29.5923 KOps/s 29.5292 KOps/s $\color{#35bf28}+0.21\%$
test_step_mdp_speed[False-True-True-True-False] 77.8660μs 21.4902μs 46.5329 KOps/s 46.5002 KOps/s $\color{#35bf28}+0.07\%$
test_step_mdp_speed[False-True-True-False-True] 82.1040μs 21.7108μs 46.0600 KOps/s 46.7328 KOps/s $\color{#d91a1a}-1.44\%$
test_step_mdp_speed[False-True-True-False-False] 0.6232ms 13.0823μs 76.4391 KOps/s 76.2426 KOps/s $\color{#35bf28}+0.26\%$
test_step_mdp_speed[False-True-False-True-True] 73.1570μs 35.4856μs 28.1804 KOps/s 28.1034 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[False-True-False-True-False] 2.4709ms 23.2262μs 43.0548 KOps/s 43.0314 KOps/s $\color{#35bf28}+0.05\%$
test_step_mdp_speed[False-True-False-False-True] 94.4360μs 23.2407μs 43.0279 KOps/s 43.2254 KOps/s $\color{#d91a1a}-0.46\%$
test_step_mdp_speed[False-True-False-False-False] 47.0670μs 14.7916μs 67.6060 KOps/s 67.7288 KOps/s $\color{#d91a1a}-0.18\%$
test_step_mdp_speed[False-False-True-True-True] 98.4330μs 37.1724μs 26.9017 KOps/s 26.8291 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[False-False-True-True-False] 84.7170μs 24.9277μs 40.1161 KOps/s 40.2664 KOps/s $\color{#d91a1a}-0.37\%$
test_step_mdp_speed[False-False-True-False-True] 58.4790μs 23.2719μs 42.9702 KOps/s 43.0802 KOps/s $\color{#d91a1a}-0.26\%$
test_step_mdp_speed[False-False-True-False-False] 74.2890μs 14.7717μs 67.6969 KOps/s 67.4709 KOps/s $\color{#35bf28}+0.33\%$
test_step_mdp_speed[False-False-False-True-True] 90.7190μs 38.4425μs 26.0128 KOps/s 25.7311 KOps/s $\color{#35bf28}+1.10\%$
test_step_mdp_speed[False-False-False-True-False] 85.9800μs 26.1390μs 38.2570 KOps/s 37.6422 KOps/s $\color{#35bf28}+1.63\%$
test_step_mdp_speed[False-False-False-False-True] 83.5460μs 24.6083μs 40.6367 KOps/s 39.6757 KOps/s $\color{#35bf28}+2.42\%$
test_step_mdp_speed[False-False-False-False-False] 52.7980μs 16.2789μs 61.4293 KOps/s 59.8885 KOps/s $\color{#35bf28}+2.57\%$
test_values[generalized_advantage_estimate-True-True] 10.3092ms 9.8872ms 101.1410 Ops/s 101.3545 Ops/s $\color{#d91a1a}-0.21\%$
test_values[vec_generalized_advantage_estimate-True-True] 28.6156ms 26.7399ms 37.3973 Ops/s 40.8117 Ops/s $\textbf{\color{#d91a1a}-8.37\%}$
test_values[td0_return_estimate-False-False] 0.2597ms 0.1894ms 5.2812 KOps/s 5.2108 KOps/s $\color{#35bf28}+1.35\%$
test_values[td1_return_estimate-False-False] 29.1624ms 24.4895ms 40.8339 Ops/s 40.6220 Ops/s $\color{#35bf28}+0.52\%$
test_values[vec_td1_return_estimate-False-False] 31.0071ms 27.0700ms 36.9412 Ops/s 40.6402 Ops/s $\textbf{\color{#d91a1a}-9.10\%}$
test_values[td_lambda_return_estimate-True-False] 59.0084ms 35.5091ms 28.1618 Ops/s 28.3347 Ops/s $\color{#d91a1a}-0.61\%$
test_values[vec_td_lambda_return_estimate-True-False] 29.3204ms 27.0839ms 36.9223 Ops/s 40.7559 Ops/s $\textbf{\color{#d91a1a}-9.41\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.7361ms 8.4422ms 118.4532 Ops/s 117.7975 Ops/s $\color{#35bf28}+0.56\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.3459ms 1.8816ms 531.4622 Ops/s 544.4634 Ops/s $\color{#d91a1a}-2.39\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5414ms 0.3621ms 2.7614 KOps/s 2.6844 KOps/s $\color{#35bf28}+2.87\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 44.0936ms 42.1358ms 23.7328 Ops/s 23.7565 Ops/s $\color{#d91a1a}-0.10\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.3197ms 3.4844ms 286.9931 Ops/s 285.5705 Ops/s $\color{#35bf28}+0.50\%$
test_dqn_speed[False-None] 5.9713ms 1.4689ms 680.7643 Ops/s 694.1548 Ops/s $\color{#d91a1a}-1.93\%$
test_dqn_speed[False-backward] 1.9798ms 1.8891ms 529.3607 Ops/s 518.6062 Ops/s $\color{#35bf28}+2.07\%$
test_dqn_speed[True-None] 0.8455ms 0.5576ms 1.7935 KOps/s 1.7569 KOps/s $\color{#35bf28}+2.08\%$
test_dqn_speed[True-backward] 1.0489ms 0.9834ms 1.0169 KOps/s 997.7144 Ops/s $\color{#35bf28}+1.92\%$
test_dqn_speed[reduce-overhead-None] 0.7141ms 0.5588ms 1.7895 KOps/s 1.7480 KOps/s $\color{#35bf28}+2.37\%$
test_dqn_speed[reduce-overhead-backward] 1.1070ms 0.9920ms 1.0081 KOps/s 982.3501 Ops/s $\color{#35bf28}+2.62\%$
test_ddpg_speed[False-None] 3.4068ms 2.9589ms 337.9602 Ops/s 337.4086 Ops/s $\color{#35bf28}+0.16\%$
test_ddpg_speed[False-backward] 4.2977ms 4.0940ms 244.2579 Ops/s 240.0044 Ops/s $\color{#35bf28}+1.77\%$
test_ddpg_speed[True-None] 1.9375ms 1.4429ms 693.0554 Ops/s 681.6354 Ops/s $\color{#35bf28}+1.68\%$
test_ddpg_speed[True-backward] 2.5779ms 2.3736ms 421.3036 Ops/s 416.6515 Ops/s $\color{#35bf28}+1.12\%$
test_ddpg_speed[reduce-overhead-None] 1.6788ms 1.4304ms 699.0807 Ops/s 685.4358 Ops/s $\color{#35bf28}+1.99\%$
test_ddpg_speed[reduce-overhead-backward] 2.4386ms 2.3142ms 432.1154 Ops/s 420.8692 Ops/s $\color{#35bf28}+2.67\%$
test_sac_speed[False-None] 9.7491ms 8.2676ms 120.9537 Ops/s 117.5027 Ops/s $\color{#35bf28}+2.94\%$
test_sac_speed[False-backward] 12.6946ms 11.1473ms 89.7082 Ops/s 88.4006 Ops/s $\color{#35bf28}+1.48\%$
test_sac_speed[True-None] 3.3157ms 2.6175ms 382.0470 Ops/s 370.8676 Ops/s $\color{#35bf28}+3.01\%$
test_sac_speed[True-backward] 4.9197ms 4.6975ms 212.8781 Ops/s 218.2968 Ops/s $\color{#d91a1a}-2.48\%$
test_sac_speed[reduce-overhead-None] 3.3765ms 2.6357ms 379.4119 Ops/s 368.7205 Ops/s $\color{#35bf28}+2.90\%$
test_sac_speed[reduce-overhead-backward] 4.9848ms 4.5959ms 217.5829 Ops/s 227.4166 Ops/s $\color{#d91a1a}-4.32\%$
test_redq_speed[False-None] 21.9622ms 14.2489ms 70.1807 Ops/s 72.8705 Ops/s $\color{#d91a1a}-3.69\%$
test_redq_speed[False-backward] 29.2981ms 23.7222ms 42.1545 Ops/s 42.1113 Ops/s $\color{#35bf28}+0.10\%$
test_redq_speed[True-None] 9.0178ms 7.4162ms 134.8393 Ops/s 129.1329 Ops/s $\color{#35bf28}+4.42\%$
test_redq_speed[True-backward] 16.5572ms 15.4350ms 64.7878 Ops/s 65.6463 Ops/s $\color{#d91a1a}-1.31\%$
test_redq_speed[reduce-overhead-None] 9.6190ms 7.7291ms 129.3817 Ops/s 130.6327 Ops/s $\color{#d91a1a}-0.96\%$
test_redq_speed[reduce-overhead-backward] 16.3457ms 15.2327ms 65.6481 Ops/s 62.8956 Ops/s $\color{#35bf28}+4.38\%$
test_redq_deprec_speed[False-None] 15.6399ms 13.8446ms 72.2302 Ops/s 69.6937 Ops/s $\color{#35bf28}+3.64\%$
test_redq_deprec_speed[False-backward] 21.0543ms 19.9705ms 50.0739 Ops/s 48.3764 Ops/s $\color{#35bf28}+3.51\%$
test_redq_deprec_speed[True-None] 7.0771ms 5.8903ms 169.7702 Ops/s 167.0676 Ops/s $\color{#35bf28}+1.62\%$
test_redq_deprec_speed[True-backward] 12.7881ms 11.1615ms 89.5936 Ops/s 91.5073 Ops/s $\color{#d91a1a}-2.09\%$
test_redq_deprec_speed[reduce-overhead-None] 6.6227ms 5.9146ms 169.0718 Ops/s 173.2073 Ops/s $\color{#d91a1a}-2.39\%$
test_redq_deprec_speed[reduce-overhead-backward] 11.7847ms 11.0710ms 90.3262 Ops/s 89.3162 Ops/s $\color{#35bf28}+1.13\%$
test_td3_speed[False-None] 8.8605ms 8.4122ms 118.8744 Ops/s 116.7541 Ops/s $\color{#35bf28}+1.82\%$
test_td3_speed[False-backward] 12.6915ms 11.1615ms 89.5939 Ops/s 90.3013 Ops/s $\color{#d91a1a}-0.78\%$
test_td3_speed[True-None] 2.6877ms 2.3487ms 425.7656 Ops/s 431.3957 Ops/s $\color{#d91a1a}-1.31\%$
test_td3_speed[True-backward] 5.0666ms 4.4970ms 222.3704 Ops/s 247.7598 Ops/s $\textbf{\color{#d91a1a}-10.25\%}$
test_td3_speed[reduce-overhead-None] 2.5834ms 2.3205ms 430.9482 Ops/s 432.1735 Ops/s $\color{#d91a1a}-0.28\%$
test_td3_speed[reduce-overhead-backward] 5.1118ms 4.1955ms 238.3498 Ops/s 241.6005 Ops/s $\color{#d91a1a}-1.35\%$
test_cql_speed[False-None] 38.4939ms 36.7477ms 27.2126 Ops/s 26.3021 Ops/s $\color{#35bf28}+3.46\%$
test_cql_speed[False-backward] 47.8616ms 46.6654ms 21.4292 Ops/s 20.6625 Ops/s $\color{#35bf28}+3.71\%$
test_cql_speed[True-None] 24.5294ms 23.1873ms 43.1271 Ops/s 43.8339 Ops/s $\color{#d91a1a}-1.61\%$
test_cql_speed[True-backward] 31.7751ms 30.3317ms 32.9688 Ops/s 33.6436 Ops/s $\color{#d91a1a}-2.01\%$
test_cql_speed[reduce-overhead-None] 24.2563ms 22.8339ms 43.7946 Ops/s 43.7901 Ops/s $\color{#35bf28}+0.01\%$
test_cql_speed[reduce-overhead-backward] 31.8315ms 30.2689ms 33.0372 Ops/s 33.1111 Ops/s $\color{#d91a1a}-0.22\%$
test_a2c_speed[False-None] 9.1985ms 7.5544ms 132.3726 Ops/s 127.4662 Ops/s $\color{#35bf28}+3.85\%$
test_a2c_speed[False-backward] 17.1221ms 14.9319ms 66.9707 Ops/s 65.4133 Ops/s $\color{#35bf28}+2.38\%$
test_a2c_speed[True-None] 6.0577ms 5.0631ms 197.5089 Ops/s 198.5632 Ops/s $\color{#d91a1a}-0.53\%$
test_a2c_speed[True-backward] 13.0103ms 12.2249ms 81.8004 Ops/s 84.9751 Ops/s $\color{#d91a1a}-3.74\%$
test_a2c_speed[reduce-overhead-None] 6.1878ms 5.0198ms 199.2102 Ops/s 200.8424 Ops/s $\color{#d91a1a}-0.81\%$
test_a2c_speed[reduce-overhead-backward] 12.8633ms 11.8202ms 84.6006 Ops/s 85.1927 Ops/s $\color{#d91a1a}-0.69\%$
test_ppo_speed[False-None] 8.8869ms 7.8980ms 126.6144 Ops/s 129.2685 Ops/s $\color{#d91a1a}-2.05\%$
test_ppo_speed[False-backward] 16.9105ms 15.9579ms 62.6650 Ops/s 64.4221 Ops/s $\color{#d91a1a}-2.73\%$
test_ppo_speed[True-None] 5.8336ms 5.4075ms 184.9271 Ops/s 186.9856 Ops/s $\color{#d91a1a}-1.10\%$
test_ppo_speed[True-backward] 13.8206ms 12.0527ms 82.9688 Ops/s 85.1751 Ops/s $\color{#d91a1a}-2.59\%$
test_ppo_speed[reduce-overhead-None] 6.6968ms 5.7949ms 172.5657 Ops/s 190.9157 Ops/s $\textbf{\color{#d91a1a}-9.61\%}$
test_ppo_speed[reduce-overhead-backward] 12.5862ms 11.8222ms 84.5867 Ops/s 86.0391 Ops/s $\color{#d91a1a}-1.69\%$
test_reinforce_speed[False-None] 8.8768ms 7.2368ms 138.1819 Ops/s 145.7555 Ops/s $\textbf{\color{#d91a1a}-5.20\%}$
test_reinforce_speed[False-backward] 11.5613ms 10.7428ms 93.0858 Ops/s 96.5111 Ops/s $\color{#d91a1a}-3.55\%$
test_reinforce_speed[True-None] 5.4568ms 4.4221ms 226.1349 Ops/s 223.8095 Ops/s $\color{#35bf28}+1.04\%$
test_reinforce_speed[True-backward] 11.0843ms 10.7876ms 92.6987 Ops/s 93.8350 Ops/s $\color{#d91a1a}-1.21\%$
test_reinforce_speed[reduce-overhead-None] 5.0999ms 4.4648ms 223.9727 Ops/s 238.2502 Ops/s $\textbf{\color{#d91a1a}-5.99\%}$
test_reinforce_speed[reduce-overhead-backward] 11.4100ms 10.7599ms 92.9375 Ops/s 93.3732 Ops/s $\color{#d91a1a}-0.47\%$
test_iql_speed[False-None] 39.8288ms 34.1168ms 29.3110 Ops/s 29.1372 Ops/s $\color{#35bf28}+0.60\%$
test_iql_speed[False-backward] 48.7500ms 46.6492ms 21.4366 Ops/s 21.1964 Ops/s $\color{#35bf28}+1.13\%$
test_iql_speed[True-None] 17.9832ms 16.4339ms 60.8497 Ops/s 60.8222 Ops/s $\color{#35bf28}+0.05\%$
test_iql_speed[True-backward] 29.6081ms 28.2984ms 35.3376 Ops/s 35.1268 Ops/s $\color{#35bf28}+0.60\%$
test_iql_speed[reduce-overhead-None] 17.4763ms 16.3295ms 61.2390 Ops/s 60.5987 Ops/s $\color{#35bf28}+1.06\%$
test_iql_speed[reduce-overhead-backward] 29.6531ms 28.1457ms 35.5294 Ops/s 35.1752 Ops/s $\color{#35bf28}+1.01\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 8.1533ms 5.1083ms 195.7593 Ops/s 192.0812 Ops/s $\color{#35bf28}+1.91\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8109ms 0.5339ms 1.8729 KOps/s 1.8819 KOps/s $\color{#d91a1a}-0.48\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7568ms 0.5058ms 1.9771 KOps/s 1.9315 KOps/s $\color{#35bf28}+2.36\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.5468ms 4.9903ms 200.3897 Ops/s 201.6597 Ops/s $\color{#d91a1a}-0.63\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3.0443ms 0.5308ms 1.8839 KOps/s 1.9198 KOps/s $\color{#d91a1a}-1.87\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.9428ms 0.5034ms 1.9867 KOps/s 1.9792 KOps/s $\color{#35bf28}+0.38\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.9782ms 1.6583ms 603.0314 Ops/s 560.9122 Ops/s $\textbf{\color{#35bf28}+7.51\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.0712ms 1.5708ms 636.6110 Ops/s 627.2764 Ops/s $\color{#35bf28}+1.49\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.4573ms 5.1455ms 194.3431 Ops/s 196.0280 Ops/s $\color{#d91a1a}-0.86\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 3.6338ms 0.6774ms 1.4761 KOps/s 1.4875 KOps/s $\color{#d91a1a}-0.76\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9114ms 0.6388ms 1.5655 KOps/s 1.5464 KOps/s $\color{#35bf28}+1.24\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.7872ms 5.0173ms 199.3099 Ops/s 201.9157 Ops/s $\color{#d91a1a}-1.29\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9735ms 0.5283ms 1.8929 KOps/s 1.8592 KOps/s $\color{#35bf28}+1.81\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7629ms 0.5111ms 1.9564 KOps/s 1.9371 KOps/s $\color{#35bf28}+1.00\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.4603ms 4.9652ms 201.4032 Ops/s 200.3825 Ops/s $\color{#35bf28}+0.51\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.1257ms 0.5414ms 1.8472 KOps/s 1.8708 KOps/s $\color{#d91a1a}-1.26\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7530ms 0.5016ms 1.9935 KOps/s 1.9867 KOps/s $\color{#35bf28}+0.34\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.7058ms 5.1928ms 192.5726 Ops/s 199.0967 Ops/s $\color{#d91a1a}-3.28\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1902ms 0.6973ms 1.4341 KOps/s 1.5074 KOps/s $\color{#d91a1a}-4.86\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9395ms 0.6489ms 1.5410 KOps/s 1.5502 KOps/s $\color{#d91a1a}-0.59\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.6724ms 4.3978ms 227.3868 Ops/s 247.9874 Ops/s $\textbf{\color{#d91a1a}-8.31\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.8589s 19.4977ms 51.2880 Ops/s 428.0941 Ops/s $\textbf{\color{#d91a1a}-88.02\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.9769ms 1.3449ms 743.5620 Ops/s 736.8841 Ops/s $\color{#35bf28}+0.91\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 6.0168ms 4.3751ms 228.5648 Ops/s 23.5690 Ops/s $\textbf{\color{#35bf28}+869.77\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 5.8964ms 2.3068ms 433.5062 Ops/s 406.8675 Ops/s $\textbf{\color{#35bf28}+6.55\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.2154ms 1.4623ms 683.8626 Ops/s 757.0481 Ops/s $\textbf{\color{#d91a1a}-9.67\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 5.8544ms 4.3857ms 228.0115 Ops/s 215.6672 Ops/s $\textbf{\color{#35bf28}+5.72\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 9.2994ms 2.5217ms 396.5575 Ops/s 408.0335 Ops/s $\color{#d91a1a}-2.81\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 5.7338ms 1.5429ms 648.1369 Ops/s 621.7276 Ops/s $\color{#35bf28}+4.25\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 60.2047ms 49.4147ms 20.2369 Ops/s 19.5470 Ops/s $\color{#35bf28}+3.53\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 18.2418ms 14.2314ms 70.2671 Ops/s 69.9396 Ops/s $\color{#35bf28}+0.47\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 60.0995ms 50.5098ms 19.7982 Ops/s 19.4398 Ops/s $\color{#35bf28}+1.84\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 15.7171ms 14.3896ms 69.4945 Ops/s 68.4444 Ops/s $\color{#35bf28}+1.53\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 56.5788ms 48.4642ms 20.6338 Ops/s 20.4085 Ops/s $\color{#35bf28}+1.10\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 17.6718ms 15.7630ms 63.4398 Ops/s 63.1028 Ops/s $\color{#35bf28}+0.53\%$

@vmoens
Update
677e543 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Mar 12, 2025
@vmoens
[Feature] Local and Remote WeightUpdaters
ed68115 
ghstack-source-id: d361b5a55af36c51eb6c42efdc7ccbec240acd7a
Pull Request resolved: #2848
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot