Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Doc] Better doc for ditributed RBs #2378

Open
wants to merge 38 commits into
base: main
Choose a base branch
from
Open

[Doc] Better doc for ditributed RBs #2378

wants to merge 38 commits into from

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Aug 7, 2024 •
edited
Loading

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Aug 7, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2378

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 1 Cancelled Job, 7 Unrelated Failures

As of commit cb23ae2 with merge base 607db8b (image):

NEW FAILURES - The following jobs have failed:

CANCELLED JOB - The following job was cancelled. Please retry:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 7, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 7, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}12$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 55.4847ms 54.6867ms 18.2860 Ops/s 18.2526 Ops/s $\color{#35bf28}+0.18\%$
test_sync 35.4435ms 30.2205ms 33.0902 Ops/s 32.6067 Ops/s $\color{#35bf28}+1.48\%$
test_async 48.0751ms 29.3927ms 34.0221 Ops/s 34.7889 Ops/s $\color{#d91a1a}-2.20\%$
test_simple 0.4417s 0.3805s 2.6282 Ops/s 2.5856 Ops/s $\color{#35bf28}+1.65\%$
test_transformed 0.5843s 0.5280s 1.8941 Ops/s 1.8514 Ops/s $\color{#35bf28}+2.31\%$
test_serial 1.2256s 1.1657s 0.8578 Ops/s 0.8441 Ops/s $\color{#35bf28}+1.63\%$
test_parallel 1.1181s 1.0524s 0.9502 Ops/s 0.9375 Ops/s $\color{#35bf28}+1.36\%$
test_step_mdp_speed[True-True-True-True-True] 0.1513ms 23.9508μs 41.7523 KOps/s 42.5497 KOps/s $\color{#d91a1a}-1.87\%$
test_step_mdp_speed[True-True-True-True-False] 45.3350μs 13.6642μs 73.1839 KOps/s 75.1084 KOps/s $\color{#d91a1a}-2.56\%$
test_step_mdp_speed[True-True-True-False-True] 42.4300μs 13.5019μs 74.0635 KOps/s 76.0174 KOps/s $\color{#d91a1a}-2.57\%$
test_step_mdp_speed[True-True-True-False-False] 58.7940μs 7.9267μs 126.1552 KOps/s 128.5276 KOps/s $\color{#d91a1a}-1.85\%$
test_step_mdp_speed[True-True-False-True-True] 55.6340μs 25.5585μs 39.1259 KOps/s 40.5174 KOps/s $\color{#d91a1a}-3.43\%$
test_step_mdp_speed[True-True-False-True-False] 37.5700μs 15.1546μs 65.9866 KOps/s 67.2349 KOps/s $\color{#d91a1a}-1.86\%$
test_step_mdp_speed[True-True-False-False-True] 43.5520μs 15.4002μs 64.9342 KOps/s 68.1581 KOps/s $\color{#d91a1a}-4.73\%$
test_step_mdp_speed[True-True-False-False-False] 35.4260μs 9.5423μs 104.7961 KOps/s 108.7150 KOps/s $\color{#d91a1a}-3.60\%$
test_step_mdp_speed[True-False-True-True-True] 60.1230μs 26.9891μs 37.0519 KOps/s 38.2750 KOps/s $\color{#d91a1a}-3.20\%$
test_step_mdp_speed[True-False-True-True-False] 41.2370μs 16.8526μs 59.3379 KOps/s 61.5165 KOps/s $\color{#d91a1a}-3.54\%$
test_step_mdp_speed[True-False-True-False-True] 57.3170μs 15.3867μs 64.9910 KOps/s 67.9388 KOps/s $\color{#d91a1a}-4.34\%$
test_step_mdp_speed[True-False-True-False-False] 33.3320μs 9.6613μs 103.5055 KOps/s 109.8698 KOps/s $\textbf{\color{#d91a1a}-5.79\%}$
test_step_mdp_speed[True-False-False-True-True] 61.5860μs 28.2987μs 35.3373 KOps/s 36.6256 KOps/s $\color{#d91a1a}-3.52\%$
test_step_mdp_speed[True-False-False-True-False] 48.2100μs 18.1806μs 55.0037 KOps/s 56.1238 KOps/s $\color{#d91a1a}-2.00\%$
test_step_mdp_speed[True-False-False-False-True] 50.1540μs 16.1988μs 61.7329 KOps/s 60.0447 KOps/s $\color{#35bf28}+2.81\%$
test_step_mdp_speed[True-False-False-False-False] 30.8480μs 10.6743μs 93.6827 KOps/s 95.7507 KOps/s $\color{#d91a1a}-2.16\%$
test_step_mdp_speed[False-True-True-True-True] 58.4090μs 26.9877μs 37.0539 KOps/s 38.0887 KOps/s $\color{#d91a1a}-2.72\%$
test_step_mdp_speed[False-True-True-True-False] 45.4450μs 16.7910μs 59.5558 KOps/s 60.6591 KOps/s $\color{#d91a1a}-1.82\%$
test_step_mdp_speed[False-True-True-False-True] 46.5570μs 17.2622μs 57.9299 KOps/s 59.0138 KOps/s $\color{#d91a1a}-1.84\%$
test_step_mdp_speed[False-True-True-False-False] 32.5910μs 10.8274μs 92.3584 KOps/s 97.6789 KOps/s $\textbf{\color{#d91a1a}-5.45\%}$
test_step_mdp_speed[False-True-False-True-True] 68.5880μs 28.0204μs 35.6882 KOps/s 36.2684 KOps/s $\color{#d91a1a}-1.60\%$
test_step_mdp_speed[False-True-False-True-False] 93.4850μs 18.3542μs 54.4835 KOps/s 56.4093 KOps/s $\color{#d91a1a}-3.41\%$
test_step_mdp_speed[False-True-False-False-True] 45.9760μs 18.6307μs 53.6750 KOps/s 54.0976 KOps/s $\color{#d91a1a}-0.78\%$
test_step_mdp_speed[False-True-False-False-False] 46.0730μs 11.9331μs 83.8007 KOps/s 85.4748 KOps/s $\color{#d91a1a}-1.96\%$
test_step_mdp_speed[False-False-True-True-True] 3.2282ms 30.1735μs 33.1416 KOps/s 34.1786 KOps/s $\color{#d91a1a}-3.03\%$
test_step_mdp_speed[False-False-True-True-False] 68.8820μs 19.6436μs 50.9072 KOps/s 52.4045 KOps/s $\color{#d91a1a}-2.86\%$
test_step_mdp_speed[False-False-True-False-True] 43.8620μs 18.8066μs 53.1727 KOps/s 52.8643 KOps/s $\color{#35bf28}+0.58\%$
test_step_mdp_speed[False-False-True-False-False] 39.1740μs 12.0268μs 83.1474 KOps/s 86.0083 KOps/s $\color{#d91a1a}-3.33\%$
test_step_mdp_speed[False-False-False-True-True] 0.1494ms 31.5241μs 31.7217 KOps/s 32.9139 KOps/s $\color{#d91a1a}-3.62\%$
test_step_mdp_speed[False-False-False-True-False] 47.4690μs 20.7667μs 48.1540 KOps/s 48.7551 KOps/s $\color{#d91a1a}-1.23\%$
test_step_mdp_speed[False-False-False-False-True] 47.3680μs 20.2386μs 49.4105 KOps/s 50.5967 KOps/s $\color{#d91a1a}-2.34\%$
test_step_mdp_speed[False-False-False-False-False] 46.9280μs 13.2660μs 75.3808 KOps/s 77.3450 KOps/s $\color{#d91a1a}-2.54\%$
test_values[generalized_advantage_estimate-True-True] 10.2631ms 9.2501ms 108.1068 Ops/s 110.8054 Ops/s $\color{#d91a1a}-2.44\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.4875ms 33.9353ms 29.4678 Ops/s 31.6307 Ops/s $\textbf{\color{#d91a1a}-6.84\%}$
test_values[td0_return_estimate-False-False] 0.2011ms 0.1611ms 6.2066 KOps/s 6.3890 KOps/s $\color{#d91a1a}-2.85\%$
test_values[td1_return_estimate-False-False] 23.6438ms 22.5276ms 44.3900 Ops/s 44.6311 Ops/s $\color{#d91a1a}-0.54\%$
test_values[vec_td1_return_estimate-False-False] 35.4703ms 33.7426ms 29.6361 Ops/s 31.4567 Ops/s $\textbf{\color{#d91a1a}-5.79\%}$
test_values[td_lambda_return_estimate-True-False] 34.0210ms 32.0667ms 31.1850 Ops/s 31.1367 Ops/s $\color{#35bf28}+0.15\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.3547ms 33.8835ms 29.5129 Ops/s 31.6939 Ops/s $\textbf{\color{#d91a1a}-6.88\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.2426ms 7.7637ms 128.8051 Ops/s 129.8027 Ops/s $\color{#d91a1a}-0.77\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.1855ms 1.9813ms 504.7146 Ops/s 567.2789 Ops/s $\textbf{\color{#d91a1a}-11.03\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5745ms 0.3414ms 2.9289 KOps/s 2.9379 KOps/s $\color{#d91a1a}-0.31\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 47.9069ms 45.8392ms 21.8154 Ops/s 23.4169 Ops/s $\textbf{\color{#d91a1a}-6.84\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.6995ms 2.8789ms 347.3528 Ops/s 351.2498 Ops/s $\color{#d91a1a}-1.11\%$
test_dqn_speed 6.0692ms 1.2333ms 810.8439 Ops/s 810.9851 Ops/s $\color{#d91a1a}-0.02\%$
test_ddpg_speed 3.2766ms 2.5822ms 387.2629 Ops/s 389.0756 Ops/s $\color{#d91a1a}-0.47\%$
test_sac_speed 8.0722ms 7.5983ms 131.6088 Ops/s 132.9999 Ops/s $\color{#d91a1a}-1.05\%$
test_redq_speed 13.3449ms 11.7431ms 85.1564 Ops/s 85.2050 Ops/s $\color{#d91a1a}-0.06\%$
test_redq_deprec_speed 13.3725ms 11.8852ms 84.1383 Ops/s 83.9938 Ops/s $\color{#35bf28}+0.17\%$
test_td3_speed 7.8493ms 7.4705ms 133.8599 Ops/s 133.0553 Ops/s $\color{#35bf28}+0.60\%$
test_cql_speed 35.3548ms 33.4043ms 29.9363 Ops/s 30.0599 Ops/s $\color{#d91a1a}-0.41\%$
test_a2c_speed 7.3511ms 6.8484ms 146.0203 Ops/s 146.7050 Ops/s $\color{#d91a1a}-0.47\%$
test_ppo_speed 8.6224ms 7.1394ms 140.0672 Ops/s 141.4372 Ops/s $\color{#d91a1a}-0.97\%$
test_reinforce_speed 6.9854ms 6.0749ms 164.6128 Ops/s 164.5592 Ops/s $\color{#35bf28}+0.03\%$
test_iql_speed 31.7072ms 30.7984ms 32.4692 Ops/s 33.3447 Ops/s $\color{#d91a1a}-2.63\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.0435ms 4.4400ms 225.2264 Ops/s 226.0509 Ops/s $\color{#d91a1a}-0.36\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7545ms 0.4459ms 2.2425 KOps/s 2.2405 KOps/s $\color{#35bf28}+0.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5987ms 0.4293ms 2.3293 KOps/s 2.3542 KOps/s $\color{#d91a1a}-1.06\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.0112ms 4.3516ms 229.7986 Ops/s 229.5458 Ops/s $\color{#35bf28}+0.11\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7909ms 0.4467ms 2.2384 KOps/s 2.2444 KOps/s $\color{#d91a1a}-0.27\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6200ms 0.4195ms 2.3836 KOps/s 2.3836 KOps/s $-0.00\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7503ms 1.5980ms 625.7755 Ops/s 609.6883 Ops/s $\color{#35bf28}+2.64\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.1445ms 1.5073ms 663.4385 Ops/s 664.1456 Ops/s $\color{#d91a1a}-0.11\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.8245ms 4.4613ms 224.1482 Ops/s 221.8392 Ops/s $\color{#35bf28}+1.04\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7727ms 0.5811ms 1.7208 KOps/s 1.7379 KOps/s $\color{#d91a1a}-0.98\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8073ms 0.5539ms 1.8054 KOps/s 1.8137 KOps/s $\color{#d91a1a}-0.46\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.5814ms 4.5024ms 222.1036 Ops/s 226.2082 Ops/s $\color{#d91a1a}-1.81\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.1871ms 0.5196ms 1.9247 KOps/s 2.2564 KOps/s $\textbf{\color{#d91a1a}-14.70\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6198ms 0.4296ms 2.3278 KOps/s 2.3478 KOps/s $\color{#d91a1a}-0.85\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.1174ms 4.4345ms 225.5066 Ops/s 230.0560 Ops/s $\color{#d91a1a}-1.98\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.6235ms 0.4550ms 2.1979 KOps/s 2.2706 KOps/s $\color{#d91a1a}-3.20\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6934ms 0.4243ms 2.3566 KOps/s 2.3849 KOps/s $\color{#d91a1a}-1.19\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.7647ms 4.5027ms 222.0866 Ops/s 220.8794 Ops/s $\color{#35bf28}+0.55\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8904ms 0.5886ms 1.6990 KOps/s 1.7299 KOps/s $\color{#d91a1a}-1.79\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.3758ms 0.5848ms 1.7101 KOps/s 1.8285 KOps/s $\textbf{\color{#d91a1a}-6.47\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1225s 7.9883ms 125.1833 Ops/s 173.0482 Ops/s $\textbf{\color{#d91a1a}-27.66\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 15.3910ms 11.9838ms 83.4459 Ops/s 83.7672 Ops/s $\color{#d91a1a}-0.38\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.6926ms 1.0683ms 936.0748 Ops/s 969.9825 Ops/s $\color{#d91a1a}-3.50\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1028s 5.5709ms 179.5047 Ops/s 176.5661 Ops/s $\color{#35bf28}+1.66\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.0829ms 12.1974ms 81.9850 Ops/s 83.1191 Ops/s $\color{#d91a1a}-1.36\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.5744ms 1.0454ms 956.5702 Ops/s 967.4765 Ops/s $\color{#d91a1a}-1.13\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1050s 5.7289ms 174.5532 Ops/s 127.4447 Ops/s $\textbf{\color{#35bf28}+36.96\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 16.5409ms 12.8608ms 77.7555 Ops/s 82.0904 Ops/s $\textbf{\color{#d91a1a}-5.28\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.7356ms 1.2399ms 806.5482 Ops/s 857.2737 Ops/s $\textbf{\color{#d91a1a}-5.92\%}$
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 7, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1053s 0.1049s 9.5311 Ops/s 9.5246 Ops/s $\color{#35bf28}+0.07\%$
test_sync 94.1566ms 91.7009ms 10.9050 Ops/s 11.1997 Ops/s $\color{#d91a1a}-2.63\%$
test_async 0.1747s 85.2263ms 11.7335 Ops/s 11.5224 Ops/s $\color{#35bf28}+1.83\%$
test_single_pixels 0.1156s 0.1153s 8.6714 Ops/s 8.6205 Ops/s $\color{#35bf28}+0.59\%$
test_sync_pixels 77.2738ms 75.8828ms 13.1782 Ops/s 13.1692 Ops/s $\color{#35bf28}+0.07\%$
test_async_pixels 0.1474s 70.6665ms 14.1510 Ops/s 14.3217 Ops/s $\color{#d91a1a}-1.19\%$
test_simple 0.7500s 0.7496s 1.3341 Ops/s 1.2551 Ops/s $\textbf{\color{#35bf28}+6.29\%}$
test_transformed 1.0789s 1.0044s 0.9956 Ops/s 0.9838 Ops/s $\color{#35bf28}+1.20\%$
test_serial 2.2400s 2.1642s 0.4621 Ops/s 0.4455 Ops/s $\color{#35bf28}+3.72\%$
test_parallel 1.9526s 1.9177s 0.5215 Ops/s 0.5247 Ops/s $\color{#d91a1a}-0.62\%$
test_step_mdp_speed[True-True-True-True-True] 0.1118ms 37.7325μs 26.5024 KOps/s 25.7914 KOps/s $\color{#35bf28}+2.76\%$
test_step_mdp_speed[True-True-True-True-False] 0.1152ms 21.0789μs 47.4408 KOps/s 46.8882 KOps/s $\color{#35bf28}+1.18\%$
test_step_mdp_speed[True-True-True-False-True] 57.0510μs 20.8477μs 47.9669 KOps/s 46.0218 KOps/s $\color{#35bf28}+4.23\%$
test_step_mdp_speed[True-True-True-False-False] 32.5010μs 11.9513μs 83.6731 KOps/s 82.7071 KOps/s $\color{#35bf28}+1.17\%$
test_step_mdp_speed[True-True-False-True-True] 67.5810μs 39.8318μs 25.1056 KOps/s 24.4457 KOps/s $\color{#35bf28}+2.70\%$
test_step_mdp_speed[True-True-False-True-False] 48.2010μs 23.3628μs 42.8031 KOps/s 42.4017 KOps/s $\color{#35bf28}+0.95\%$
test_step_mdp_speed[True-True-False-False-True] 53.4810μs 23.7422μs 42.1190 KOps/s 41.9582 KOps/s $\color{#35bf28}+0.38\%$
test_step_mdp_speed[True-True-False-False-False] 52.5710μs 14.2790μs 70.0330 KOps/s 69.3973 KOps/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[True-False-True-True-True] 70.5810μs 42.4285μs 23.5691 KOps/s 22.9329 KOps/s $\color{#35bf28}+2.77\%$
test_step_mdp_speed[True-False-True-True-False] 49.4300μs 25.8810μs 38.6384 KOps/s 38.5194 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[True-False-True-False-True] 47.9510μs 23.0654μs 43.3550 KOps/s 42.5724 KOps/s $\color{#35bf28}+1.84\%$
test_step_mdp_speed[True-False-True-False-False] 0.1882ms 14.2799μs 70.0285 KOps/s 69.8393 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[True-False-False-True-True] 73.6320μs 44.4324μs 22.5061 KOps/s 22.1795 KOps/s $\color{#35bf28}+1.47\%$
test_step_mdp_speed[True-False-False-True-False] 57.8010μs 27.9552μs 35.7716 KOps/s 35.7914 KOps/s $\color{#d91a1a}-0.06\%$
test_step_mdp_speed[True-False-False-False-True] 51.4910μs 25.7185μs 38.8824 KOps/s 38.7610 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[True-False-False-False-False] 0.1007ms 16.4895μs 60.6448 KOps/s 60.6045 KOps/s $\color{#35bf28}+0.07\%$
test_step_mdp_speed[False-True-True-True-True] 73.5410μs 42.0758μs 23.7666 KOps/s 23.5991 KOps/s $\color{#35bf28}+0.71\%$
test_step_mdp_speed[False-True-True-True-False] 0.2026ms 25.7082μs 38.8980 KOps/s 38.7209 KOps/s $\color{#35bf28}+0.46\%$
test_step_mdp_speed[False-True-True-False-True] 0.1228ms 28.1178μs 35.5647 KOps/s 36.0121 KOps/s $\color{#d91a1a}-1.24\%$
test_step_mdp_speed[False-True-True-False-False] 40.7900μs 16.3431μs 61.1878 KOps/s 60.9113 KOps/s $\color{#35bf28}+0.45\%$
test_step_mdp_speed[False-True-False-True-True] 73.9920μs 44.3411μs 22.5524 KOps/s 22.7994 KOps/s $\color{#d91a1a}-1.08\%$
test_step_mdp_speed[False-True-False-True-False] 58.7010μs 28.0795μs 35.6131 KOps/s 35.2614 KOps/s $\color{#35bf28}+1.00\%$
test_step_mdp_speed[False-True-False-False-True] 51.4410μs 30.2793μs 33.0259 KOps/s 33.1269 KOps/s $\color{#d91a1a}-0.30\%$
test_step_mdp_speed[False-True-False-False-False] 42.4600μs 18.3685μs 54.4410 KOps/s 53.4557 KOps/s $\color{#35bf28}+1.84\%$
test_step_mdp_speed[False-False-True-True-True] 4.0066ms 47.5480μs 21.0314 KOps/s 21.5504 KOps/s $\color{#d91a1a}-2.41\%$
test_step_mdp_speed[False-False-True-True-False] 46.9810μs 30.5007μs 32.7861 KOps/s 32.7280 KOps/s $\color{#35bf28}+0.18\%$
test_step_mdp_speed[False-False-True-False-True] 48.8510μs 30.3792μs 32.9173 KOps/s 33.3190 KOps/s $\color{#d91a1a}-1.21\%$
test_step_mdp_speed[False-False-True-False-False] 36.6300μs 18.4097μs 54.3193 KOps/s 53.7553 KOps/s $\color{#35bf28}+1.05\%$
test_step_mdp_speed[False-False-False-True-True] 67.6110μs 49.3872μs 20.2482 KOps/s 20.8142 KOps/s $\color{#d91a1a}-2.72\%$
test_step_mdp_speed[False-False-False-True-False] 56.5210μs 32.3818μs 30.8815 KOps/s 30.7614 KOps/s $\color{#35bf28}+0.39\%$
test_step_mdp_speed[False-False-False-False-True] 50.9210μs 32.2343μs 31.0228 KOps/s 31.3422 KOps/s $\color{#d91a1a}-1.02\%$
test_step_mdp_speed[False-False-False-False-False] 37.0110μs 20.5773μs 48.5971 KOps/s 48.3937 KOps/s $\color{#35bf28}+0.42\%$
test_values[generalized_advantage_estimate-True-True] 25.7371ms 24.2563ms 41.2264 Ops/s 42.7201 Ops/s $\color{#d91a1a}-3.50\%$
test_values[vec_generalized_advantage_estimate-True-True] 88.6644ms 2.6623ms 375.6190 Ops/s 368.5389 Ops/s $\color{#35bf28}+1.92\%$
test_values[td0_return_estimate-False-False] 89.0210μs 65.7662μs 15.2054 KOps/s 15.4122 KOps/s $\color{#d91a1a}-1.34\%$
test_values[td1_return_estimate-False-False] 58.1059ms 56.4223ms 17.7235 Ops/s 19.2551 Ops/s $\textbf{\color{#d91a1a}-7.95\%}$
test_values[vec_td1_return_estimate-False-False] 1.4499ms 1.0779ms 927.6944 Ops/s 935.5398 Ops/s $\color{#d91a1a}-0.84\%$
test_values[td_lambda_return_estimate-True-False] 92.3624ms 89.5108ms 11.1718 Ops/s 12.1114 Ops/s $\textbf{\color{#d91a1a}-7.76\%}$
test_values[vec_td_lambda_return_estimate-True-False] 1.4138ms 1.0705ms 934.1812 Ops/s 946.1193 Ops/s $\color{#d91a1a}-1.26\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 26.0344ms 25.6491ms 38.9877 Ops/s 41.3141 Ops/s $\textbf{\color{#d91a1a}-5.63\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9410ms 0.7223ms 1.3845 KOps/s 1.4398 KOps/s $\color{#d91a1a}-3.84\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8300ms 0.6556ms 1.5253 KOps/s 1.5467 KOps/s $\color{#d91a1a}-1.38\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6112ms 1.4576ms 686.0483 Ops/s 692.6227 Ops/s $\color{#d91a1a}-0.95\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8043ms 0.6677ms 1.4977 KOps/s 1.5062 KOps/s $\color{#d91a1a}-0.57\%$
test_dqn_speed 1.7976ms 1.3565ms 737.2094 Ops/s 730.4673 Ops/s $\color{#35bf28}+0.92\%$
test_ddpg_speed 3.0534ms 2.7512ms 363.4727 Ops/s 358.3719 Ops/s $\color{#35bf28}+1.42\%$
test_sac_speed 8.3056ms 7.9501ms 125.7851 Ops/s 126.3884 Ops/s $\color{#d91a1a}-0.48\%$
test_redq_speed 12.4424ms 10.4638ms 95.5672 Ops/s 96.3032 Ops/s $\color{#d91a1a}-0.76\%$
test_redq_deprec_speed 0.1064s 12.4618ms 80.2452 Ops/s 91.7464 Ops/s $\textbf{\color{#d91a1a}-12.54\%}$
test_td3_speed 8.2569ms 7.9211ms 126.2458 Ops/s 126.9315 Ops/s $\color{#d91a1a}-0.54\%$
test_cql_speed 26.9463ms 25.5280ms 39.1727 Ops/s 39.3316 Ops/s $\color{#d91a1a}-0.40\%$
test_a2c_speed 6.4623ms 5.7146ms 174.9912 Ops/s 180.2172 Ops/s $\color{#d91a1a}-2.90\%$
test_ppo_speed 6.1383ms 5.9613ms 167.7486 Ops/s 168.9371 Ops/s $\color{#d91a1a}-0.70\%$
test_reinforce_speed 4.8743ms 4.6032ms 217.2392 Ops/s 221.2330 Ops/s $\color{#d91a1a}-1.81\%$
test_iql_speed 20.3950ms 19.5801ms 51.0721 Ops/s 50.0384 Ops/s $\color{#35bf28}+2.07\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.8593ms 6.6680ms 149.9689 Ops/s 149.1871 Ops/s $\color{#35bf28}+0.52\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.8656ms 0.5226ms 1.9134 KOps/s 1.9061 KOps/s $\color{#35bf28}+0.38\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7160ms 0.5042ms 1.9832 KOps/s 1.9992 KOps/s $\color{#d91a1a}-0.80\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.9417ms 6.5983ms 151.5542 Ops/s 149.1465 Ops/s $\color{#35bf28}+1.61\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.1229ms 0.5159ms 1.9382 KOps/s 1.9451 KOps/s $\color{#d91a1a}-0.36\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6934ms 0.4962ms 2.0155 KOps/s 2.0278 KOps/s $\color{#d91a1a}-0.61\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.2768ms 1.9969ms 500.7813 Ops/s 506.1874 Ops/s $\color{#d91a1a}-1.07\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.3220ms 1.9056ms 524.7562 Ops/s 534.9428 Ops/s $\color{#d91a1a}-1.90\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.9329ms 6.7641ms 147.8397 Ops/s 146.7246 Ops/s $\color{#35bf28}+0.76\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8955ms 0.6685ms 1.4959 KOps/s 1.4994 KOps/s $\color{#d91a1a}-0.23\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.6298ms 0.6800ms 1.4706 KOps/s 1.5329 KOps/s $\color{#d91a1a}-4.06\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.9238ms 6.6638ms 150.0651 Ops/s 148.5269 Ops/s $\color{#35bf28}+1.04\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.0200ms 0.5228ms 1.9126 KOps/s 1.9219 KOps/s $\color{#d91a1a}-0.48\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7183ms 0.5038ms 1.9848 KOps/s 2.0014 KOps/s $\color{#d91a1a}-0.83\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.8840ms 6.5318ms 153.0979 Ops/s 151.5647 Ops/s $\color{#35bf28}+1.01\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.1213ms 0.5182ms 1.9299 KOps/s 1.9477 KOps/s $\color{#d91a1a}-0.91\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6756ms 0.4934ms 2.0269 KOps/s 2.0117 KOps/s $\color{#35bf28}+0.76\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.0011ms 6.8154ms 146.7276 Ops/s 146.8097 Ops/s $\color{#d91a1a}-0.06\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.7839ms 0.6673ms 1.4986 KOps/s 1.4742 KOps/s $\color{#35bf28}+1.66\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8578ms 0.6442ms 1.5523 KOps/s 1.5248 KOps/s $\color{#35bf28}+1.81\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1385s 7.8214ms 127.8538 Ops/s 94.9957 Ops/s $\textbf{\color{#35bf28}+34.59\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 18.9291ms 16.3553ms 61.1422 Ops/s 60.6712 Ops/s $\color{#35bf28}+0.78\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.8316ms 1.3494ms 741.0766 Ops/s 712.0128 Ops/s $\color{#35bf28}+4.08\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1354s 7.7818ms 128.5050 Ops/s 130.0201 Ops/s $\color{#d91a1a}-1.17\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 18.8381ms 16.3452ms 61.1801 Ops/s 61.6428 Ops/s $\color{#d91a1a}-0.75\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.3040ms 1.2636ms 791.3658 Ops/s 718.5229 Ops/s $\textbf{\color{#35bf28}+10.14\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1326s 10.4239ms 95.9334 Ops/s 126.0570 Ops/s $\textbf{\color{#d91a1a}-23.90\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.9822ms 16.4409ms 60.8239 Ops/s 61.0092 Ops/s $\color{#d91a1a}-0.30\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.5432ms 1.4608ms 684.5361 Ops/s 746.3161 Ops/s $\textbf{\color{#d91a1a}-8.28\%}$
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
2 participants
@vmoens @facebook-github-bot