feat: Implement na.rm handling for sum(), min(), max(), any() and all(), with fallback for window functions

[krlmlr]

Closes #205.

[github-actions]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if cb4cd4f is merged into main:

• ❗🐌001_tpch_01: 31.2ms -> 1.31s [+4005.45%, +4202.07%]
• ❗🐌001_tpch_02: 72.7ms -> 272ms [+251.54%, +296.22%]
• ❗🐌001_tpch_03: 34.5ms -> 193ms [+444%, +475.81%]
• ✔️001_tpch_04: 30.3ms -> 30.4ms [-2.25%, +3.28%]
• ❗🐌001_tpch_05: 59.4ms -> 207ms [+242.76%, +253.2%]
• ❗🐌001_tpch_06: 12.1ms -> 164ms [+1226.53%, +1275.32%]
• ❗🐌001_tpch_07: 83.7ms -> 250ms [+185.4%, +211.01%]
• ❗🐌001_tpch_08: 110ms -> 323ms [+161.78%, +226.87%]
• ❗🐌001_tpch_09: 69.6ms -> 215ms [+203.23%, +213.27%]
• ❗🐌001_tpch_10: 50.8ms -> 204ms [+291.68%, +313.55%]
• ❗🐌001_tpch_11: 44.2ms -> 482ms [+920.03%, +1061.7%]
• ❗🐌001_tpch_12: 30.2ms -> 420ms [+1228.02%, +1352.32%]
• ❗🐌001_tpch_13: 19.1ms -> 387ms [+1882.89%, +1971.64%]
• ❗🐌001_tpch_14: 17.9ms -> 225ms [+1102.16%, +1207.43%]
• ❗🐌001_tpch_15: 37.6ms -> 348ms [+804.02%, +848.16%]
• ✔️001_tpch_16: 37.1ms -> 37.7ms [-2.22%, +5.33%]
• ❗🐌001_tpch_17: 35.6ms -> 239ms [+531.69%, +612.53%]
• ❗🐌001_tpch_18: 32.8ms -> 249ms [+621.89%, +695.63%]
• ❗🐌001_tpch_19: 41.1ms -> 240ms [+454.69%, +512.49%]
• ❗🐌001_tpch_20: 55.5ms -> 249ms [+328.25%, +368.56%]
• ❗🐌001_tpch_21: 91.3ms -> 1.4s [+1379.69%, +1493.85%]
• ❗🐌001_tpch_22: 44.2ms -> 505ms [+930.99%, +1155.51%]
• ❗🐌010_tpch_01: 84.7ms -> 4.21s [+4183.58%, +5571.87%]
• ❗🐌010_tpch_02: 75.5ms -> 529ms [+547.68%, +653.99%]
• ❗🐌010_tpch_03: 60.5ms -> 511ms [+683.66%, +804.02%]
• ✔️010_tpch_04: 44.2ms -> 45.1ms [-2.13%, +6.27%]
• ❗🐌010_tpch_05: 91.7ms -> 469ms [+395.8%, +426.97%]
• ❗🐌010_tpch_06: 30.3ms -> 413ms [+1159.83%, +1365.87%]
• ❗🐌010_tpch_07: 103ms -> 431ms [+304.65%, +334.52%]
• ❗🐌010_tpch_08: 132ms -> 469ms [+237.18%, +275.38%]
• ❗🐌010_tpch_09: 119ms -> 508ms [+308.7%, +345.73%]
• ❗🐌010_tpch_10: 78.2ms -> 483ms [+467.77%, +568.07%]
• ❗🐌010_tpch_11: 39.2ms -> 839ms [+1900.03%, +2184.04%]
• ❗🐌010_tpch_12: 55.5ms -> 998ms [+1544.52%, +1850.49%]
• ❗🐌010_tpch_13: 53.2ms -> 2.41s [+4361.44%, +4516.19%]
• ❗🐌010_tpch_14: 34.5ms -> 470ms [+1205.72%, +1319.3%]
• ❗🐌010_tpch_15: 56.2ms -> 823ms [+1331.73%, +1398.33%]
• ✔️010_tpch_16: 41.7ms -> 41.4ms [-6.66%, +5.21%]
• ❗🐌010_tpch_17: 53.4ms -> 450ms [+725.93%, +758.35%]
• ❗🐌010_tpch_18: 53.2ms -> 818ms [+1397.68%, +1480.96%]
• ❗🐌010_tpch_19: 83.5ms -> 465ms [+445.38%, +469.37%]
• ❗🐌010_tpch_20: 64.8ms -> 450ms [+572.46%, +616.88%]
• ❗🐌010_tpch_21: 244ms -> 9.9s [+3872.63%, +4040.6%]
• ❗🐌010_tpch_22: 53.8ms -> 734ms [+1230.7%, +1297.63%]
• ❗🐌100_tpch_01: 323ms -> 16s [+4303.26%, +5395.59%]
• ❗🐌100_tpch_02: 127ms -> 1.83s [+1280.4%, +1409.37%]
• ❗🐌100_tpch_03: 168ms -> 1.93s [+965.89%, +1121.1%]
• ✔️100_tpch_04: 146ms -> 147ms [-10.73%, +11.92%]
• ❗🐌100_tpch_05: 272ms -> 2.1s [+645.6%, +694.91%]
• ❗🐌100_tpch_06: 95.9ms -> 2.06s [+1974.32%, +2132.87%]
• ❗🐌100_tpch_07: 224ms -> 2.25s [+816.01%, +990.92%]
• ❗🐌100_tpch_08: 273ms -> 2.47s [+708.18%, +907.04%]
• ❗🐌100_tpch_09: 367ms -> 2.38s [+443.92%, +656.22%]
• ❗🐌100_tpch_10: 220ms -> 2.09s [+776.46%, +923.49%]
• ❗🐌100_tpch_11: 98.9ms -> 4.23s [+3968.1%, +4374.11%]
• ❗🐌100_tpch_12: 178ms -> 4.63s [+2463.78%, +2535.01%]
• ❗🐌100_tpch_13: 347ms -> 26.9s [+7309.35%, +8000.18%]
• ❗🐌100_tpch_14: 115ms -> 2.41s [+1926.87%, +2068.36%]
• ❗🐌100_tpch_15: 201ms -> 5.1s [+2147.68%, +2728.28%]
• 🚀100_tpch_16: 127ms -> 119ms [-10.64%, -1.9%]
• ❗🐌100_tpch_17: 173ms -> 2.38s [+1222.36%, +1332.01%]
• ❗🐌100_tpch_18: 199ms -> 6.83s [+3018.8%, +3627.81%]
• ❗🐌100_tpch_19: 277ms -> 2.36s [+702.59%, +802.08%]
• ❗🐌100_tpch_20: 190ms -> 3.29s [+598.85%, +2669.26%]
• ❗🐌100_tpch_21: 1.44s -> 1.69m [+6748.48%, +7134.87%]
• ❗🐌100_tpch_22: 157ms -> 4.08s [+2415.17%, +2568.61%]

Further explanation regarding interpretation and methodology can be found in the documentation.

[krlmlr]

⚠️⚠️⚠️

Await benchmark results.

⚠️⚠️⚠️

[github-actions]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if aa4f841 is merged into main:

• ❗🐌001_tpch_01: 21.9ms -> 25.3ms [+10.59%, +20.43%]
• ❗🐌001_tpch_02: 63.1ms -> 63.7ms [+0.38%, +1.38%]
• ✔️001_tpch_03: 39.5ms -> 39.6ms [-0.94%, +1.4%]
• ❗🐌001_tpch_04: 22.1ms -> 27.9ms [+24.06%, +28.77%]
• ✔️001_tpch_05: 57.2ms -> 57ms [-0.99%, +0.32%]
• ❗🐌001_tpch_06: 11.3ms -> 11.7ms [+0.48%, +5.56%]
• ❗🐌001_tpch_07: 70.9ms -> 76.3ms [+6.98%, +8.3%]
• ❗🐌001_tpch_08: 96.8ms -> 98.5ms [+0.93%, +2.47%]
• ✔️001_tpch_09: 67.7ms -> 68ms [-0.04%, +0.74%]
• ❗🐌001_tpch_10: 42.2ms -> 48.1ms [+13.41%, +14.88%]
• ❗🐌001_tpch_11: 37.8ms -> 39.8ms [+4.21%, +6.48%]
• ❗🐌001_tpch_12: 20.5ms -> 22.5ms [+8.15%, +11.67%]
• ❗🐌001_tpch_13: 18.1ms -> 18.9ms [+1.64%, +6.72%]
• ❗🐌001_tpch_14: 16.1ms -> 19.7ms [+20.24%, +23.32%]
• ❗🐌001_tpch_15: 35.4ms -> 38.9ms [+9.4%, +10.88%]
• ✔️001_tpch_16: 34.5ms -> 34.4ms [-1.24%, +0.64%]
• ❗🐌001_tpch_17: 30.4ms -> 30.8ms [+0.18%, +3.04%]
• ❗🐌001_tpch_18: 22ms -> 29.8ms [+33.37%, +38.51%]
• 🚀001_tpch_19: 44ms -> 38ms [-14.84%, -12.48%]
• ❗🐌001_tpch_20: 49.8ms -> 50.8ms [+0.8%, +3.09%]
• ❗🐌001_tpch_21: 71.1ms -> 81.8ms [+12.68%, +17.35%]
• ❗🐌001_tpch_22: 45.3ms -> 45.6ms [+0.01%, +1.41%]
• ❗🐌010_tpch_01: 80.7ms -> 92.1ms [+7.63%, +20.65%]
• ✔️010_tpch_02: 69.1ms -> 70.3ms [-0.08%, +3.51%]
• ✔️010_tpch_03: 56.8ms -> 58.4ms [-0.18%, +5.72%]
• ✔️010_tpch_04: 41.5ms -> 42.4ms [-1.27%, +5.38%]
• ✔️010_tpch_05: 87.4ms -> 87.9ms [-1.28%, +2.62%]
• ❗🐌010_tpch_06: 28.3ms -> 30.2ms [+0.39%, +12.49%]
• ✔️010_tpch_07: 101ms -> 102ms [-1.31%, +3.19%]
• ❗🐌010_tpch_08: 125ms -> 129ms [+0.72%, +5.1%]
• ✔️010_tpch_09: 115ms -> 116ms [-1.16%, +3.72%]
• ✔️010_tpch_10: 73.9ms -> 74.8ms [-1.37%, +3.83%]
• ❗🐌010_tpch_11: 37.2ms -> 39ms [+1.71%, +7.72%]
• ✔️010_tpch_12: 51.3ms -> 52.2ms [-2.86%, +6.49%]
• ✔️010_tpch_13: 51.7ms -> 54.9ms [-0.46%, +12.93%]
• ❗🐌010_tpch_14: 34ms -> 37.5ms [+5.15%, +15.24%]
• ❗🐌010_tpch_15: 52.9ms -> 56.7ms [+1.54%, +12.67%]
• ✔️010_tpch_16: 38.7ms -> 39ms [-4.51%, +6.01%]
• ✔️010_tpch_17: 53.3ms -> 52.2ms [-7.52%, +3.47%]
• ❗🐌010_tpch_18: 52.2ms -> 58.9ms [+4.84%, +20.62%]
• ✔️010_tpch_19: 81.9ms -> 83.8ms [-0.65%, +5.38%]
• ✔️010_tpch_20: 62.9ms -> 65ms [-1.53%, +7.93%]
• ❗🐌010_tpch_21: 233ms -> 258ms [+8.08%, +13.71%]
• ❗🐌010_tpch_22: 50.7ms -> 51.9ms [+0.04%, +4.81%]
• ✔️100_tpch_01: 304ms -> 381ms [-1.53%, +52.32%]
• ✔️100_tpch_02: 130ms -> 128ms [-21.87%, +19.3%]
• ✔️100_tpch_03: 176ms -> 171ms [-9.08%, +3.74%]
• ✔️100_tpch_04: 154ms -> 153ms [-8.61%, +7.41%]
• ✔️100_tpch_05: 258ms -> 263ms [-12.72%, +17.02%]
• ✔️100_tpch_06: 99.4ms -> 92.9ms [-17.42%, +4.28%]
• ✔️100_tpch_07: 230ms -> 219ms [-10.12%, +0.49%]
• ✔️100_tpch_08: 257ms -> 257ms [-7.34%, +7.42%]
• ✔️100_tpch_09: 332ms -> 345ms [-6.27%, +14.32%]
• ✔️100_tpch_10: 204ms -> 213ms [-1.79%, +9.79%]
• ✔️100_tpch_11: 87.7ms -> 83.9ms [-24.55%, +15.93%]
• ✔️100_tpch_12: 187ms -> 192ms [-12.69%, +18.17%]
• ❗🐌100_tpch_13: 312ms -> 350ms [+8.51%, +15.95%]
• ✔️100_tpch_14: 119ms -> 122ms [-8.22%, +13.01%]
• ✔️100_tpch_15: 203ms -> 205ms [-8.78%, +10.64%]
• ✔️100_tpch_16: 129ms -> 133ms [-11.57%, +17.47%]
• ✔️100_tpch_17: 182ms -> 184ms [-5.83%, +8.22%]
• ❗🐌100_tpch_18: 194ms -> 231ms [+8.42%, +30.22%]
• ✔️100_tpch_19: 270ms -> 282ms [-5.91%, +14.51%]
• ✔️100_tpch_20: 175ms -> 168ms [-13.07%, +5.64%]
• ✔️100_tpch_21: 1.34s -> 1.41s [-4.38%, +13.43%]
• ✔️100_tpch_22: 152ms -> 147ms [-10.39%, +4.68%]

Further explanation regarding interpretation and methodology can be found in the documentation.

[krlmlr]

Review benchmarks after merging.

[github-actions]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 57ed3c8 is merged into main:

• ❗🐌001_tpch_01: 22.3ms -> 24.6ms [+5.41%, +15.05%]
• ❗🐌001_tpch_02: 62ms -> 62.4ms [+0.28%, +1.25%]
• ❗🐌001_tpch_03: 38.4ms -> 38.7ms [+0.17%, +1.8%]
• ❗🐌001_tpch_04: 21.8ms -> 27.3ms [+23.9%, +26.43%]
• ✔️001_tpch_05: 55.9ms -> 56.1ms [-0.62%, +0.99%]
• ✔️001_tpch_06: 11.4ms -> 11.6ms [-1.14%, +4.04%]
• ❗🐌001_tpch_07: 69.5ms -> 74.6ms [+6.54%, +8.11%]
• ✔️001_tpch_08: 95.1ms -> 96ms [0%, +2%]
• ✔️001_tpch_09: 67.2ms -> 66.8ms [-1.14%, +0.01%]
• ❗🐌001_tpch_10: 41.9ms -> 47.1ms [+11.63%, +13.2%]
• ❗🐌001_tpch_11: 37.2ms -> 38ms [+1.15%, +3.52%]
• ❗🐌001_tpch_12: 20.3ms -> 27.1ms [+30.5%, +36.91%]
• ❗🐌001_tpch_13: 17.6ms -> 18.4ms [+1.62%, +7.09%]
• ❗🐌001_tpch_14: 16.5ms -> 18.2ms [+6.4%, +13.64%]
• ❗🐌001_tpch_15: 35ms -> 37.3ms [+5.73%, +7.82%]
• ✔️001_tpch_16: 33.9ms -> 33.9ms [-0.94%, +1.21%]
• ❗🐌001_tpch_17: 29.4ms -> 30.5ms [+2.2%, +5.36%]
• ❗🐌001_tpch_18: 21.8ms -> 27.9ms [+26.51%, +30.08%]
• 🚀001_tpch_19: 44ms -> 37.8ms [-17.46%, -10.91%]
• ✔️001_tpch_20: 49ms -> 49.8ms [0%, +2.96%]
• ❗🐌001_tpch_21: 70.8ms -> 79.7ms [+11.51%, +13.57%]
• 🚀001_tpch_22: 44.6ms -> 39.9ms [-11.13%, -9.54%]
• ❗🐌010_tpch_01: 76.4ms -> 91.6ms [+12.97%, +26.65%]
• ✔️010_tpch_02: 69.1ms -> 70.2ms [-0.21%, +3.26%]
• ❗🐌010_tpch_03: 55.8ms -> 57.6ms [+0.73%, +5.63%]
• ✔️010_tpch_04: 41.8ms -> 41.7ms [-4.06%, +3.81%]
• ❗🐌010_tpch_05: 86.1ms -> 88ms [+1.39%, +2.98%]
• ✔️010_tpch_06: 28.5ms -> 30.1ms [-2.67%, +14.41%]
• ✔️010_tpch_07: 99.7ms -> 100ms [-0.87%, +1.48%]
• ✔️010_tpch_08: 125ms -> 128ms [-0.94%, +5.66%]
• ❗🐌010_tpch_09: 113ms -> 115ms [+0.41%, +2.82%]
• ❗🐌010_tpch_10: 71.3ms -> 73.8ms [+1.37%, +5.82%]
• ❗🐌010_tpch_11: 36.7ms -> 39ms [+3.6%, +9.26%]
• ❗🐌010_tpch_12: 49.7ms -> 51.8ms [+1.09%, +7.41%]
• ✔️010_tpch_13: 50.5ms -> 51.6ms [-1.58%, +6.05%]
• ❗🐌010_tpch_14: 32.5ms -> 34.8ms [+5.13%, +8.69%]
• ✔️010_tpch_15: 52.8ms -> 54.2ms [-3.47%, +9.08%]
• ✔️010_tpch_16: 37.9ms -> 38.6ms [-2.48%, +6.27%]
• ✔️010_tpch_17: 51.2ms -> 52.7ms [-1.39%, +7.48%]
• ✔️010_tpch_18: 54.4ms -> 57.9ms [-1.96%, +15.04%]
• ❗🐌010_tpch_19: 81.4ms -> 84.4ms [+1.03%, +6.26%]
• ✔️010_tpch_20: 62.6ms -> 63.7ms [-1.76%, +5.32%]
• ❗🐌010_tpch_21: 233ms -> 248ms [+4.4%, +9.08%]
• ❗🐌010_tpch_22: 51.1ms -> 53ms [+1.46%, +6.12%]
• ❗🐌100_tpch_01: 281ms -> 358ms [+6.67%, +48.05%]
• ✔️100_tpch_02: 128ms -> 132ms [-13.2%, +18.8%]
• ✔️100_tpch_03: 171ms -> 172ms [-3.27%, +4.26%]
• ✔️100_tpch_04: 157ms -> 153ms [-18.76%, +13.52%]
• ✔️100_tpch_05: 276ms -> 283ms [-1.79%, +7.06%]
• ✔️100_tpch_06: 94.2ms -> 92.3ms [-19.47%, +15.45%]
• ✔️100_tpch_07: 236ms -> 222ms [-20.12%, +8.18%]
• ✔️100_tpch_08: 255ms -> 249ms [-13.49%, +9.05%]
• ❗🐌100_tpch_09: 320ms -> 350ms [+5.71%, +12.83%]
• ✔️100_tpch_10: 204ms -> 210ms [-2.1%, +7.45%]
• ✔️100_tpch_11: 84.4ms -> 83.5ms [-12.21%, +10.17%]
• ✔️100_tpch_12: 181ms -> 189ms [-7%, +15.89%]
• ✔️100_tpch_13: 333ms -> 346ms [-2.37%, +10.35%]
• ✔️100_tpch_14: 119ms -> 114ms [-17.02%, +8.78%]
• ✔️100_tpch_15: 208ms -> 203ms [-10.35%, +5.33%]
• ✔️100_tpch_16: 120ms -> 120ms [-4.9%, +5.21%]
• ✔️100_tpch_17: 176ms -> 180ms [-7.29%, +12.4%]
• ❗🐌100_tpch_18: 192ms -> 205ms [+0.47%, +12.34%]
• ✔️100_tpch_19: 255ms -> 272ms [-11.45%, +25.25%]
• ✔️100_tpch_20: 170ms -> 165ms [-10.99%, +5.36%]
• ❗🐌100_tpch_21: 1.29s -> 1.38s [+1.47%, +12.82%]
• ✔️100_tpch_22: 151ms -> 150ms [-8.86%, +7.8%]

Further explanation regarding interpretation and methodology can be found in the documentation.

[krlmlr]

@hannes: This is the first time when benchmarks are getting slower. This PR turns SUM(x) into CASE WHEN COUNT(x) = 0 THEN 0 ELSE SUM(x), I wonder if this is something that DuckDB could optimize:

https://github.com/tidyverse/duckplyr/pull/566/files#diff-b04d9d86f52094f1fe4afa2e16c116e3dfda7a17f43c34ea854b716eb3953bafR63