Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Point to the native CSV reader if encountering data frames read with readr #469

Merged
merged 2 commits into from
Jan 17, 2025
Merged

feat: Point to the native CSV reader if encountering data frames read with readr #469

merged 2 commits into from
Jan 17, 2025

Conversation

krlmlr
Copy link
Member

@krlmlr krlmlr commented Jan 17, 2025 •
edited
Loading

Closes #127.

@krlmlr krlmlr changed the title f 127 readr chore: Clean up source in error Jan 17, 2025
@krlmlr krlmlr enabled auto-merge (squash) January 17, 2025 16:05
@krlmlr krlmlr marked this pull request as draft January 17, 2025 16:06
auto-merge was automatically disabled January 17, 2025 16:06

Pull request was converted to draft

@krlmlr krlmlr marked this pull request as ready for review January 17, 2025 16:27
@github-actions GitHub Actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 2d1c20c is merged into main:

  • ✔️001_tpch_01: 24ms -> 24.2ms [-3.58%, +5.52%]
  • ✔️001_tpch_02: 67.5ms -> 67.5ms [-0.77%, +0.85%]
  • ✔️001_tpch_03: 38.9ms -> 39.4ms [-0.94%, +3.27%]
  • ✔️001_tpch_04: 22.2ms -> 22.2ms [-1.31%, +1.41%]
  • ✔️001_tpch_05: 57.1ms -> 57.1ms [-0.91%, +0.96%]
  • ✔️001_tpch_06: 14.2ms -> 14.2ms [-2.47%, +2.9%]
  • ✔️001_tpch_07: 74.4ms -> 74.7ms [-0.35%, +0.97%]
  • ✔️001_tpch_08: 97.7ms -> 98.5ms [-0.47%, +2.23%]
  • ✔️001_tpch_09: 72.6ms -> 72.6ms [-1.27%, +1.18%]
  • ✔️001_tpch_10: 48.4ms -> 48.3ms [-1.06%, +0.61%]
  • ✔️001_tpch_11: 32.2ms -> 32.3ms [-0.78%, +1.74%]
  • ✔️001_tpch_12: 27ms -> 27.3ms [-0.89%, +2.82%]
  • ✔️001_tpch_13: 25.1ms -> 24.9ms [-1.92%, +0.57%]
  • ✔️001_tpch_14: 20.2ms -> 20.2ms [-1.04%, +1.48%]
  • ✔️001_tpch_15: 31.4ms -> 31.6ms [-1.37%, +2.91%]
  • ✔️001_tpch_16: 40ms -> 39.7ms [-2.42%, +0.77%]
  • ✔️001_tpch_17: 25.9ms -> 26ms [-0.62%, +1.35%]
  • ✔️001_tpch_18: 22.8ms -> 22.6ms [-2.83%, +1%]
  • ✔️001_tpch_19: 66.7ms -> 66.5ms [-1.31%, +0.69%]
  • ✔️001_tpch_20: 50.4ms -> 50.7ms [-0.45%, +1.56%]
  • ✔️001_tpch_21: 79.6ms -> 79.4ms [-1.66%, +1.34%]
  • ✔️001_tpch_22: 66.8ms -> 66.9ms [-0.99%, +1.14%]
  • ✔️010_tpch_01: 81.6ms -> 80.9ms [-7.49%, +5.86%]
  • ✔️010_tpch_02: 70.5ms -> 70.2ms [-3.17%, +2.32%]
  • ✔️010_tpch_03: 58.9ms -> 59ms [-0.88%, +1.33%]
  • ✔️010_tpch_04: 43.4ms -> 43.7ms [-3.06%, +4.27%]
  • ✔️010_tpch_05: 89.9ms -> 89.9ms [-0.77%, +0.75%]
  • ✔️010_tpch_06: 32.1ms -> 33.2ms [-7.81%, +14.45%]
  • ✔️010_tpch_07: 106ms -> 106ms [-1.46%, +1.48%]
  • ✔️010_tpch_08: 129ms -> 128ms [-2.54%, +0.6%]
  • ✔️010_tpch_09: 119ms -> 115ms [-5.69%, +0.15%]
  • ✔️010_tpch_10: 74.9ms -> 74.8ms [-1.63%, +1.49%]
  • ✔️010_tpch_11: 38.6ms -> 38.2ms [-4.86%, +2.4%]
  • ✔️010_tpch_12: 58.3ms -> 57.3ms [-6.29%, +2.93%]
  • ✔️010_tpch_13: 52.7ms -> 53.4ms [-2.28%, +5.11%]
  • ✔️010_tpch_14: 38.3ms -> 37.6ms [-7.11%, +2.96%]
  • ✔️010_tpch_15: 54.1ms -> 56.1ms [-3.74%, +11.02%]
  • ✔️010_tpch_16: 44.4ms -> 43.9ms [-3.85%, +1.52%]
  • ✔️010_tpch_17: 56.1ms -> 55.1ms [-7.86%, +4.5%]
  • ✔️010_tpch_18: 52.8ms -> 53.7ms [-3.71%, +7.15%]
  • ✔️010_tpch_19: 115ms -> 117ms [-0.19%, +3.08%]
  • ✔️010_tpch_20: 66.7ms -> 66.8ms [-6.18%, +6.53%]
  • ✔️010_tpch_21: 238ms -> 237ms [-2.72%, +1.92%]
  • ✔️010_tpch_22: 76.4ms -> 76.6ms [-1.08%, +1.77%]
  • ✔️100_tpch_01: 322ms -> 339ms [-6.9%, +17.73%]
  • ✔️100_tpch_02: 127ms -> 132ms [-8.19%, +15.98%]
  • ✔️100_tpch_03: 178ms -> 180ms [-6.7%, +8.88%]
  • ✔️100_tpch_04: 156ms -> 152ms [-14.06%, +8.55%]
  • ✔️100_tpch_05: 267ms -> 273ms [-5.08%, +9.05%]
  • ✔️100_tpch_06: 105ms -> 109ms [-15.77%, +22.54%]
  • ✔️100_tpch_07: 223ms -> 234ms [-6.07%, +16.76%]
  • ✔️100_tpch_08: 261ms -> 266ms [-4.09%, +7.99%]
  • ✔️100_tpch_09: 344ms -> 353ms [-13.46%, +18.56%]
  • ✔️100_tpch_10: 215ms -> 217ms [-4.99%, +6.29%]
  • ✔️100_tpch_11: 94.3ms -> 83.6ms [-41.22%, +18.62%]
  • ✔️100_tpch_12: 192ms -> 186ms [-7.39%, +1.93%]
  • ✔️100_tpch_13: 312ms -> 307ms [-3.71%, +0.29%]
  • ✔️100_tpch_14: 122ms -> 129ms [-7.69%, +19.52%]
  • ✔️100_tpch_15: 217ms -> 214ms [-9.46%, +7.21%]
  • ✔️100_tpch_16: 127ms -> 123ms [-10.56%, +4.03%]
  • ✔️100_tpch_17: 187ms -> 171ms [-21.89%, +5.66%]
  • ✔️100_tpch_18: 195ms -> 199ms [-4.1%, +7.84%]
  • ✔️100_tpch_19: 296ms -> 304ms [-9.09%, +14.17%]
  • ✔️100_tpch_20: 184ms -> 175ms [-14.75%, +6%]
  • ✔️100_tpch_21: 1.31s -> 1.32s [-3.55%, +5.37%]
  • ✔️100_tpch_22: 182ms -> 171ms [-22.88%, +10.45%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@krlmlr krlmlr changed the title chore: Clean up source in error feat: Point to the native CSV reader if encountering data frames read with readr Jan 17, 2025
@krlmlr krlmlr requested a review from hadley January 17, 2025 19:01
@krlmlr krlmlr merged commit 3121fb9 into main Jan 17, 2025
21 checks passed
@krlmlr krlmlr deleted the f-127-readr branch January 17, 2025 19:32
@github-actions GitHub Actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 44f9dc1 is merged into main:

  • ✔️001_tpch_01: 24.1ms -> 23.6ms [-6.13%, +2.17%]
  • ✔️001_tpch_02: 68.9ms -> 68.9ms [-1.21%, +1.21%]
  • ✔️001_tpch_03: 39.9ms -> 40ms [-1.63%, +1.75%]
  • ✔️001_tpch_04: 22.2ms -> 22.3ms [-0.56%, +1.65%]
  • ✔️001_tpch_05: 58ms -> 58.2ms [-0.77%, +1.42%]
  • ✔️001_tpch_06: 14.3ms -> 14.4ms [-3.65%, +4.68%]
  • ✔️001_tpch_07: 76.4ms -> 76.9ms [-1.03%, +2.33%]
  • ✔️001_tpch_08: 101ms -> 99.4ms [-2.85%, +0.65%]
  • ✔️001_tpch_09: 75.2ms -> 74.9ms [-1.75%, +1.09%]
  • ✔️001_tpch_10: 49.6ms -> 49.5ms [-1.54%, +1.05%]
  • ✔️001_tpch_11: 32.3ms -> 32.2ms [-2%, +1.5%]
  • ✔️001_tpch_12: 27.3ms -> 27.5ms [-0.39%, +2.3%]
  • ✔️001_tpch_13: 26.2ms -> 26.2ms [-3.24%, +2.86%]
  • ✔️001_tpch_14: 20.5ms -> 20.6ms [-1.61%, +2.71%]
  • ✔️001_tpch_15: 31.8ms -> 31.7ms [-2.36%, +1.72%]
  • ✔️001_tpch_16: 40.6ms -> 40.3ms [-2.2%, +0.55%]
  • ✔️001_tpch_17: 26.4ms -> 26.2ms [-2.1%, +0.73%]
  • ✔️001_tpch_18: 22.7ms -> 22.5ms [-3.48%, +1.59%]
  • ✔️001_tpch_19: 68.8ms -> 68.1ms [-2.26%, +0.31%]
  • ✔️001_tpch_20: 51.2ms -> 51.7ms [-0.85%, +2.54%]
  • ✔️001_tpch_21: 80.1ms -> 81.1ms [-0.26%, +2.94%]
  • ✔️001_tpch_22: 68.2ms -> 68.1ms [-1.08%, +0.93%]
  • ✔️010_tpch_01: 80.2ms -> 80ms [-3.49%, +3.01%]
  • ✔️010_tpch_02: 70.1ms -> 68.9ms [-3.76%, +0.38%]
  • ✔️010_tpch_03: 58.7ms -> 59ms [-0.46%, +1.22%]
  • ✔️010_tpch_04: 42.6ms -> 43ms [-0.47%, +2.37%]
  • ✔️010_tpch_05: 90.9ms -> 93.7ms [-0.6%, +6.7%]
  • 🚀010_tpch_06: 33.7ms -> 31.4ms [-12.64%, -0.95%]
  • ✔️010_tpch_07: 106ms -> 106ms [-1.14%, +1.1%]
  • ✔️010_tpch_08: 128ms -> 127ms [-1.75%, +0.52%]
  • ✔️010_tpch_09: 117ms -> 117ms [-1.11%, +1.42%]
  • ✔️010_tpch_10: 75.8ms -> 75.8ms [-2.64%, +2.72%]
  • ✔️010_tpch_11: 38.6ms -> 38.8ms [-3.4%, +4.37%]
  • ✔️010_tpch_12: 58.8ms -> 58.6ms [-6.2%, +5.73%]
  • ✔️010_tpch_13: 52.4ms -> 53.2ms [-0.45%, +3.44%]
  • ✔️010_tpch_14: 37.4ms -> 37.4ms [-1.25%, +1.23%]
  • ✔️010_tpch_15: 52.2ms -> 55ms [-2.5%, +13.03%]
  • ✔️010_tpch_16: 44.6ms -> 44.1ms [-3.31%, +1.26%]
  • ✔️010_tpch_17: 54.8ms -> 55.2ms [-4.23%, +5.71%]
  • ✔️010_tpch_18: 53.6ms -> 53.9ms [-8.2%, +9.25%]
  • ✔️010_tpch_19: 116ms -> 117ms [-0.55%, +2.51%]
  • ✔️010_tpch_20: 66.6ms -> 66.2ms [-4.38%, +3.1%]
  • ✔️010_tpch_21: 238ms -> 240ms [-2.06%, +3.36%]
  • ✔️010_tpch_22: 76.1ms -> 76.2ms [-1.94%, +2.05%]
  • ❗🐌100_tpch_01: 291ms -> 338ms [+8.35%, +23.53%]
  • ✔️100_tpch_02: 131ms -> 140ms [-13.35%, +26.86%]
  • ✔️100_tpch_03: 175ms -> 179ms [-7.59%, +12.44%]
  • ✔️100_tpch_04: 156ms -> 155ms [-4.76%, +3.2%]
  • ✔️100_tpch_05: 264ms -> 267ms [-8.98%, +10.68%]
  • ✔️100_tpch_06: 106ms -> 104ms [-18.52%, +14.8%]
  • ✔️100_tpch_07: 234ms -> 240ms [-5.67%, +10.33%]
  • ✔️100_tpch_08: 260ms -> 255ms [-14.92%, +11.19%]
  • ✔️100_tpch_09: 340ms -> 335ms [-12.93%, +10.15%]
  • ✔️100_tpch_10: 213ms -> 212ms [-6.27%, +5.02%]
  • ✔️100_tpch_11: 82.4ms -> 82.1ms [-1.1%, +0.24%]
  • ✔️100_tpch_12: 199ms -> 199ms [-9.82%, +9.28%]
  • ✔️100_tpch_13: 335ms -> 311ms [-26.42%, +11.9%]
  • ✔️100_tpch_14: 121ms -> 122ms [-18.17%, +20.46%]
  • ✔️100_tpch_15: 220ms -> 222ms [-9.71%, +11.39%]
  • ✔️100_tpch_16: 126ms -> 125ms [-9.54%, +7.75%]
  • ✔️100_tpch_17: 179ms -> 180ms [-9.41%, +9.52%]
  • ✔️100_tpch_18: 199ms -> 197ms [-17.08%, +15.73%]
  • ✔️100_tpch_19: 310ms -> 295ms [-21.35%, +11.44%]
  • ❗🐌100_tpch_20: 170ms -> 175ms [+0.77%, +5.88%]
  • ❗🐌100_tpch_21: 1.31s -> 1.38s [+0.9%, +9.26%]
  • ✔️100_tpch_22: 170ms -> 170ms [-4.07%, +4.68%]

Further explanation regarding interpretation and methodology can be found in the documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hadley hadley Awaiting requested review from hadley

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Should as_duckplyr_df() work with tibbles from readr::read_csv()?
1 participant
@krlmlr