Test all examples from library-user-guide & user-guide docs

[ugoa]

Which issue does this PR close?

• Closes Run / Test all examples in Documentation #14435
• Closes Hide boilerplate in documentation examples #14557

Rationale for this change

What changes are included in this PR?

Library User Guide:

• docs/source/library-user-guide/adding-udfs.md
• docs/source/library-user-guide/api-health.md
• docs/source/library-user-guide/building-logical-plans.md
• docs/source/library-user-guide/catalogs.md
• docs/source/library-user-guide/custom-table-providers.md
• docs/source/library-user-guide/extending-operators.md
• docs/source/library-user-guide/extensions.md
• docs/source/library-user-guide/index.md
• docs/source/library-user-guide/profiling.md
• docs/source/library-user-guide/query-optimizer.md
• docs/source/library-user-guide/using-the-dataframe-api.md
• docs/source/library-user-guide/using-the-sql-api.md
• docs/source/library-user-guide/working-with-exprs.md

User Guide:

• docs/source/user-guide/example-usage.md
• docs/source/user-guide/faq.md
• docs/source/user-guide/introduction.md
• docs/source/user-guide/cli/index.rst
• docs/source/user-guide/cli/overview.md
• docs/source/user-guide/cli/datasources.md
• docs/source/user-guide/cli/usage.md
• docs/source/user-guide/cli/installation.md
• docs/source/user-guide/configs.md
• docs/source/user-guide/concepts-readings-events.md
• docs/source/user-guide/dataframe.md
• docs/source/user-guide/explain-usage.md
• docs/source/user-guide/expressions.md
• docs/source/user-guide/crate-configuration.md
• docs/source/user-guide/sql/sql_status.md
• docs/source/user-guide/sql/index.rst
• docs/source/user-guide/sql/scalar_functions.md
• docs/source/user-guide/sql/write_options.md
• docs/source/user-guide/sql/explain.md
• docs/source/user-guide/sql/information_schema.md
• docs/source/user-guide/sql/select.md
• docs/source/user-guide/sql/special_functions.md
• docs/source/user-guide/sql/dml.md
• docs/source/user-guide/sql/operators.md
• docs/source/user-guide/sql/ddl.md
• docs/source/user-guide/sql/aggregate_functions.md
• docs/source/user-guide/sql/data_types.md
• docs/source/user-guide/sql/subqueries.md
• docs/source/user-guide/sql/window_functions.md

Are these changes tested?

Yes

Are there any user-facing changes?

No

[alamb]

why do we need snmalloc?

[alamb]

Per the comment below I don't think we can use snmalloc in the dev dependencies without messing up the other builds -- can you please remove that (and maybe add a note to the example that tried to use it?)

[alamb]

Ah, I see the snmalloc comes from running

docs/source/user-guide/crate-configuration.md

I think the issue there is that snmalloc takes over the global allocator,

[alamb]

Thank you so much @ugoa -- this is looking really cool

Let's try and get this PR ready for merge -- it is already an amazing improvement. I'll try and fix up the tests and get it in later today. Then maybe we can make additional improvements as follow ons

[alamb]

Per the comment below I don't think we can use snmalloc in the dev dependencies without messing up the other builds -- can you please remove that (and maybe add a note to the example that tried to use it?)

[alamb]

nice!

[ugoa]

Thanks to DeepSeek generating this code snippet haha, too powerful

[alamb]

I also double checked the "hide lines starting with # and it looks great!"

[alamb]

I will push a few commits to fix this

[alamb]

Making the examples in the doc rendred with sql rather than ``` was a great idea. However, since those files are automatically generated from the source code we need to update the generation script, such as

datafusion/datafusion/core/src/bin/print_functions_docs.rs

Line 196 in f7a4909

```

And add the sql there (there are equivalents for the other types of functions)

Perhaps you can make a follow on PR?

[alamb]

I plan to merge this PR in once the tests pass

[ugoa]

Making the examples in the doc rendred with sql rather than ``` was a great idea. However, since those files are automatically generated from the source code we need to update the generation script

Ah I didn't know that, so I can generate the doc by run cargo run --bin print_functions_docs -- scalar right?

And add the sql there (there are equivalents for the other types of functions)
Perhaps you can make a follow on PR?

Sure I can open another PR to address this, cheers.

[alamb]

Making the examples in the doc rendred with sql rather than ``` was a great idea. However, since those files are automatically generated from the source code we need to update the generation script

Ah I didn't know that, so I can generate the doc by run cargo run --bin print_functions_docs -- scalar right?

The way I do it is

./dev/update_function_docs.sh

And add the sql there (there are equivalents for the other types of functions)
Perhaps you can make a follow on PR?

Sure I can open another PR to address this, cheers.

🙏 thank you so much

[alamb]

Ah, I see now the issue is that we'll try and test all examples: https://github.com/apache/datafusion/actions/runs/13241873169/job/36958947894?pr=14544

So they need to be marked as SQL to avoid CI failures. I'll look into it

[ugoa]

So they need to be marked as SQL to avoid CI failures. I'll look into it

Yeah, with just ``` it will be treated as rust code and be run and tested

[alamb]

Yeah, with just ``` it will be treated as rust code and be run and tested

I pushed a fix in e0b360c

Let's see if we can get a clean run

[alamb]

So close...

[alamb]

🚀

Thanks again @ugoa -- this is great work

FYI @tshauck who I think started this documentation many months ago

[ugoa]

My pleasure! I learned a lot during the time as well.

[ugoa]

Hey @alamb , btw we probably need to remove the installation of cmake in the .github/actions/setup-builder/action.yaml since it was added for snmalloc-rs = "0.3" which we don't need anymore

[alamb]

Hey @alamb , btw we probably need to remove the installation of cmake in the .github/actions/setup-builder/action.yaml since it was added for snmalloc-rs = "0.3" which we don't need anymore

Makes sense -- thank you @ugoa

Can you make a PR to do so?

[ugoa]

@alamb Sure, here it is #14606, please help review.