Add library guide for table provider and catalog providers

[tshauck]

Which issue does this PR close?

Closes # Related to #7014

This won't finish #7014, but adds the skeleton described in that issue as well as content for the custom table provider and catalog. I can also add some info on Scalar UDFs, but am not sure I know enough about the others to write something w/o a summary to expand on.

Hope this is what you'all are looking for, certainly open to feedback.

Finally, I'm getting some unrelated (I think) warnings, are these known?

/Users/thauck/personal/code/github.com/tshauck/arrow-datafusion/docs/temp/user-guide/example-usage.md:24: WARNING: 'myst' cross-reference target not found: '../../../datafusion-examples' [myst.xref_missing]
/Users/thauck/personal/code/github.com/tshauck/arrow-datafusion/docs/temp/user-guide/example-usage.md:211: WARNING: Pygments lexer name 'rust,ignore' is not known
/Users/thauck/personal/code/github.com/tshauck/arrow-datafusion/docs/temp/user-guide/sql/data_types.md:31: WARNING: Could not lex literal_block '❯ select arrow_typeof(interval \'1 month\');\n+-------------------------------------+\n| arrowtypeof(IntervalYearMonth("1")) |\n+-------------------------------------+\n| Interval(YearMonth)                 |\n+-------------------------------------+\n' as "sql". Highlighting skipped.
...

Rationale for this change

It was requested in #7014. I also would've found this useful for exon.

Are these changes tested?

I locally generated the docs, per the README. Ex..

Are there any user-facing changes?

Yes, this would update the docs users could read.

[tshauck]

I think the current failure is related to #7286... I'll rebase and relook when it's merged.

[alamb]

Thank you @tshauck -- I will review this later today

[alamb]

Thank you @tshauck -- the library guide is awesome

I left some suggestions, but I also think this PR can be merged as is. Really 👏 thank you for starting this skeleton

cc @MrPowers -- I think this is very much in the spirit of what you and I spoke about earlier this summer for a guide.

[alamb]

I think as part of merging this PR I can file some subtasks to fill out these other sections.

[alamb]

I wonder if the docs/source/library-user-guide/execution-plans.md section should direct here as it has an example of execution plans 🤔

[tshauck]

execution-plans.md looks like this now:

[alamb]

Leaving links to the docs.rs versions might be more maintainable

[tshauck]

Apologies, not sure how to do this. I tried https://docs.rs/crate/datafusion/latest/source/, but the examples aren't there from what I can tell.

[tshauck]

Thanks for all the feedback! I'll work through it later this evening and follow up on anything.

[tshauck]

Thanks for the review, I think I got to the feedback. I can try to do the UDF page, but I don't have enough experience with the other areas to provide good info (though I'd like to read the one on using exprs 😄 ).

[liukun4515]

it's a good job and start the guide to user and teach user to use the datafusion in their project as a lib

LGTM

[alamb]

Great work -- thanks again @tshauck -- I will file some additional tickets for the "Coming Soon" sections as well

[alamb]

Follow on tickets filed: #7014 (comment)