Initial draft of policies and guidelines for libcudf usage.

[vyasr]

Description

This PR adds a section to the developer documentation about various libcudf design decisions that affect users. These policies are important for us to document and communicate consistently. I am not sure what the best place for this information is, but I think the developer docs are a good place to start since until we address #11481 we don't have a great way to publish any non-API user-facing libcudf documentation. I've created this draft PR to solicit feedback from other libcudf devs about other policies that we should be documenting in a similar manner. Once everyone is happy with the contents, I would suggest that we merge this into the dev docs for now and then revisit a better place once we've tackled #11481.

Partly addresses #5505, #1781.

Resolves #4511.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[hyperbolic2346]

Small clarification request.

[davidwendt]

This extends to output sizes and results as well.

In general there is no validation that an operation will produce a column that fits in the 2GB limit. There are a few minor exceptions where the validation does not require an extra kernel launch.

Also, operations that may produce overflow (integer or floating-point arithmetic) are also not checked. Overflow produces undefined results. For example, integer overflow to negative values is not guaranteed.

[vyasr]

My intent here was not to provide exhaustive lists of things we will and will not check, but rather to provide some examples of cases to illustrate the difference between introspective and non-introspective operations. I am wary of trying to maintain a complete list that will inevitably go out of date the day after it is published. Do you think it's worth trying to keep a larger list?

[davidwendt]

I suppose I would like to keep a list of things that keep being brought up (like these two here) so that I can point to this document to say this is by design.

[bdice]

I agree this does not need to be exhaustive, but it would be nice if this policy guide can serve as a mini-FAQ of "why is libcudf designed this way?" I think that the concrete examples @davidwendt mentioned help illustrate the philosophy of the library.

[vyasr]

I've added some, and I've reworded this section to reflect that we want to keep a list but make no promises that it's exhaustive.

[jrhemstad]

Happy to see us writing down some things that have been tribal knowledge for a while now.

[bdice]

Only minor comments. Thanks!

[vyasr]

@davidwendt if you are satisfied with how I've addressed your comments feel free to merge this once you approve.

[davidwendt]

@gpucibot merge