[DOC] Improve Contributing guide

[jrhemstad]

The current Contributing Guide is a mishmash of instructions for C++/Python and ancillary libraries.

Furthermore, it doesn't even link to the libcudf Developer Guide.

I think we should have clearer sections broken up by Python/C++/Java/etc. Make sure these include all relevant references to developer guides, documentation, and any other links that would be helpful to first time developers and contributors (rapids-compose?).

[jrhemstad]

@shwina weren't you working on a Python developer guide?

[shwina]

cc: @vyasr, who has been working on the dev guide for Python.

[vyasr]

There is further discussion of this on #6481. The current draft of the Python developer guide is in an evolving Google document. Additionally, some information for developers is contained in the cudf documentation, with more cudf-specific information here. I agree that we could use some more structure for these guides.

My suggestion would be to break up guides not only by different components (libcudf, JNI, Python) as you suggested, but also by different topics: setup and installation, PR-related information (e.g. gpuCI, label checker, etc), style guide, developer guide. The reason is that some of these are shared across components, while others are very different for different components. We should also centralize on a location for this documentation. Some very high-level information like gpuCI info makes sense to be shared across all of RAPIDS, so it can stay in the overall RAPIDS docs repo and is out of scope for this issue. For the rest, I would have a slight preference for putting all of it in the built Sphinx documentation, generalizing it to include sections for the different components. Doing that would make it easier to link between different documents and different components, and it would also make it easier to find for external contributors via the official cudf documentation page.

[shwina]

Based on Vyas' suggestions, I'm envisioning the following:

• We flesh out the Python documentation (Sphinx) further, to include setup/installation, as well as a developer/style/contributing guide(s).

• The C++ documentation (Doxygen) can evolve independently, with Python docs linking to C++ where needed.

• The Contributing guide (GitHub) can be more of a landing page linking out to the C++/Python doc pages as required.

[vyasr]

Yes, I think that sounds right. The contributing guide should be the point of entry to the documentation. The one difference that I would suggest (in line with making it the entry point) is putting the contributing guide in the Sphinx docs where it is more visible. Experienced members of the cudf team (libcudf/cudf python) will know where to look in the repo to find the component-specific documentation, but newer members or external contributors will not and having a centralized, easy-to-access starting point would help. Both C++ and Python have both API documentation (Sphinx for Python, doxygen for C++) that lives with the code as well as higher-level design documentation in separate files (currently Sphinx for Python and Markdown files for C++). I don't know what the Java team does, but I would recommend migrating the Markdown libcudf developer guide into our Sphinx rst so that it is also accessible from the same place as the high-level Python dev docs (what I'm working on in the Google doc). The API docs can remain as they are, although at some point it might be nice for the Sphinx docs to link to the libcudf doxygen in the same way that we link to the inline Python API docs.

[shwina]

but I would recommend migrating the Markdown libcudf developer guide into our Sphinx rst so that it is also accessible from the same place as the high-level Python dev docs

This could be somewhat controversial; Sphinx is more the tool of choice for Python documentation generation. But I defer to @jrhemstad here.

[vyasr]

That's a fair point. I'm less concerned with a particular choice of format etc and more concerned about the putting the documentation somewhere more visible and that has support for more formatting than Github's Markdown viewer. Linking to the current libcudf developer docs from a high level contributing guide would work, but since none of the subpages or subheaders would be part of the main documentation finding the docs would depend on readers finding a single link on the contributing guide. I would be fine with a compromise where we moved the top-level contributing guide alone to Sphinx and made the links to the libcudf developer docs easily discoverable but left those documents in the source repo where they currently are. I think that would be enough to improve visibility.

[github-actions]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

[github-actions]

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

[vyasr]

We are close to having developer docs completed for cudf Python. Once that's done, I think the question we can get back to addressing the questions on this issue of how we split up the contributing guide between components. @shwina would you like to see this split up?

[shwina]

I'd rather that the contributor guide be more a landing page for any contributor, providing general information.

It should link to the individual developer guides for build instructions and best practices.

[vyasr]

We've done some pretty thorough reworking of the contributing docs since this issue was opened. I think we can close it for now, and revisit if new questions come up.