Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DISCUSS] cuDF internal/developer documentation #6481

Closed
shwina opened this issue Oct 9, 2020 · 9 comments · Fixed by #11199
Closed

[DISCUSS] cuDF internal/developer documentation #6481

shwina opened this issue Oct 9, 2020 · 9 comments · Fixed by #11199
Assignees
@vyasr
Labels
feature request New feature or request Python Affects Python cuDF API.

Comments

@shwina
Copy link
Contributor

shwina commented Oct 9, 2020

Because cuDF is designed and written differently from many other Python libraries, it's important that we document our internals and design choices carefully. Currently, it can be a significant effort for someone to start contributing to the cuDF source code due to the lack of developer documentation.

We're developing an outline for our dev docs here:

https://docs.google.com/document/d/1gJOw2-DuzfpNz1CPiwgw3cf3Ag1h8tu_j_kPFYCgg8k/edit#heading=h.3fq7j7kityfk

Feedback and suggestions are very much appreciated.
@shwina shwina added feature request New feature or request Needs Triage Need team to review and classify Python Affects Python cuDF API. help wanted and removed Needs Triage Need team to review and classify help wanted labels Oct 9, 2020
@github-actions
Copy link

This issue has been marked rotten due to no recent activity in the past 90d. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

@shwina shwina self-assigned this Feb 16, 2021
@shwina
Copy link
Contributor Author

shwina commented Feb 16, 2021

Still very much needed.

@github-actions github-actions bot removed the rotten label Feb 16, 2021
@github-actions
Copy link

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

@quasiben
Copy link
Member

Maybe a first iteration on this would be developing some thoughts on where code lives and why between Cython and Python ?

@shwina
Copy link
Contributor Author

shwina commented May 26, 2021

Yeah that sounds like a really good place to start.

@harrism
Copy link
Member

harrism commented May 27, 2021

In case it helps, libcudf has a developer guide here: https://github.com/rapidsai/cudf/blob/branch-21.06/cpp/docs/DEVELOPER_GUIDE.md

@shwina
Copy link
Contributor Author

shwina commented May 27, 2021

I'll target this for 21.08 along with the array refactor.

@vyasr vyasr assigned vyasr and unassigned shwina Jul 19, 2021
@vyasr
Copy link
Contributor

vyasr commented Jul 19, 2021

I've been working on this and can take it off @shwina's plate.

@vyasr vyasr mentioned this issue Sep 20, 2021
Closed
@vyasr vyasr mentioned this issue May 10, 2022
Merged
rapids-bot bot pushed a commit that referenced this issue May 23, 2022
@vyasr
Add a library_design.md file documenting the core Python data structu…
a9220bd 
…res and their relationship (#10817)

This PR adds an `library_design.md` file discussing cuDF's internal architecture, including its core data structures, their purpose, and how they related to pandas and libcudf objects. The document is not short, but it aims to avoid being too long by focusing mainly on the layout between classes and how they interact. I do not discuss implementation details for specific functionality (e.g. the Merge or GroupBy classes), nor do I go into detail on the layout of files on disk. The emphasis is on understanding the different principal components and how they fit together.

This PR contributes to #6481. Subsequent PRs will focus on other aspects of a developer guide, such as a more information on how to contribute, write tests, benchmark, and write documentation.

Authors:
  - Vyas Ramasubramani (https://github.com/vyasr)

Approvers:
  - Bradley Dice (https://github.com/bdice)
  - Ashwin Srinath (https://github.com/shwina)

URL: #10817
@vyasr
Copy link
Contributor

vyasr commented Jul 9, 2022

We should be able to close this issue between #11217, #11199, #11122, #11235, and #10817

@vyasr vyasr mentioned this issue Aug 4, 2022
Closed
@vyasr vyasr mentioned this issue Oct 26, 2022
Merged
@rapids-bot rapids-bot bot closed this as completed in eaa0706 Nov 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vyasr vyasr

Labels
feature request New feature or request Python Affects Python cuDF API.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add developer docs for writing tests vyasr/cudf
4 participants
@harrism @quasiben @vyasr @shwina