Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOCS] Add docs pointing to daft-launcher #3369

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

raunakab
Copy link
Contributor

@raunakab raunakab commented Nov 20, 2024

Overview

  • added a small section pointing towards daft-launcher and how to install/use it
  • placed underneath the distributed-computing section
  • also added a small note to the ray section pointing towards the daft-launcher docs

@raunakab raunakab marked this pull request as ready for review November 20, 2024 22:49
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Nov 20, 2024
Copy link

codspeed-hq bot commented Nov 20, 2024

CodSpeed Performance Report

Merging #3369 will improve performances by 53.15%

Comparing docs/add-launcher-docs (4af1d01) with main (60ae62f)

Summary

⚡ 2 improvements
✅ 15 untouched benchmarks

Benchmarks breakdown

Benchmark main docs/add-launcher-docs Change
test_iter_rows_first_row[100 Small Files] 388.4 ms 335.1 ms +15.91%
test_show[100 Small Files] 23.1 ms 15.1 ms +53.15%

Copy link

graphite-app bot commented Nov 20, 2024

Graphite Automations

"Request reviewers once CI passes" took an action on this PR • (11/20/24)

1 reviewer was added to this PR based on Andrew Gazelka's automation.

Copy link

codecov bot commented Nov 20, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 76.73%. Comparing base (60ae62f) to head (4af1d01).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3369      +/-   ##
==========================================
- Coverage   77.35%   76.73%   -0.63%     
==========================================
  Files         685      685              
  Lines       83639    84764    +1125     
==========================================
+ Hits        64703    65044     +341     
- Misses      18936    19720     +784     

see 13 files with indirect coverage changes

---- 🚨 Try these New Features:

Copy link

github-actions bot commented Nov 21, 2024

🚀 Deployed on https://deploy-preview-3369--daft-docs-preview.netlify.app

@ccmao1130
Copy link
Contributor

For MVP it looks good to me, but I think there's a better way we can incorporate Daft Launcher in the Distributed Computing page because right now (besides looking at the left menu) users have to scroll all the way to the bottom to click to the section.

Have you also thought about how/where to reference Daft Launcher on the Ray page? https://www.getdaft.io/projects/docs/en/stable/user_guide/integrations/ray.html

@raunakab
Copy link
Contributor Author

For MVP it looks good to me, but I think there's a better way we can incorporate Daft Launcher in the Distributed Computing page because right now (besides looking at the left menu) users have to scroll all the way to the bottom to click to the section.

Have you also thought about how/where to reference Daft Launcher on the Ray page? https://www.getdaft.io/projects/docs/en/stable/user_guide/integrations/ray.html

I can add it there if needed. Just didn't know that that was something that we wanted.

@raunakab
Copy link
Contributor Author

CodSpeed Performance Report

Merging #3369 will degrade performances by 32.94%

Comparing docs/add-launcher-docs (e5b209d) with main (3394a66)

Summary

⚡ 2 improvements ❌ 1 regressions ✅ 14 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark main docs/add-launcher-docs Change
test_count[1 Small File] 3.8 ms 3.5 ms +10.96%
test_iter_rows_first_row[100 Small Files] 324.1 ms 279.8 ms +15.83%
test_show[100 Small Files] 15.9 ms 23.7 ms -32.94%

I'm just changing some docs! Argh!

@ccmao1130
Copy link
Contributor

Hmm maybe just add something like "To run Daft on a distributed Ray cluster, see Daft Launcher [link to Daft Launcher docs on Daft docs]" wherever you see fit on the Ray page (maybe either right before "Usage" section or at the bottom of the page)

.gitignore Outdated Show resolved Hide resolved
@@ -11,6 +11,7 @@ requirements.txt
build
.cython_build
.hypothesis
.ropeproject
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this? We should avoid any autodoc thing that isn't standardized across the project I think, to avoid spurious refactors of the code when different developers work on the code

@@ -96,7 +96,7 @@
"learn/install": "../install.html",
"learn/user_guides/dataframes": "intro-dataframes.html",
"learn/user_guides/types_and_ops": "intro-dataframes.html",
"learn/user_guides/remote_cluster_execution": "distributed-computing.html",
"learn/user_guides/remote_cluster_execution": "user_guide/distributed-computing",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also add a redirect from the poweruser/distributed-computing page to distributed-computing I believe


.. toctree::

distributed-computing/daft-launcher
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's avoid nesting, just make daft-launcher a section here?

@jaychia
Copy link
Contributor

jaychia commented Nov 25, 2024

We can also verify your PR is doing what we want it to do here: https://deploy-preview-3369--daft-docs-preview.netlify.app/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants