Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update docs to mention builtin orchestrator #5140

Merged
merged 5 commits into from
Nov 19, 2024

Conversation

CamronStaley
Copy link
Contributor

@CamronStaley CamronStaley commented Nov 18, 2024

What changes are proposed in this pull request?

Update docs to mention builtin orchestrator

What areas of FiftyOne does this PR affect?

  • App: FiftyOne application changes
  • Build: Build and test infrastructure changes
  • Core: Core fiftyone Python library changes
  • Documentation: FiftyOne documentation changes
  • Other

Summary by CodeRabbit

  • New Features

    • Added new sections: "Dataset Zoo" and "Model Zoo" in the documentation.
    • Introduced a section on "FiftyOne Teams Built-in Orchestrator" for managing delegated operations.
    • Enhanced documentation for plugin development with detailed guides and visual aids.
    • Added a new TimelineView for custom animations in the app.
  • Bug Fixes

    • Clarified references to the orchestration tool in various documentation sections.
    • Fixed various issues related to overlay z-index and performance improvements in the app.
  • Documentation

    • Expanded and refined existing sections for clarity and usability.
    • Streamlined setup instructions for the built-in orchestrator in the Teams context.
    • Updated release notes with enhancements and fixes across different components.

@CamronStaley CamronStaley self-assigned this Nov 18, 2024
Copy link
Contributor

coderabbitai bot commented Nov 18, 2024

Walkthrough

The pull request introduces several updates to the FiftyOne documentation, particularly focusing on the index.rst, developing_plugins.rst, index.rst for plugins, using_plugins.rst, and teams_plugins.rst files. Key changes include the clarification of the orchestration tool associated with FiftyOne Teams, the addition of new sections for "Dataset Zoo" and "Model Zoo," and enhancements to the guidance on developing and using plugins. The updates aim to improve clarity and usability without altering existing functionalities.

Changes

File Path Change Summary
docs/source/index.rst Updated orchestration tool reference; added new sections: Dataset Zoo and Model Zoo.
docs/source/plugins/developing_plugins.rst Restructured content, detailed plugin components, expanded sections on plugin types (Python and JavaScript), added best practices, and included visual aids.
docs/source/plugins/index.rst Updated orchestration tool reference; no additional changes.
docs/source/plugins/using_plugins.rst Added a section on FiftyOne Teams Builtin Orchestrator, refined existing sections, expanded Delegated operations and removed content related to Apache Airflow.
docs/source/teams/teams_plugins.rst Clarified use of plugins within Teams, detailed orchestrator setup, added new environment variables, and enhanced management of delegated operations.
docs/source/release-notes.rst Updated release notes for versions 2.1.3 and 1.0.2, including model additions, bug fixes, and enhancements across various sections.

Possibly related PRs

  • FiftyOne Teams 1.7/FiftyOne v0.24.0 release notes #4433: This PR updates the release notes for FiftyOne Teams 1.7, which includes changes to the description of executing long-running tasks within the FiftyOne App, similar to the updates made in the main PR regarding the FiftyOne Plugins section.
  • Teams 1.7.1/OSS 0.24.1 release notes #4456: This PR also modifies the release notes and includes updates related to the plugin descriptions, which aligns with the changes made in the main PR regarding the FiftyOne Plugins section.
  • add script to post process docs #4838: This PR introduces a script that processes documentation, including the addition of new sections marked with __SUB_NEW__, which directly relates to the changes made in the main PR regarding the Dataset Zoo and Model Zoo.
  • updated release notes #5021: This PR updates the release notes for FiftyOne Teams 2.1.2, which includes enhancements to delegated operations, reflecting the ongoing improvements in documentation and functionality noted in the main PR.

Suggested labels

documentation

Suggested reviewers

  • findtopher

🐇 In the garden where knowledge grows,
FiftyOne's plugins now brightly glow.
With new sections added, clarity reigns,
Developers cheer, no more pains!
From datasets to models, all in sight,
Hop along, let's code with delight! 🌼


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (5)
docs/source/plugins/index.rst (1)

14-15: LGTM with a minor suggestion for clarity

The update appropriately mentions the FiftyOne Teams Builtin Orchestrator. However, the phrase "like the" suggests there might be other orchestrators, which could be confusing if the builtin orchestrator is the primary or only supported option.

Consider this minor rewording for better clarity:

-that execute on a connected workflow orchestration tool like the FiftyOne Teams
-Builtin Orchestrator.
+that execute on the FiftyOne Teams Builtin Orchestrator, a connected workflow
+orchestration tool.
docs/source/index.rst (1)

Line range hint 611-612: Remove the __SUB_NEW__ markers

The __SUB_NEW__ markers appear to be temporary annotations and should be removed as they are not standard RST directives.

-   Dataset Zoo __SUB_NEW__ <dataset_zoo/index>
-   Model Zoo __SUB_NEW__ <model_zoo/index>
+   Dataset Zoo <dataset_zoo/index>
+   Model Zoo <model_zoo/index>
docs/source/plugins/using_plugins.rst (1)

952-975: Consider improving the formatting consistency

While the content is informative, there are some minor formatting inconsistencies:

  • Line 959: The sentence doesn't start with a capital letter "would"
  • Line 961: Consider rephrasing "We recognize that supporting Airflow..." to maintain a more formal documentation tone
  • Line 973-974: The sentence appears incomplete, missing "Think of" at the start to match the similar note in other sections

Apply these changes to improve consistency:

-would typically take too long for a user to wait for them to complete within the app.
+Would typically take too long for a user to wait for them to complete within the app.

-We recognize that supporting Airflow or other workflow orchestrators can be an
+Setting up Airflow or other workflow orchestrators can be an

-FiftyOne Teams as the single source of truth on which you
+Think of FiftyOne Teams as the single source of truth on which you
docs/source/teams/teams_plugins.rst (2)

357-358: Consider enhancing the builtin orchestrator introduction.

The introduction could be more explicit about the builtin orchestrator being the default and preferred choice. Consider adding a brief mention of its advantages, such as:

  • Zero configuration required
  • Seamless integration with FiftyOne Teams
  • Built-in monitoring and management
-App that are executed on a connected workflow orchestrator like the provided
-FiftyOne builtin orchestator.
+App that are executed on a workflow orchestrator. FiftyOne Teams comes with a 
+builtin orchestrator that requires zero configuration and is ready to use out 
+of the box. Alternatively, you can connect to external orchestrators like 
+Apache Airflow.

440-444: Enhance the builtin orchestrator configuration details.

The section could benefit from:

  1. More details about the default worker configuration
  2. Clearer separation between builtin and additional orchestrator options
  3. Fixed punctuation
-FiftyOne Teams offers a builtin orchestrator which is configured as part of your
-deployment. Three workers are provisioned by default, to get help scaling up
-or down contact your Voxel51 support team. If you'd like to configure an additional
-orchestrator, such as Airflow, continue reading below.
+FiftyOne Teams offers a builtin orchestrator that comes pre-configured with your
+deployment. By default, it provisions three workers that can handle multiple
+concurrent operations. To adjust the number of workers based on your workload,
+contact your Voxel51 support team.
+
+Advanced Configuration:
+If you'd like to configure an additional orchestrator, such as Apache Airflow,
+continue reading below.
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 9a5eb8e and 876ff3b.

📒 Files selected for processing (5)
  • docs/source/index.rst (1 hunks)
  • docs/source/plugins/developing_plugins.rst (1 hunks)
  • docs/source/plugins/index.rst (1 hunks)
  • docs/source/plugins/using_plugins.rst (2 hunks)
  • docs/source/teams/teams_plugins.rst (2 hunks)
🔇 Additional comments (3)
docs/source/index.rst (1)

378-379: LGTM: Clear and accurate orchestrator reference

The update appropriately references the FiftyOne Teams Builtin Orchestrator, which aligns with the PR's objective to improve documentation clarity about this feature.

docs/source/plugins/using_plugins.rst (1)

834-834: LGTM: Reference link addition

The added reference to the builtin orchestrator section is properly formatted and correctly links to the new content.

docs/source/plugins/developing_plugins.rst (1)

1132-1132: Update reference to orchestrator

The change from Apache Airflow to "FiftyOne Teams Builtin Orchestrator" aligns with the PR objectives to update documentation about the builtin orchestrator.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (2)
docs/source/teams/teams_plugins.rst (1)

357-358: Fix typo and improve readability

There's a typo in "orchestator" and the sentence structure could be improved.

-App that are executed on a connected workflow orchestrator like the provided
-FiftyOne builtin orchestator.
+App that are executed on a connected workflow orchestrator, such as the provided
+FiftyOne builtin orchestrator.
docs/source/plugins/using_plugins.rst (1)

952-967: Consider enhancing the builtin orchestrator section

While the content effectively introduces the builtin orchestrator, consider adding:

  1. A brief overview of the key features or capabilities
  2. Examples of common use cases
  3. Any specific requirements or prerequisites

The note about contacting for FiftyOne Teams is well-placed and relevant.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 876ff3b and a42224f.

📒 Files selected for processing (2)
  • docs/source/plugins/using_plugins.rst (2 hunks)
  • docs/source/teams/teams_plugins.rst (2 hunks)
🔇 Additional comments (2)
docs/source/teams/teams_plugins.rst (1)

440-442: LGTM!

The information about the builtin orchestrator's default configuration and support options is clear and well-documented.

docs/source/plugins/using_plugins.rst (1)

834-834: LGTM: Clear reference to the builtin orchestrator

The addition of the reference to the FiftyOne Teams Builtin Orchestrator enhances the documentation by providing users with information about this deployment option.

Copy link
Contributor

@brimoor brimoor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some more updates and now LGTM

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (2)
docs/source/teams/teams_plugins.rst (1)

432-433: Fix RST heading format inconsistency

The heading underline should match the length of the heading text for consistency with other RST headings in the document.

-Configuring your orchestrator(s)
-________________________________
+Configuring your orchestrator(s)
+_______________________________
docs/source/plugins/using_plugins.rst (1)

874-917: Improve RST structure consistency.

The section structure could be improved for better maintainability:

  1. Use consistent section markers (either all dashes or all underscores).
  2. Fix indentation in the RST structure.

Apply this diff to improve the structure:

 .. _delegated-orchestrator:
 
 Setting up an orchestrator
-__________________________
+-------------------------
 
 .. _delegated-orchestrator-open-source:
 
 FiftyOne Open Source
--------------------
+------------------
 
 FiftyOne Open Source users can run delegated operations via the
 :ref:`fiftyone delegated launch <cli-fiftyone-delegated-launch>` CLI command:
 
 .. code-block:: shell
 
     fiftyone delegated launch
 
 This command starts a service that will continuously check for any queued
 delegated operations and execute them serially in its process.
 
 .. _delegated-orchestrator-teamas:
 
 FiftyOne Teams
---------------
+-------------
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between a42224f and a6da1ed.

📒 Files selected for processing (6)
  • docs/source/index.rst (1 hunks)
  • docs/source/plugins/developing_plugins.rst (2 hunks)
  • docs/source/plugins/index.rst (1 hunks)
  • docs/source/plugins/using_plugins.rst (2 hunks)
  • docs/source/release-notes.rst (1 hunks)
  • docs/source/teams/teams_plugins.rst (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • docs/source/index.rst
  • docs/source/plugins/index.rst
🔇 Additional comments (10)
docs/source/teams/teams_plugins.rst (4)

357-357: LGTM: Clear description of execution environment

The updated text clearly explains that delegated operations are executed on a connected compute cluster.


435-441: LGTM: Clear explanation of orchestrator options

The content effectively explains the builtin orchestrator while maintaining flexibility with external options.


453-457: LGTM: Clear explanation of the Runs page and execution queue

The content effectively explains the Runs page functionality and the queue-based execution model.


461-462: LGTM: Important clarification about operation tracking

The note clearly distinguishes between tracked (scheduled) and untracked (immediate) operations.

docs/source/plugins/using_plugins.rst (2)

832-840: LGTM! Clear and concise introduction to delegated operations.

The explanation effectively communicates the purpose and benefits of delegated operations.


841-842: Verify the cross-reference exists.

Please ensure that the :ref:teams-delegated-operations`` target exists in the documentation.

✅ Verification successful

Cross-references are valid and correctly defined

Both referenced sections exist in the documentation:

  • teams-delegated-operations is defined in docs/source/teams/teams_plugins.rst
  • delegated-orchestrator-open-source is defined in docs/source/plugins/using_plugins.rst
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for the referenced section label in RST files
rg ".. _teams-delegated-operations:" -t rst

Length of output: 113


Script:

#!/bin/bash
# Check if the referenced section exists in the current file
rg "delegated-orchestrator-open-source" -t rst

Length of output: 351

docs/source/plugins/developing_plugins.rst (3)

1131-1132: LGTM: Clear explanation of delegated operations

The added text provides a clear and accurate explanation of how delegated operations can be used to schedule long-running tasks in the background while continuing to use the App.


1136-1141: LGTM: Clear distinction between Teams and Open Source capabilities

The added text effectively clarifies the differences in delegated operation capabilities between:

  • FiftyOne Teams: Out-of-the-box support with connected compute cluster
  • FiftyOne Open Source: Small-scale support with local execution

3567-3567: LGTM: Accurate reference to delegated execution

The added text correctly references the delegated execution feature, maintaining consistency with earlier documentation.

docs/source/release-notes.rst (1)

Line range hint 1-24: LGTM! Release notes structure follows best practices

The release notes are well-organized with:

  • Clear version numbering and release dates
  • Consistent section headers (App, Core, Brain etc.)
  • Proper reStructuredText formatting

Copy link
Contributor

@swheaton swheaton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have changes locally, do not merge

@swheaton
Copy link
Contributor

Although if BM is good with it as exists now maybe I'll just throw my changes away and move on.

@swheaton swheaton dismissed their stale review November 19, 2024 16:56

nvm, throwing away my local changes

Copy link
Contributor

@swheaton swheaton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, defer to product

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants