feat: improve dev doc (#972)

* Remove duplicate debugging section. * Fix how to submit PR and testing. * Reorganize the contribution section. * Add an installation guide under getting started.
georgia-tech-db · Aug 28, 2023 · 8a6653b · 8a6653b
1 parent 405b773
commit 8a6653b
Show file tree

Hide file tree

Showing 33 changed files with 300 additions and 237 deletions.
diff --git a/docs/_toc.yml b/docs/_toc.yml
@@ -4,6 +4,9 @@ parts:
   - caption: Overview
     chapters:
       - file: source/overview/getting-started
+        sections:
+          - file: source/overview/getting-started/install-guide
+            title: Installation Guide
       - file: source/overview/concepts
       # - file: source/overview/faq
 
@@ -72,20 +75,43 @@ parts:
 
   - caption: Developer Guide
     chapters:
-      - file: source/contribute/index
+      - file: source/dev-guide/architecture
+        title: Architecture Design of EvaDB
+
+      - file: source/dev-guide/contribute
         title: Contributing to EvaDB
+        sections:
+          - file: source/dev-guide/contribute/setup-dev
+            title: Setup Environment
+          - file: source/contribute/contribute/testing
+            title: Testing
+          - file: source/dev-guide/contribute/submit-pr
+            title: Submit a PR
+          - file: source/dev-guide/contribute/code-style
+            title: Code Style
+          - file: source/dev-guide/contribute/troubleshoot
+            title: Troubleshooting
 
-      - file: source/contribute/debugging
+      - file: source/dev-guide/debugging
         title: Debugging EvaDB
+        sections:
+          - file: source/dev-guide/debugger/vscode-debugger
+            title: VSCode Debugger
+          - file: source/dev-guide/debugger/alternative
+            title: Alternaitve Debugger
 
-      - file: source/contribute/extend
+      - file: source/dev-guide/extend
         title: Extending EvaDB
         sections:
-          - file: source/contribute/new-data-source
+          - file: source/dev-guide/extend/new-data-source
             title: Structured Data Source Integration
-          - file: source/contribute/new-command
+          - file: source/dev-guide/extend/new-command
             title: Operators
 
-
-      - file: source/contribute/release
+      - file: source/dev-guide/release
         title: Releasing EvaDB
+        sections:
+          - file: source/dev-guide/release/pypi-account
+            title: Setup PyPI Account
+          - file: source/dev-guide/release/release-steps
+            title: How to Release
diff --git a/docs/source/contribute/extend.rst b/docs/source/contribute/extend.rst
diff --git a/docs/source/contribute/index.rst b/docs/source/contribute/index.rst
diff --git a/docs/source/dev-guide/architecture.rst b/docs/source/dev-guide/architecture.rst
@@ -0,0 +1,5 @@
+Architecture Diagram
+========================
+
+.. image:: ../../images/evadb/evadb-arch.png
+   :width: 1200
diff --git a/docs/source/dev-guide/contribute.rst b/docs/source/dev-guide/contribute.rst
@@ -0,0 +1,13 @@
+Contributing
+----------------
+
+We welcome all kinds of contributions to EvaDB.
+
+-  `Code reviews <https://github.com/georgia-tech-db/evadb/pulls>`_
+-  `Improving documentation <https://github.com/georgia-tech-db/evadb/tree/master/docs>`_
+-  `Tutorials and applications <https://github.com/georgia-tech-db/evadb/tree/master/tutorials>`_
+-  `New features <new-command.html>`__
+
+This document goes over how to contribute to EvaDB in details.
+
+.. tableofcontents::
diff --git a/docs/source/dev-guide/contribute/code-style.rst b/docs/source/dev-guide/contribute/code-style.rst
@@ -0,0 +1,20 @@
+Code Style
+============
+
+We use the `black <https://github.com/psf/black>`__ code style for
+formatting the Python code. For docstrings and documentation, we use
+`Google Pydoc format <https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html>`__.
+
+.. code-block:: python
+
+   def function_with_types_in_docstring(param1, param2) -> bool:
+       """Example function with types documented in the docstring.
+
+       Additional explanatory text can be added in paragraphs.
+
+       Args:
+           param1 (int): The first parameter.
+           param2 (str): The second parameter.
+
+       Returns:
+           bool: The return value. True for success, False otherwise.
diff --git a/docs/source/dev-guide/contribute/setup-dev.rst b/docs/source/dev-guide/contribute/setup-dev.rst
@@ -0,0 +1,36 @@
+Development Environment
+=====================================================
+
+Checkout Latest EvaDB
+---------------------
+
+First, you will need to checkout the repository from GitHub and build EvaDB from the source. 
+
+.. code-block:: bash
+
+   git clone https://github.com/georgia-tech-db/evadb.git && cd evadb
+
+Build EvaDB Locally
+-------------------
+
+Follow the following instructions to build EvaDB locally. We recommend using a virtual environment and the pip package manager. 
+
+.. code-block:: bash
+
+   python3 -m venv test_evadb_venv
+   source test_evadb_venv/bin/activate
+   pip install --upgrade pip
+   pip install -e ".[dev]"
+   
+After installing the package locally, you can make changes and run the test cases to check their impact.
+
+.. note::
+
+   EvaDB provides multiple installation options for extending its functionalities. 
+   Please see :doc:`Installation Guide <getting-started/install-guide>` for all options.
+
+Other options can be installed with the ``dev`` environment.
+
+.. code-block:: bash
+   
+   pip install -e ".[dev,ludwig]"
diff --git a/docs/source/dev-guide/contribute/submit-pr.rst b/docs/source/dev-guide/contribute/submit-pr.rst
@@ -0,0 +1,38 @@
+Submitting a PR
+============================
+
+For every open PR, we only run unit tests and short integration test to facilitate merging features quickly. 
+Once PR passes those tests, it will be merged into our ``staging`` branch for more comprehensive integration, version, and
+application tests. 
+
+Follow the following steps to contribute to EvaDB.
+
+Sync with ``Staging`` Branch
+----------------------------
+
+Merge the most recent changes from the ``staging`` branch
+
+.. code-block:: bash
+
+       git remote add origin [email protected]:georgia-tech-db/evadb.git
+       git pull . origin/staging
+
+Testing
+-------
+
+Run the :doc:`test script <./testing>` to ensure that all the test cases pass.
+
+Documentation
+-------------
+
+If you are adding a new EvaDB command, add an illustrative example usage in the 
+`documentation <https://github.com/georgia-tech-db/evadb/tree/master/docs>`_.
+
+Formatting
+----------
+
+Run the following command to ensure that code is properly formatted.
+
+.. code-block:: python
+
+      python script/formatting/formatter.py 
diff --git a/docs/source/dev-guide/contribute/testing.rst b/docs/source/dev-guide/contribute/testing.rst
@@ -0,0 +1,31 @@
+Testing
+=========
+
+Check if your local changes broke any unit or integration tests by running the following script:
+
+.. code-block:: bash
+
+   bash script/test/test.sh
+
+By default, it will run the full test suite. You can also run subset of test suites.
+
+.. code-block:: bash
+
+   # Unit tests.
+   bash script/test/test.sh -m UNIT
+
+   # Integration tests.
+   bash script/test/test.sh -m "SHORT INTEGRATION" 
+   bash script/test/test.sh -m "LONG INTEGRATION" 
+
+If you want to run a specific test file, use the following command.
+
+.. code-block:: bash
+
+   PYTHONPATH="." python -m pytest test/integration_tests/test_select_executor.py
+
+Use the following command to run a specific test case within a specific test file.
+
+.. code-block:: bash
+
+   PYTHONPATH="." python -m pytest test/integration_tests/test_select_executor.py -k 'test_should_load_and_select_in_table'
diff --git a/docs/source/dev-guide/contribute/troubleshoot.rst b/docs/source/dev-guide/contribute/troubleshoot.rst
@@ -0,0 +1,4 @@
+Troubleshooting
+====================
+
+If the test suite fails with a `PermissionDenied` exception, update the `path_prefix` attribute under the `storage` section in the EvaDB configuration file (``~/.evadb/evadb.yml``) to a directory where you have write privileges.
diff --git a/docs/source/contribute/debugging.rst → ...source/dev-guide/debugger/alternative.rst b/docs/source/contribute/debugging.rst → ...source/dev-guide/debugger/alternative.rst
@@ -1,26 +1,5 @@
-Debugging
------------
-
-We recommend Visual Studio Code with a debugger for debugging EvaDB. This tutorial presents a detailed step-by-step process of using the debugger.
-
-Setup debugger
-================
-
-1. Install the `Python extension <https://marketplace.visualstudio.com/items?itemName=ms-python.python>`__ in Visual Studio Code.
-
-2. Install the `Python Test Explorer extension <https://marketplace.visualstudio.com/items?itemName=LittleFoxTeam.vscode-python-test-adapter>`__.
-
-3. Follow these instructions to run a particular test case from the file:
-`Getting started <https://github.com/kondratyev-nv/vscode-python-test-adapter>`__.
-
-.. image:: images/eva-debug-1.jpg
-   :width: 1200
-
-.. image:: images/eva-debug-2.jpg
-   :width: 1200
-
-Alternative: Manually Setup Debugger for EvaDB
-====================================================
+Alternative: Manually Setup Debugger
+====================================
 
 When you press the debug icon, you will be given an option to create a ``launch.json`` file.
 
@@ -65,4 +44,4 @@ You can modify the fields of the above JSON file as follows:
     example, the path for the conda environment for Eva has been specified.
 
 Using these configuration variables, you can run the debugger both locally as
-well as on a remote server.
+well as on a remote server.