Update documentation

sed/develop/_sources/index.md.txt

@@ -0,0 +1,54 @@
+---
+myst:
+  html_meta:
+    "description lang=en": |
+      Top-level documentation for sed, with links to the rest
+      of the site..
+html_theme.sidebar_secondary.remove: true
+---
+
+# SED documentation
+
+SED (Single Event Data Frame) is a collection of routines and utilities to handle photoelectron resolved datastreams.
+It features lazy evaluation of dataframe processing using dask, numba-accelerated multi-dimensional binning, calibration and correction for trARPES (Time- and angle-resolved photoemission spectroscopy) datasets.
+The package ensures provenance and FAIR data through metadata tracking, usage of the community defined NeXus format.
+
+## User guide
+
+We introduce different functionalities of the package by several step-by-step use guides:
+
+```{toctree}
+:maxdepth: 1
+
+user_guide/index
+
+```
+
+## Examples
+
+Several example notebooks to demonstrate the functionality of SED for end-to-end data analysis workflows.
+
+```{toctree}
+:maxdepth: 2
+
+workflows/index
+```
+
+## API
+
+```{toctree}
+:maxdepth: 2
+
+sed/api
+```
+
+
+## Community and contribution guide
+
+Information about the community behind this theme and how you can contribute.
+
+```{toctree}
+:maxdepth: 2
+
+misc/contribution
+```

sed/develop/_sources/misc/contributing.rst.txt

@@ -0,0 +1,132 @@
+==============================
+Contributing to sed
+==============================
+
+Welcome to the sed project, a collaboration of the Open Community of Multidimensional Photoemission Spectroscopy.
+
+Whether you are a beamline scientist hoping to create a loader for your data, or would like to add a new feature to the project, we welcome your contributions.
+
+This guide will walk you through the process of setting up your development environment, and the workflow for contributing to the project.
+
+
+Getting Started
+===============
+
+1. **Clone the Repository:**
+
+   - If you are a member of the repository, clone the repository to your local machine:
+
+    .. code-block:: bash
+
+        git clone https://github.com/OpenCOMPES/sed.git
+
+   - If you are not a member of the repository, clone your fork of the repository to your local machine:
+
+    .. code-block:: bash
+
+        git clone https://github.com/yourusername/sed.git
+
+
+
+2. **Install Python and Poetry:**
+   - Ensure you have Python 3.8, 3.9, 3.10 or 3.11 and poetry installed.
+
+    .. code-block:: bash
+
+        pip install pipx
+        pipx install poetry
+
+3. **Clone Repository:**
+
+    .. code-block:: bash
+
+        git clone https://github.com/OpenCOMPES/sed.git
+
+4. **Install Dependencies:**
+   - Navigate to the project directory and install the project dependencies (including development ones) using Poetry:
+
+    .. code-block:: bash
+
+        poetry install --dev
+
+
+Development Workflow
+=====================
+
+.. note::
+   This guide assumes that you have Python (version 3.8, 3.9, 3.10, 3.11) and poetry with dev dependencies installed on your machine.
+
+1. **Install pre-commit hooks:** To ensure your code is formatted correctly, install pre-commit hooks:
+
+    .. code-block:: bash
+
+        pip install pre-commit
+
+
+2. **Create a Branch:** Create a new branch for your feature or bug fix and make changes:
+
+    .. code-block:: bash
+
+        git checkout -b feature-branch
+
+
+3. **Write Tests:** If your contribution introduces new features or fixes a bug, add tests to cover your changes.
+
+4. **Run Tests:** To ensure no functionality is broken, run the tests:
+
+    .. code-block:: bash
+
+        pytest tests
+
+
+5. **Commit Changes:** Commit your changes with a clear and concise commit message:
+
+    .. code-block:: bash
+
+        git commit -a -m "Your commit message"
+
+
+6. **Push Changes:** Push your changes to your fork:
+
+    .. code-block:: bash
+
+        git push origin feature-branch
+
+
+7. **Open a Pull Request:** Open a pull request against the `main` branch of sed.
+
+Pull Request Guidelines
+=======================
+
+Please give a brief description of the changes you have made in your pull request.
+If your pull request fixes an issue, please reference the issue number in the pull request description.
+
+Before your pull request can be merged, it must pass the following checks:
+
+- **Linting Check**
+
+- **Tests Check**
+
+- **Code Review:** A maintainer will review your code and provide feedback if necessary.
+
+- **Rebase with Main:** Ensure your branch is up-to-date with the latest changes from the `main` branch.
+
+Once all checks are successful and your code is approved, it will be merged into the main branch.
+
+Developing a Loader
+===================
+If you are developing a loader for your beamline, please follow the guidelines below.
+
+1. **Create a Loader:**
+
+   - Create a new loader in the `sed/loaders` directory.
+   - The loader should be a subclass of `sed.loader.base.loader.BaseLoader` and implement a few methods. See :ref:`base_loader` for more information.
+   - Give your class a `__name__` attribute, which is used to select the loader in user config files (See the generic loader for example).
+   - At the end of your module, provide a `LOADER = YourNameLoader` variable, which is used to register that loader in the registry. See :ref:`loader_interface`.
+
+2. **Write Tests:**
+
+   - Write tests for your loader in the `tests/loaders` directory.
+   - You can also include a small test data in the `tests/data` directory.
+
+3. **Add Loader to Documentation:** Add your loader to the documentation in `docs/sed/loaders.rst`.

sed/develop/_sources/misc/contribution.md.txt

@@ -0,0 +1,8 @@
+# Development
+
+```{toctree}
+:maxdepth: 2
+
+contributing
+maintain
+```

sed/develop/_sources/misc/maintain.rst.txt

@@ -0,0 +1,156 @@
+How to Maintain
+===============
+
+Documentation
+-------------
+**Build Locally:**
+
+Users can generate documentation locally using the following steps:
+
+1. **Install Dependencies:**
+
+.. code-block:: bash
+
+    pip install pipx
+    pipx install poetry
+
+1. **Clone Repository:**
+
+.. code-block:: bash
+
+    git clone https://github.com/OpenCOMPES/sed.git
+
+3. **Navigate to Repository:**
+
+.. code-block:: bash
+
+    cd sed
+
+4. **Copy Tutorial Files:**
+
+Doing this step will slow down the build process significantly. It also requires two datasets so 20 GB of free space is required.
+
+.. code-block:: bash
+
+    cp -r tutorial docs/
+    cp -r sed/config docs/sed
+
+5. **Create a virtual environment:**
+
+.. code-block:: bash
+
+    poetry shell
+
+6. **Install Dependencies:**
+
+.. code-block:: bash
+
+    poetry install --with docs
+
+7. **Build Documentation:**
+
+.. code-block:: bash
+
+    poetry run sphinx-build -b html docs _build
+
+8. **View Documentation:**
+
+Open the generated HTML documentation in the `_build` directory.
+
+**GitHub Workflow:**
+
+The documentation workflow is designed to automatically build and deploy documentation. Additionally, maintainers of sed repository can manually trigger the documentation workflow from the Actions tab.
+Here's how the workflow works:
+
+1. **Workflow Configuration:**
+   - The documentation workflow is triggered on push events to the main branch for specific paths and files related to documentation.
+   - Manual execution is possible using the workflow_dispatch event from the Actions tab.
+
+   .. code-block:: yaml
+
+      on:
+        push:
+          branches: [ main ]
+          paths:
+            - sed/**/*
+            - pyproject.toml
+            - tutorial/**
+            - .github/workflows/documentation.yml
+        workflow_dispatch:
+
+2. **Permissions:**
+   - The workflow sets permissions for the GITHUB_TOKEN to allow deployment to GitHub Pages.
+   - Permissions include read access to contents and write access to pages.
+
+   .. code-block:: yaml
+
+      permissions:
+        contents: read
+        pages: write
+        id-token: write
+
+3. **Concurrent Deployment:**
+   - Only one concurrent deployment is allowed to prevent conflicts.
+   - Future idea would be to have different deployment for different versions.
+   - Runs queued between an in-progress run and the latest queued run are skipped.
+
+   .. code-block:: yaml
+
+      concurrency:
+        group: "pages"
+        cancel-in-progress: false
+
+4. **Workflow Steps:**
+   - The workflow is divided into two jobs: build and deploy.
+
+     a. **Build Job:**
+        - Sets up the build environment, checks out the repository, and installs necessary dependencies using Poetry.
+        - Installs notebook dependencies and Pandoc.
+        - Copies tutorial files to the docs directory and removes unnecessary notebooks.
+        - Downloads RAW data for tutorials.
+        - Builds Sphinx documentation.
+
+     b. **Deploy Job:**
+        - Deploys the built documentation to GitHub Pages.
+
+5. **Manual Execution:**
+   - To manually trigger the workflow, go to the Actions tab on GitHub.
+   - Click on "Run workflow" for the "documentation" workflow.
+
+
+Release
+-------
+
+**Creating a Release**
+
+To create a release, follow these steps:
+
+   a. **Create a Git Release on Github:**
+
+      - On the "tags" page, select "releases", and press "Draft a new release".
+      - At "choose a tag", type in the name of the new release tag. Make sure to have a **v** prefix in the tag name, e.g. **v0.1.10**.
+      - Confirm creation of the tag, and press "Generate release notes". Edit the notes as appropriate (e.g. remove auto-generated update PRs).
+      - Press "Publish release". This will create the new tag and release entry, and issue the build and upload to PyPI.
+
+   b. **Check PyPI for the Published Package:**
+
+      - Visit the PyPI page (https://pypi.org/project/sed-processor/).
+      - Confirm that the new version (e.g., 0.1.10) has been published.
+
+   c. **If you don't see update on PyPI:**
+
+      - Visit the GitHub Actions page and monitor the Release workflow (https://github.com/OpenCOMPES/sed/actions/workflows/release.yml).
+      - Check if errors occurred during the release process.
+
+
+**Understanding the Release Workflow**
+
+- *Release Job:*
+    - This workflow is responsible for versioning and releasing the package.
+    - A release job runs on every git tag push (e.g., `git tag v0.1.5`) and publishes the package to PyPI.
+    - If the publish is successful, the version in the `pyproject.toml` file is updated and pushed to the main branch.
+
+- *Prerelease Job:*
+    - This workflow is triggered automatically on every pull request (PR) to the main branch.
+    - It increments the version number for prerelease (e.g., from 0.1.5 to 0.1.6a0 to 0.1.6a1) and publishes the package to PyPI.
+    - If the publish is successful, the version in the `pyproject.toml` file is updated and pushed to the main branch.

sed/develop/_sources/sed/api.rst.txt

@@ -0,0 +1,17 @@
+======
+API
+======
+
+.. toctree::
+   :maxdepth: 1
+
+   core
+   dfops
+   loader
+   binning
+   calibrator
+   dataset
+   diagnostic
+   io
+   metadata
+   config

sed/develop/_sources/sed/binning.rst.txt

@@ -0,0 +1,21 @@
+Binning
+===================================================
+Main functions
+###################################################
+
+.. automodule:: sed.binning
+   :members:
+   :undoc-members:
+
+
+
+Used helper functions
+##################################################
+
+.. automodule:: sed.binning.numba_bin
+   :members:
+   :undoc-members:
+
+.. automodule:: sed.binning.utils
+   :members:
+   :undoc-members: