Documentation setup and first draft

.github/workflows/documentation.yml

@@ -0,0 +1,87 @@
+# Basic workflow to build and publish book
+name: Documentation
+
+# Controls when the workflow will run
+on:
+  # Triggers the workflow on push or pull request events but only for the "main" branch
+  push:
+    branches: [ "main" ]
+  pull_request:
+    branches: [ "main" ]
+
+  # Allows you to run this workflow manually from the Actions tab
+  workflow_dispatch:
+
+# List of one or more jobs that can run sequentially or in parallel
+jobs:
+  # Build step
+  buildjob:
+    name: Building Documentation
+
+    # Select ubuntu: LTS versions should be preferred
+    runs-on: ubuntu-22.04
+
+    # Full sequence of tasks being executed as part of the job
+    steps:
+      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
+      - uses: actions/checkout@v4
+
+      # Setup python 3.10. Not mandatory to use this version, but it is the one available in ubuntu 22.04
+      - name: Python 3 setup
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
+
+      # Verify the installation is complete, with no issues
+      - name: Display python version
+        run: python -c "import sys; print(sys.version)"
+
+      # Install jupyter book package
+      - name: Install additional packages
+        run: pip install jupyter-book
+
+      # Compile the documentation
+      - name: Building documentation
+        run: jupyter-book build --path-output . docs/
+
+      # Configure pages
+      - name: Setup pages
+        id: pages
+        uses: actions/configure-pages@v3
+      - name: Build with Jekyll
+        uses: actions/jekyll-build-pages@v1
+        with:
+          source: ./_build/html
+          destination: ./_site
+
+      # Upload artifacts
+      - name: Upload artifacts after building
+        uses: actions/upload-pages-artifact@v2
+        with:
+          path: _build/html
+
+  deploy:
+    name: Deploy to Github Pages
+
+    # Assign permissions to the job
+    permissions:
+      contents: read
+      pages: write
+      id-token: write
+
+    # Moving to ubuntu latest as everything is already built
+    runs-on: ubuntu-latest
+
+    # Setting dependency
+    needs: buildjob
+
+    # Extra (default) settings to enforce branch/deployment protection rules.
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+
+    # Finally deploying everything
+    steps:
+      - name: Deploy artifact
+        id: deployment
+        uses: actions/deploy-pages@v3

docs/_config.yml

@@ -0,0 +1,32 @@
+# Book settings
+
+title: HighRatePlatformDocumentation
+author: "ICSC - Spoke 2"
+logo: "icsc.png"
+copyright: "2024"
+
+# Force re-execution of notebooks on each build
+execute:
+  execute_notebooks: force
+
+# Define the name of the latex output file for PDF builds
+latex:
+  use_jupyterbook_latex: false
+  latex_documents:
+    targetname: UserGuide.tex
+
+# Add a bibtex file so that we can create citations
+bibtex_bibfiles:
+  - references.bib
+
+# Information about where the book exists on the web
+repository:
+  url: https://github.com/ICSC-Spoke2-repo/HighRateAnalysis-WP5
+  branch: main
+
+# Add repository buttons to your book: remember, issues are not supported by GitLab
+html:
+  announcement: "Alpha Version 0.1"
+  home_page_in_navbar: false
+  use_issues_button: true
+  use_repository_button: true

docs/_toc.yml

@@ -0,0 +1,9 @@
+# Table of Contents
+
+format: jb-book
+root: sections/intro
+chapters:
+- file: sections/infrastructure
+- file: sections/access
+- file: sections/usage
+- file: sections/troubleshooting

docs/references.bib

@@ -0,0 +1,11 @@
+@article{JBooks,
+  author    = {Executable Books Community},
+  title     = {{Jupyter Books}},
+  month     = {feb},
+  year      = {2020},
+  publisher = {Zenodo},
+  journal   = {},
+  version   = {0.15.1},
+  doi       = {10.5281/zenodo.4539666},
+  url       = {https://github.com/executablebooks/jupyter-book/releases/tag/v0.15.1}
+}

docs/sections/access.md

@@ -0,0 +1,56 @@
+---
+myst:
+  substitutions:
+    key1: "Accounts should be requested with the motivation: **spoke2 tests**.
+    <br>
+    Once approved, another request is needed, in order to be added to the `highrate` group. Please, use the '*Join a group*' button in the '*Group request tab*', adding the same motivation as before, when requested."
+    key2: |
+      ```{important}
+      {{ key1 }}
+      ```
+    key3: |
+      ```{figure} ../img/iam.png
+      :width: 400px
+      ```
+    key4: |
+      ```{figure} ../img/request.png
+      ```
+---
+
+# Resource Access
+This section will be dedicated to the instructions in order to access the facility.
+
+The entrypoint is https://hub.192.135.24.49.myip.cloud.infn.it and it will redirect the user to the destination JupyterHub, deployed with [WP5 JHub Helm Chart](https://github.com/ttedeschi/HighRateAnalysis-WP5/tree/main/stable/jhub-aas).
+
+Before accessing the hub itself, an authentication step is needed, relying on an IAM-DEMO account. 
+
+## Account Requests
+If the user doesn't already have an account, please use the same authentication page available at the following link: https://iam-demo.cloud.cnaf.infn.it/
+
+|            |                       |
+| ---------- | --------------------- |
+| {{ key3 }} | {{ key2 }} {{ key4 }} |
+
+A short video of the entire procedure is available [here](https://drive.google.com/file/d/1xe7JnEDpsPZBlCtI_krstdwg1aNZ4Xkf/view?usp=sharing)
+
+## Configuration
+Once logged-in, the user has to choose an image to be loaded on the cluster. Choosing a custom image is an option, however some ready-to-use images are available:
+```{list-table}
+:header-rows: 1
+* - *Image*
+  - *System*
+  - *Included Package(s)*
+  - *Default*
+* - [ghcr.io/ttedeschi/jlab:wp5-alma8-0.0.34](https://github.com/ttedeschi/custom-images/blob/0.0.34/jupyterlab/Dockerfile.wp5-alma8)
+  - AlmaLinux8
+  - Python 3.11 + Dask
+  - Yes
+* - [ghcr.io/ttedeschi/jlab:wp5-alma8-0.0.40](https://github.com/ttedeschi/custom-images/blob/0.0.40/jupyterlab/Dockerfile.wp5-alma8)
+  - AlmaLinux8
+  - Python 3.11 + Dask + ROOT 6.30
+  - No
+```
+
+:::{note}
+In both cases, a Dask cluster can be spawned on Kubernetes nodes thanks to the *Dask LabExtension*. When using [ghcr.io/ttedeschi/jlab:wp5-alma8-0.0.40](https://github.com/ttedeschi/custom-images/blob/0.0.40/jupyterlab/Dockerfile.wp5-alma8) image, Dask workers are all deployed with the same ROOT version.
+:::

docs/sections/infrastructure.md

@@ -0,0 +1,6 @@
+# Infrastructure Details
+The resources currently available and exploitable (k8s cluster) are listed below:
+- master node with 8 CPUs, 16 MB RAM, 30 GB disk
+- 9 worker nodes with 16 CPUs, 33 MB RAM, 155 GB disk
+
+Additional details and computing integration(s) will be added in the future, if available

docs/sections/intro.md

@@ -0,0 +1,14 @@
+# High Rate Analysis User Guide
+This guide is meant to be the fundamental reference for all people willing to understand how to use the High Rate Analysis platform, based on INFN Cloud resources. It is built with [Jupyter Book](https://jupyterbook.org) {cite}`JBooks`.
+
+:::{warning}
+Work in progress documentation! In case of comments, questions or issues, please contact [Francesco G. Gravili](mailto:[email protected]) or refer to the GitHub repository linked on top of the page.
+:::
+
+# Acknowledgements
+The content of this guide reflects the efforts of many people: many thanks to everyone who contributed to it!
+
+# References
+```{bibliography} ../references.bib
+:style: unsrt
+```

docs/sections/troubleshooting.md

@@ -0,0 +1,15 @@
+# Troubleshooting and Notes
+Some important points to be kept in mind when using the High Rate Analysis platform:
+- Each user is assigned 10 Gb of persistent storage
+:::{danger}
+Anything stored outside the `persistent-storage` area will be lost when the JupyterLab session ends. Moreover, no redundancy nor backup is in place for the `persistent-storage` area. **ALWAYS BACKUP EVERYTHING**!
+:::
+
+- It is possible to create a custom Docker image, starting from a common [Dockerfile](https://github.com/ttedeschi/custom-images/blob/main/jupyterlab/Dockerfile.wp5-alma8) of the working images.
+:::{attention}
+Dask scheduler and workers have to be spawned both with the custom image. Please, modify [this line](https://github.com/ttedeschi/custom-images/blob/main/jupyterlab/labextension.yaml#L14) accordingly!
+:::
+
+- A monitoring dashboard setup is being prepared: although not ready yet, please report any kind of issues or bugs!
+
+- Is `cvmfs` filesystem needed? Please, provide feedback

docs/sections/usage.md

@@ -0,0 +1,14 @@
+# Examples
+:::{warning}
+These repository will converge into https://github.com/ICSC-Spoke2-repo/HighRateAnalysis-WP5 in the future
+:::
+
+Some examples are available at the following links:
+- CMS Open Data, VBS-like selection: https://github.com/ttedeschi/WP5-examples/tree/main
+- Future Colliders, FCCee: https://github.com/adonofri/INFN\_na\_interactive\_analysis/tree/main
+
+Additional information may be added here, if needed.
+
+## CMS VBS-like analysis
+
+## FCCee analysis