diff --git a/.github/workflows/create.yml b/.github/workflows/create.yml
index 2e7c79d..c4988dd 100644
--- a/.github/workflows/create.yml
+++ b/.github/workflows/create.yml
@@ -21,7 +21,7 @@ jobs:
- name: Install dependencies and build
run: |
- npm install @iconify-json/clarity @iconify-json/devicon @slidev/cli @slidev/theme-default slidev-theme-hep slidev-theme-neversink
+ npm install @iconify-json/clarity @iconify-json/devicon @slidev/cli @slidev/theme-default slidev-theme-hep slidev-theme-neversink playwright-chromium
npm run build -- --base /${{ github.event.repository.name }}/
npm list
diff --git a/package.json b/package.json
index 7013b49..7ba2baf 100644
--- a/package.json
+++ b/package.json
@@ -3,7 +3,7 @@
"version": "1.0.0",
"main": "index.js",
"scripts": {
- "build": "slidev build",
+ "build": "slidev build --download",
"test": "echo \"Error: no test specified\" && exit 1"
},
"repository": {
diff --git a/public/images/WebApp.png b/public/images/WebApp.png
new file mode 100644
index 0000000..4a0f3f6
Binary files /dev/null and b/public/images/WebApp.png differ
diff --git a/public/images/fcc.png b/public/images/fcc.png
new file mode 100644
index 0000000..333ef35
Binary files /dev/null and b/public/images/fcc.png differ
diff --git a/slides.md b/slides.md
index eb8d46b..6ebb887 100644
--- a/slides.md
+++ b/slides.md
@@ -55,26 +55,57 @@ color: gray-light
---
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Rewritten in Python, using xml-rpc, interfacing to EDG
+ Data Challenge 04 : First successful grid usage ever.
+ : First use of pilot jobs based WMS
+ 2006-2007 : DIRAC3
Full rewriting, development of the DISET protocol -- still in use today!
+ : the current DIRAC framework is still based on this work
+ section Open sourced, wider adoption
+ 2008 : Large-ish reshuffling to become multi-VO
+ : LHCbDIRAC extension separated from Vanilla DIRAC code
+ 2009 : CLIC community adopts DIRAC
+ 2011 : France-Grilles is the first multi-VO DIRAC installation
+ 2012 : Belle2, BES3, CTA adopt DIRAC
+ 2021 : Python3 full support
+```
---
@@ -86,7 +117,7 @@ title: WMS
:: title ::
-# Workload Management System
+# WMS: Workload Management System
- Pull model based on Pilot jobs
- Also "Push" solution for HPCs that do not support pilots (because of limited internet access).
- Will integrate [CWL (Common Workflow Language)](https://www.commonwl.org) as a way of defining jobs (replacing JDL)
@@ -126,15 +157,14 @@ titlewidth: is-5
:: title ::
-# Data Management System
+# DMS: Data Management System
It’s about **files**: placing, replicating, removing files
- there are **LFNs** (logical file names) → and users ONLY work with these
- **LFNs** are registered in *catalog(s)*
- where are the LFNs? (in the DIRAC File Catalog (DFC), or in Rucio)
- what are their metadata? (in the DFC, or in the LHCb Bookkeeping, or in AMGA)
-- LFNs *may* have **PFNs**, stored in **SEs**.
-- You can access those PFNs with several protocols.
+- LFNs *may* have **PFNs**, stored in **SEs**, that can be accessed with several protocols.
:: content ::
@@ -166,12 +196,12 @@ title: Productions
- A `production` is a *Data Processing transformation* (e.g. Simulation, Merge, DataReconstruction…). A Production creates jobs in the WMS (and re-sumbit them, and destroy them).
-- (Unless you are using Rucio) a *Data Manipulation transformation* replicates, or remove, data from storage elements.
+- (Unless you are using Rucio) a *Data Manipulation* `transformation` replicates, or remove, data from storage elements.
:: content ::
- The **Transformation System (TS)** is used to automate common tasks related to production activities
+ The Transformation System (TS) is used to automate common tasks related to production activities. It can handle thousands of productions, millions of files and jobs.
@@ -193,35 +223,34 @@ TransformationSystem((TransformationSystem))
```
---
-layout: top-title
+layout: top-title-two-cols
+align: c-lm-lm
color: gray-light
-align: l
-title: history
+title: tech
---
+
:: title ::
-# Brief history of DIRAC
+# Tech stack
-:: content ::
+:: left ::
+
+- DIRAC is written in python3
+ - with the exception of the DIRAC Pilot, which needs to still be able to run with Python2
+- Services are exposed at urls like `dips://box.some.where:9132/WorloadManagement/`
+ - `dips` stands for "DIRAC protocol"
+- The DIRAC framework provides also "Agents" and "Executors" to animate the system
+- As backends, MySQL and OpenSearch are supported (for different purposes)
+
+:: right ::
+
+DIRAC also provides a WebApp
+
+
+
+Implemented using `ExtJS`, and fully custom Python "bindings"
-```mermaid
-timeline
- section LHCb software
- around 2000 : MC production system: bash scripts running at production sites
- 2002 : DIRAC2
Rewritten in Python, using xml-rpc, interfacing to EDG
- Data Challenge 04 : First successful grid usage ever.
- : First use of pilot jobs based WMS
- 2006-2007 : DIRAC3
Full rewriting, development of the DISET protocol -- still in use today!
- : the current DIRAC framework is still based on this work
- section Open sourced, wider adoption
- 2008 : Large-ish reshuffling to become multi-VO
- : LHCbDIRAC extension separated from Vanilla DIRAC code
- 2009 : CLIC community adopts DIRAC
- 2011 : France-Grilles is the first multi-VO DIRAC installation
- 2012 : Belle2, BES3, CTA adopt DIRAC
- 2021 : Python3 full support
-```
---
layout: side-title
@@ -237,7 +266,7 @@ title: issues
:: content ::
-
+
- complex, with high entrance bar
- somewhat cumbersome deployment
- late on “standards”
@@ -253,14 +282,48 @@ title: issues
- no clear interface to a running DIRAC instance
+
---
layout: section
color: cyan-light
----
+---
## DiracX, the neXt DIRAC incarnation
+---
+layout: top-title-two-cols
+color: gray-light
+align: l-lm-lt
+title: requirements
+---
+
+
+Easy to test (will make it easier to code), but also modern, funny, and accessible to new developers
+
+
+
+:: title ::
+
+# Requirements
+
+:: left ::
+
+## Communities/Users requirements
+
+- Ease of use, including ease to access
+- Fast and responsive interfaces
+- Scalable and flexible
+
+:: right ::
+
+## Administrator requirements
+
+- Ease of installation and update
+- Up-to-date documentation
+- Clear confguration
+- Ready-to-use dashboards
+
---
layout: side-title
side: left
@@ -293,11 +356,11 @@ color: gray-light
# DiracX Web API
-- DIRAC Web APIs are developed using FastAPI
-- Nicely documented in Swagger (or Redoc), this what you see on the right
+- DIRAC Web APIs are developed using FastAP
+- Nicely documented in Swagger (or Redoc), this what you see on the right
-What is on the right is the certification WebApp, loaded live. Use with caution!
+What is on the right is the certification Web API, loaded live. Use with caution!
---
@@ -378,30 +441,97 @@ https://github.com/DIRACGrid/diracx-web
What is on the left is the certification WebApp, loaded live. Use with caution!
+---
+layout: default
+color: gray-light
+---
+
+# Architecture diagram
+
+
+
+---
+layout: quote
+color: sky-light
+quotesize: text-m
+authorsize: text-s
+author: 'Some of you out there'
+---
+
+"OK, but the Grid still uses proxies.
+VOMS is alive!"
---
layout: top-title
color: gray-light
-align: l
+align: lm
title: tokens
---
:: title ::
-# From X509 certs and proxies to tokens
+# On proxies and tokens
:: content ::
-DIRAC uses X509 certificates and proxies to verify identities. DiracX uses tokens. For a (long) while, **users will have both a token and proxy**.
+- **Submitting pilots** : The computing elements right now prefer the tokens (DIRAC v8 does that already)
+- **Data access** : proxies. One day, will be token
+- **Verifying a user's identity** :
+ - **DIRAC** uses only X509 proxies and certificates to verify identities
+ - **DiracX** uses only tokens
+ - --> For a (long) while, **users will have both a token and proxy**.
---
-layout: default
+layout: top-title
color: gray-light
+align: lm
+title: interactions
---
-# Architecture diagram
+:: title ::
+
+# On proxies and tokens/2
+
+:: content ::
+```mermaid
+sequenceDiagram
+ create actor U as User
+ create participant DIRAC_ProxyManager
+ U->>DIRAC_ProxyManager: dirac-proxy-init
+ create participant VOMS
+ DIRAC_ProxyManager->>VOMS: get proxy
+ destroy VOMS
+ VOMS-->>DIRAC_ProxyManager: VOMS proxy
+ create participant IAM
+ DIRAC_ProxyManager->>IAM : get token
+ destroy IAM
+ IAM-->>DIRAC_ProxyManager: IAM token
+ destroy DIRAC_ProxyManager
+ DIRAC_ProxyManager-->>U: proxy+token
+ create participant DIRAC_service
+ U->>DIRAC_service: proxy
+ create participant DiracX_service
+ U->>DiracX_service: token
+ create participant StorageElement
+ DIRAC_service->>StorageElement: proxy
+ DiracX_service->>StorageElement: token
+```
+
+
+
+---
+layout: quote
+color: sky-light
+quotesize: text-m
+authorsize: text-s
+author: 'Again, some of you out there'
+---
+
+"OK, but there are several communities using DIRAC right now.
+
+What should they do?"
---
layout: side-title
@@ -438,6 +568,29 @@ architecture-beta
fastapi:B -- T:clientx
```
+---
+layout: top-title
+color: gray-light
+align: l
+title: Extensions
+---
+
+:: title ::
+
+# On extending DiracX
+
+:: content ::
+
+
+Dirac has to support different Communities with different workflows and requirements. For example:
+- LHCb is a HEP experiment. Stores the metadata and provenance of every produced files in a LHCb-specific database (with an Oracle backend)
+- Belle2 is a HEP experiment. Uses Rucio as data management solution.
+- CTAO is an Astrophysics experiment with radically different requirements (wrt to HEP exp.) on how to process its data
+- WeNMR brings together complementary research teams in the structural biology and life science area
+
+It is sometimes necessary to extend all Dirac(X) components (including Web and Pilot). DiracX aims to provide an easy way to do so.
+
+
---
layout: top-title
@@ -474,39 +627,6 @@ Current production and only supported version, used by all DIRAC installations
DIRAC v9 and DiracX 0.1 will be released together.
----
-layout: top-title
-color: gray-light
-align: l
-title: Extensions
----
-
-:: title ::
-
-# On extending DiracX
-
-:: content ::
-
-The ol'good DIRAC is extendable: from the main code (DIRAC) to the Web and the Pilot, communites can create/run extensions.
-
-DiracX follows the same path, including DiracX-Web. We also provide a reference extension, dubbed "gubbins"
-
-
----
-layout: top-title
-color: gray-light
-align: l
-title: Development
----
-
-:: title ::
-
-# Development phylosophy
-
-:: content ::
-
-
-
---
layout: side-title
@@ -534,12 +654,15 @@ git clone https://github.com/DIRACGrid/diracx-charts
diracx-charts/run_demo.sh # do not do it now!
```
+
+
## Discuss:
- **mattermost** : https://mattermost.web.cern.ch/diracx/
- **meetings**: (almost) every week on Thursday morning (CET)
- **hackathons**: we have been doing 2-days DiracX hackathons every quarter. At CERN. [Next one in January](https://indico.cern.ch/event/1458873/)
+
---
layout: top-title-two-cols
align: cm-cm-lm
@@ -551,7 +674,6 @@ title: summary
# Summary
-
:: left ::
@@ -603,16 +725,6 @@ speed: 0.4
Federico Stagni CERN, LHCb
Andrei Tsaregorotsev CPPM (FR), EGI and LHCb
-
@@ -656,3 +768,29 @@ WebAPI:
+
+
+---
+layout: section
+color: cyan-light
+align: r
+---
+
+## Backup
+
+---
+layout: top-title
+color: gray-light
+align: l
+title: FAQ
+---
+
+:: title ::
+
+Q/A
+
+:: content ::
+
+- I am using Rucio. I could use DiracX as WMS but do not want to fiddle with DIRAC
+It will probably be possible, but we do not know when.
+- The proxy comes from DIRAC