UpskillingResearchersInML.qmd

---
title: "Upskilling Researchers in Machine Learning"
# subtitle: "Lightning talk ⚡️"
author: "Maxime Rio, Jens Brinkmann"
institute: "New Zealand eResearch Infrastructure (NeSI), The University of Auckland (UoA)"
date: 2023-10-17
date-format: full

# bibliography: refs.bib
from: markdown+emoji
# embed-resources: false
format:
  revealjs:
    self_contained: true
    # tbl-cap-location: bottom
    # number-sections: true
    theme: UoAtemplate.scss
    multiplex:
      id: 'a140d30bbdb06469'
      secret: '16962778612049606149'
    # css: ./logo.css
    # disableLayout: true
    navigation-mode: vertical
    controls-layout: bottom-right
    controls-tutorial: true
    transition: convex
    view-distance: 10
    width: 1600
    height: 900
    margin: 0.1
    # logo: "./UoALogoDarkBlueRGBLandscape.png"
    title-slide-attributes:
        data-background-image: NeSIAndUoA.jpg
        data-background-size: 20%
        # data-background-postion: right
        data-background-position: 95% 76%
        # data-background-position: bottom 10px right 20px
    # background-image: ./ResBaz_transparent_Logo_cropped.svg
    # background-opacity: 0.5
    # background-position: bottom 10px right 20px
    # background-size: contain
    # data-background-repeat: no-repeat
    # background-size: 80px
    # background-repeat: no-repeat
    # background-position: 0% 100%
    # logo: ResBaz_transparent_cropped.svg
    # self_contained: false
    reveal_plugins: ["menu"]
    reveal_options:
      menu:
        numbers: true
    header: Upskilling Researchers in Machine Learning
    header-logo: NeSIAndUoA.jpg
    hide-from-titleSlide: all
    filters:
      - reveal-header
      - line-highlight
    
editor:
  render-on-save: true
execute: 
  enabled: true

--- 

<!-- based on [this](https://conference.eresearch.edu.au/guidelines-presenter/?utm_source=sendgrid.com&utm_medium=email&utm_campaign=website): *Oral Presentations (15 minutes plus 5 minutes for questions and changeover): Short conversation starters which provide enough information to encourage the audience to engage and seek further information.* -->

# Who are we?

:::: {.columns}
::: {.column width=45%}

**Maxime Rio**

![](./UpskillingResearchersInML_Assets/flying_max.jpg){width=40% fig-align="center"}

- *Data Science Engineer @ NeSI*
- *Data Scientist @ NIWA*
- Help researchers optimise and scale-up <br> their code
- Develop ML pipelines and models
- Organise ML and data science **training**

:::

::: {.column width=55%}

**Jens Brinkmann**


![](./UpskillingResearchersInML_Assets/Jens_2022.jpg){width=30% fig-align="center"}


- *Senior eResearch Engagement Specialist @ UoA*
- Mechanical Engineer with a background in Photography/Videography
- Support researchers with their computational needs and training around that 



:::
::::

# About this talk


- We want to tell you about our **experience** with Machine Learning (ML) workshops.
- We want to share some **recommendations**.
- *You can do it!*

<!-- MAKE SURE SPACE IS HERE -->


- The [Lightning Talk earlier today](https://conference.eresearch.edu.au/developing-a-carpentries-style-machine-learning-workshop/) was mainly about answering *what* we did and providing metrics.
- [This current talk](https://conference.eresearch.edu.au/upskilling-researchers-in-machine-learning/) is focused on the **delivery**.
- [BoF](https://conference.eresearch.edu.au/ai-skill-training-pathway-bridging-gaps-and-fostering-inclusivity/) (right after this talk) will be a *broader discussion*


# A shared goal

:::: {.columns}
::: {.column width=50%}

- Introduce researchers to Machine Learning and Deep Learning
- Start with **foundational** ML aspects and build up from there
- Research-field agnostic
- **Hands-on** approach (teaching by doing)
  - **no** show and tell of existing commercial solutions
  - **no** theoretical lectures
  - **no** deep dive into the maths behind ML
- Not an exhaustive course but make attendees confident to try things by themselves

:::

::: {.column width=50%}

![](./UpskillingResearchersInML_Assets/BingAIDalle3.jpeg){width=70% fig-align="center"}

*Figure 1: [AI Created Photo (DallE3, Bing)](https://www.bing.com/images/create/a-cool-photo-showing-machine-learning/6525c46fdd2b4dabb631b07a2a527ba6?id=BeZ%2bVfneQKjueVSP0cGXoQ%3d%3d&view=detailv2&idpp=genimg&idpclose=1&FORM=SYDBIC&ssp=1&setlang=en-NZ&safesearch=moderate).*
<!-- had to be an AI generated image, we can adjust -->

:::
::::

# Workshops overview

<!-- goal: give some credibility to our recommendations and advise about recruitment process -->

## UoA workshops

:::: {.columns}
::: {.column}

- Audience: The University of Auckland (UoA) researchers
- Two runs
  - Run #1: March 2023
  - Run #2: September 2023
- Well-received
  - filtering by mandatory Expression of Interest (EoI)
  - about 100 applications for 40 spots

:::

::: {.column}

![](./UpskillingResearchersInML_Assets/ZoomParticipantsLine.svg)

:::
::::


## NeSI workshops

:::: {.columns}
::: {.column}

![](./UpskillingResearchersInML_Assets/ml101_1.jpg)
*First ML101 workshop at eResearch NZ 2021*

:::

::: {.column}

- Audience: Aotearoa -- NZ researchers
- ML 101
  - Intro to Machine Learning
  - started in 2021
  - 7 workshops (in person, online)
  - 127 attendees in total (from 10 to 32)
- ML 102
  - Intro to Deep Learning (CNNs)
  - started in 2022
  - 2 workshops (online)
  - 44 attendees in total (20 and 24)
- Mixture of direct registration and EoIs

:::
::::

<!-- numbers (check with Nisha & Matt for ML102)
ML101 2021 eRNZ ~10 
ML101 2021 Greta 19
ML101 2021 CRIs 32
ML101 2021 Uni 22
ML101 2022 Pradeesh 11
ML101 2023 May 18
ML101 2023 August 15
ML102 2022 June 20
ML102 2023 July 24
-->

## Recommendations {.center}

There will be **a lot** of interest so...

- use an Expression of Interest for registration and filter,
- 30 participants is a good number for an online training,
- expect people to not show up (if free and online).

# Platform(s)

## UoA workshops

:::: {.columns}
::: {.column}

- online only event: [Zoom.us](https://zoom.us/)
- BYOD (*bring your own device*)
- major deviation from [The Carpentries](https://carpentries.org/index.html): No local Python installs
- [Goolge Colab](https://colab.research.google.com/) ![Colab](./Google_Colaboratory_SVG_Logo.svg){ width=3% },a browser-based Jupyter Notebook using Google infrastructure (a virtual machine; a GPU can be added)

 

:::


::: {.column}
![*Google Colab in a Browser*](./UpskillingResearchersInML_Assets/colab2.jpg){fig-align="center"}
:::
::::

## NeSI workshops

:::: {.columns}

::: {.column}

<!-- TODO use a better capture on a larger screen -->
![](./UpskillingResearchersInML_Assets/penguins.png)
*JupyterLab session running on Jupyter-on-NeSI*

:::

::: {.column}

- Online and in person
  - 2 delivered in person (1 had wifi issues 😓)
  - 5 delivered online
- Use Jupyter-on-NeSI
  - JupyterHub platform
  - Requires a NeSI account
  - ML101: 2 cores & 4 GB of RAM
  - ML102: 4 cores & 8 GB of RAM
- Use Slurm-based job for GPU training (a little bit)
- Tip: make sure the Platform team does not schedule upgrades that day 😬...

:::
::::

## Recommendations {.center}

- Make it online
- Leverage online computational platforms (Google Colab, JupyterHub, Open OnDemand...)
- No need for GPU to start (or small ones on Google Colab available)

# Schedule

<!-- so this is purely about the hours and the split of content over these hours -->

## UoA workshops

:::: {.columns}
::: {.column}

**Run #1** 

| Time Budget           | Activity                 |
|-----------------------|--------------------------|
| two afternoons (8h)   | Python                   |
| one afternoon (4h)    | ML                       |
| two afternoons (8h)   | DL                       |

**Run #2 **

| Time Budget           | Activity                 |
|-----------------------|--------------------------|
| two afternoons (8h)   | ML                       |
| two afternoons (8h)   | DL                       |


:::
::: {.column}

- all workshops took place in the **same week**
- no *mixing and matching*, signing up = coming to **all sessions**
- Major adjustment for Run #2: **Python** as a **prerequisite**, not part of the series

:::
::::

## NeSI workshops

:::: {.columns}
::: {.column}

- ML 101
  - 6 hours with 3 breaks ☕
  - at first in one day
  - now split over 2 mornings
- ML 102
  - 3 hours with 2 breaks 🍵
- Independent workshops
- But organised "close" to each other

:::
::: {.column}

![](./UpskillingResearchersInML_Assets/ml101_runsheet.png)
*ML101 runsheet, used to keep track of time*

:::
::::

## Recommendations {.center}

- Split/shorter sessions (bearing in mind the scheduling challenges for researchers)
- Stick to scheduled breaks
- Follow best practices for online audiences:
  - get a Zoom DJ, some helpers, get multiple co-hosts
  - keep a QA document
  - prepare your intro and outro
  - make attendees join from the same computer running the code
  - [Webinar: Tips & tricks for hosting a successful online event](https://youtu.be/XTeCHUZ2H_w?si=OD4GDRNSV460zK7O)

# Material

## UoA workshops


| Lesson Title                                       | Status                                                                                       | Run #1   | Run #2                    |   |
|----------------------------------------------------|----------------------------------------------------------------------------------------------|----------|---------------------------|---|
| Programming with Python                            | [Released](https://swcarpentry.github.io/python-novice-inflammation/)                        | Mon, Tue |             -             |   |
| Introduction to Machine Learning with Scikit Learn | [Alpha](https://mike-ivs.github.io/machine-learning-novice-sklearn/02-regression/index.html) | Wed |  Mon, Tue |   |
| Introduction to Deep Learning                      | [Beta](https://carpentries-incubator.github.io/deep-learning-intro/aio.html)                 |    Thu, Fri   |  Wed, Thu |   |

::: {layout="[[-1], [1], [-1]]"}
![](UpskillingResearchersInML_Assets/Carps.jpg){fig-align="center"}
:::

## NeSI workshops

:::: {.columns}
::: {.column}

![](./UpskillingResearchersInML_Assets/jake.png)
*My rehearsal and source of inspiration 💓*

:::
::: {.column}

**NeSI workshops**

- ML 101 -- [github.com/nesi/sklearn_tutorial](https://github.com/nesi/sklearn_tutorial)
  - [Scikit-learn Tutorial](https://github.com/jakevdp/sklearn_tutorial) by 
[Jake Vanderplas](https://github.com/jakevdp)
  - online recordings exist <br> (very good to rehearse!)
  - Jupyter Notebook based
  - very few changes (updated package version)
- ML 102 -- [github.com/nesi/ml102_workshop](https://github.com/nesi/ml102_workshop)
  - TensorFlow tutorials
  - Jupyter Notebook based
  - added an introduction
  - added section to submit a Slurm job

:::
::::

## Recommendations {.center}

CeR and NeSI independently decided to base the workshops on **existing material**.

- Don't reinvent the wheel
- Reuse/adapt content

<!-- should we mention notebooks? -->

# Content

<!-- this differs from the previous perspective as it is more about the abstract topics covered
     IS THIS PREVIOUS STATEMENT WHAT YOU HAD IN MIND? yes :) -->

## {.center}

:::: {.columns}
::: {.column width="60%"}

**Machine Learning**

- *Data preparation*
- Supervised vs. unsupervised learning
  - regression
  - classification
  - clustering
  - dimensionality reduction
- Ensemble models (random forests)
- Validation
  - train/test/validation split
  - cross-validation
  - validation and learning curves

:::
::: {.column width="40%"}

**Deep Learning**

- Model architectures
  - Multi-layer perceptron
  - Convolutional neural network <br> (CNN | computer vision)
- Model training
  - optimisers and mini-batch
  - overfitting and early stopping
  - data augmentation
  - dropout, batch normalisation, ...
- Transfer learning

<!-- we should include the DeepLearning topic here, too? Or was the point for you to show: This was covered by both of us?
     we should add this one, good point
-->

:::
::::

## Recommendations {.center}

- **Random forest** is a **good** first non-linear model to learn
  - intuitive to understand how it works
  - good performances on tabular data
  - doesn't require too much care in terms of data preparation

- Resist the temptation of **MLP (multi-layer perceptron)** for an ML intro
  - require more notions (architecture, training, data preprocessing, ...)
  - keep it for Deep Learning introduction

# Summary

- Use Expression-of-Interest for registrations
- Make it online
- Use an online compute platform 
  - (*Google Colab*, *JupyterHub* or *Open OnDemand* at your institution)
- You don't need fancy GPUs, even though you can have some
- Re-use and adapt existing material
- Use shorter sessions and stick to breaks
- Keep MLP for the Deep Learning section, start with Random Forests


# How to get in touch {.center}

:::: {.columns}
::: {.column width="60%"}

![](./UpskillingResearchersInML_Assets/kereru-taieri-mouth-1920-2.webp)
*Kererū / New Zealand pigeon, not use(ful) for mails* 😅
*[Image Credit: Department of Conservation](https://www.doc.govt.nz/nature/native-animals/birds/birds-a-z/nz-pigeon-kereru/)*

:::

:::{.column width="40%"}

**Maxime Rio** <br> 📨 [maxime.rio@nesi.org.nz](mailto:maxime.rio@nesi.org.nz)

**Jens Brinkmann** <br> 📨 [j.brinkmann@auckland.ac.nz](mailto:j.brinkmann@auckland.ac.nz)

:::
::::

<!-- maybe a QR code or two? -->