[Review]: Intro Docker

[sstevens2]

Lesson Title

Reproducible Computational Environments Using Containers: Introduction to Docker

Lesson Repository URL

https://github.com/carpentries-incubator/docker-introduction

Lesson Website URL

https://carpentries-incubator.github.io/docker-introduction/

Lesson Description

This lesson aims to teach researchers how to use and create containers using Docker. It requires knowledge of navigating the unix shell and using a text editor but otherwise intended for indviduals with no experience using using Docker or containers.

Author Usernames

@dme26 @jcohen02 @ChristinaLK @aturner-epcc @sstevens2

Zenodo DOI

No response

Differences From Existing Lessons

To our knowledge this lesson is not similar to any of the lessons in the Carpentries Lab or Lesson programs.

Confirmation of Lesson Requirements

• is the original work of the author(s), or that any content derived from another source is reused with permission and appropriate attribution
• aligns with The Carpentries Code of Conduct
• is published under a CC-BY or CC0 license
• uses The Carpentries lesson template or Carpentries Workbench without significant customisation/adaptation.

JOSE Submission Requirements

• the lesson repository includes paper.md and paper.bib files as described in the JOSE submission guide for learning modules

Potential Reviewers

mkuzak colinsuaze gcapes mmalenta

[tobyhodges]

Thanks for submitting this lesson to The Carpentries Lab, @sstevens2, @dme26, @jcohen02, @ChristinaLK, and @aturner-epcc.

I'll be acting as Editor on this submission, and I aim to work through the Editor checklist before the end of next week. You can expect further posts to this thread after I have finished that but, given the proximity of the northern hemisphere winter holidays, a search for reviewers will need to wait for the new year.

For now, to ensure that the review process runs as smoothly as possible, please make sure you are subscribed to receive notifications from this thread. On the right sidebar of this page you should see a section headed Notifications, with a Customize link. You can click on that and make sure that you have the Subscribed option selected, to receive all notifications from the thread.

You can add a badge to display the status of the review in the README of your lesson repository with the following Markdown:

[![The Carpentries Lab Review Status](http://badges.carpentries-lab.org/15_status.svg)](https://github.com/carpentries-lab/reviews/issues/15)

[tobyhodges]

Editor Checklist - Reproducible Computational Environments Using Containers: Introduction to Docker

Accessibility

• All figures are also described in image alternative text or elsewhere in the lesson body.

Note for Reviewers: that there are no figures in the main body of the lesson, but there are a few screenshots in the Using Docker with Github Actions extra episode.

• The lesson uses appropriate heading levels:
  • h2 is used for sections within a page.

The Introduction and Introducing the Docker Command Line ("Docker command line" section) episodes, and the Container Orchestration and Creating Containers on the Cloud extras use third-level headings for sections (heading lines beginning with ### ). Please adjust these to second level headings.

• no “jumps” are present between heading levels e.g. h2->h4.
• no page contains more than one h1 element i.e. none of the source files include first-level headings.

The Using Docker with Github Actions extra episode contains several top-level headings (heading lines beginning with a single #) for sections within the page. Please adjust all of the heading levels in this extra episode down one level, # to ##, ## to ###, etc.

• The contrast ratio of text in all figures is at least 4.5:1.

Content

• The lesson teaches data and/or computational skills that could promote efficient, open, and reproducible research.
• All exercises have solutions.
• Opportunities for formative assessments are included and distributed throughout the lesson sufficiently to track learner progress. (We aim for at least one formative assessment every 10-15 minutes.)
• Any data sets used in the lesson are published under a permissive open license i.e. CC0 or equivalent.

Design

• Learning objectives are defined for the lesson and every episode.
• The target audience of the lesson is identified specifically and in sufficient detail.

Note for Reviewers: the Instructor Notes extra contains several learner profiles and a further discussion of the potential audiences for the lesson.

Repository

The lesson repository includes:

• a CC-BY or CC0 license.

The repository does contain a CC-BY license, but the copyright statement inside should be updated to remove mentions of The Carpentries lesson programs. I suggest replacing it with:

... your work is derived from work that is Copyright © [authors' names] and, where practical, linking to https://carpentries-incubator.github.io/docker-introduction/), ...

and

... the example programs and other software provided in the lesson are made available under the ...

• a CODE_OF_CONDUCT.md file that links to The Carpentries Code of Conduct.
• a list of lesson maintainers.
• tabs to display Issues and Pull Requests for the project.

Structure

• Estimated times are included in every episode for teaching and completing exercises.
• Episodes lengths are appropriate for the management of cognitive load throughout the lesson.

Supporting information

The lesson includes:

• a list of required prior skills and/or knowledge.
• setup and installation instructions.
• a glossary of key terms or links out to definitions in an external glossary e.g. Glosario.

Please add key terms to the glossary in the Reference extra. You can include terms and their definitions, or list only terms and link from these to their respective definitions in Glosario.

General

• Please adjust the contact email address (email in the _config.yml file) to point to someone who people can contact about the curriculum.

[tobyhodges]

@sstevens2, @dme26, @jcohen02, @ChristinaLK, and @aturner-epcc I have completed the editorial checks and identified only a few points I would like you to address, which you can find in the checklist response above.

Please take some time to work through and address these. You may find it helpful to open an issue for each point on the lesson repository, to help you keep track.

Please post back here after you have addressed my comments. You do not have to respond with an itemised list of all the changes you made - it is sufficient to let me know when you are ready for me to run through the checklist again before I look to assign reviewers for the lesson.

If you have any questions or concerns about any of the points I raised, please post back here. I'd be happy to answer questions and provide more details as you require.

Note that I will be on leave for the next few weeks. I will pick this thread back up when I return.

[jcohen02]

Many thanks for running through the editorial checks and providing comments @tobyhodges. We'll work through and address these comments and let you know when this has been completed. Have a good break.

For future reference, the following issues have been opened to address the above comments:

• Fix heading levels in several episodes carpentries-incubator/docker-introduction#195
• Update copyright notice carpentries-incubator/docker-introduction#196
• Add key terms to the glossary in the reference section carpentries-incubator/docker-introduction#197
• Update contact details in _config carpentries-incubator/docker-introduction#198

[sstevens2]

To my knowledge, we've closed all the issues @jcohen02 made to address issues @tobyhodges brought up with items missing from the checklist. Next steps?

[tobyhodges]

Great, thank you for the update @sstevens2. I've run through the editorial checklist again and agree that everything is looking good and the lesson is ready for review.

The next step is for me to begin looking for reviewers - a task that I will begin tomorrow.

[tobyhodges]

Thanks everyone for your patience. After some delay, I am delighted to invite @flor14 and @nanjalaruth to review this lesson. Florencia & Ruth, please can you post here to confirm that you are happy to review this lesson?

You can read more about the lesson review process in our Reviewer Guide.

[flor14]

I am ready to review @tobyhodges ! :)

[nanjalaruth]

Dear Toby, I am happy to review! Regards, Ruth.

…

On Mon, Sep 18, 2023 at 11:31 AM Toby Hodges ***@***.***> wrote: Thanks everyone for your patience. After some delay, I am delighted to invite @flor14 <https://github.com/flor14> and @nanjalaruth <https://github.com/nanjalaruth> to review this lesson. Florencia & Ruth, please can you post here to confirm that you are happy to review this lesson? You can read more about the lesson review process in our Reviewer Guide <https://github.com/carpentries-lab/reviews/blob/main/docs/reviewer_guide.md> . — Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANGRBX3RHMO7ZSLBHX2Y7MDX3APH3ANCNFSM6AAAAAASYVRG5Y> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[tobyhodges]

Excellent, thank you both. When you are ready, please post your reviews as replies in this thread. If you have any questions for me during the review, please ask.

[nanjalaruth]

Dear All,

I have gone through the lesson and reviewed it based on the checklist. I have also added a few minor comments to the document. See attached.

[nanjalaruth]

Reviewer Checklist

Accessibility

• The alternative text of all figures is accurate and sufficiently detailed.
  • Large and/or complex figures may not be described completely in the alt text of the image and instead be described elsewhere in the main body of the episode.
• The lesson content does not make extensive use of colloquialisms, region- or culture-specific references, or idioms.
• The lesson content does not make extensive use of contractions (“can’t” instead of “cannot”, “we’ve” instead of “we have”, etc).

Content

• The lesson content:
  • conforms to The Carpentries Code of Conduct.
  • meets the objectives defined by the authors.
  • is appropriate for the target audience identified for the lesson.
  • is accurate.
  • is descriptive and easy to understand.
  • is appropriately structured to manage cognitive load.
  • does not use dismissive language.
• Tools used in the lesson are open source or, where tools used are closed source/proprietary, there is a good reason for this e.g. no open source alternatives are available or widely-used in the lesson domain.
• Any example data sets used in the lesson are accessible, well-described, available under a CC0 license, and representative of data typically encountered in the domain.
• The lesson does not make use of superfluous data sets, e.g. increasing cognitive load for learners by introducing a new data set instead of reusing another that is already present in the lesson.
• The example tasks and narrative of the lesson are appropriate and realistic.
• The solutions to all exercises are accurate and sufficiently explained.
• The lesson includes exercises in a variety of formats.
• Exercise tasks and formats are appropriate for the expected experience level of the target audience.
• All lesson and episode objectives are assessed by exercises or another opportunity for formative assessment.
• Exercises are designed with diagnostic power.

Design

• Learning objectives for the lesson and its episodes are clear, descriptive, and measurable. They focus on the skills being taught and not the functions/tools e.g. “filter the rows of a data frame based on the contents of one or more columns,” rather than “use the filter function on a data frame.”

• [] The target audience identified for the lesson is specific and realistic.

• I am not sure I saw a section highlighting the target audience, has this been mentioned?

Supporting information

• The list of required prior skills and/or knowledge is complete and accurate.
• The setup and installation instructions are complete, accurate, and easy to follow.
• No key terms are missing from the lesson glossary or are not linked to definitions in an external glossary e.g. Glosario.

General

• Introducing containers - Is it possible to show a graphical representation of how containers are created from Dockerfile?
• Finding containers on Docker Hub - Is it also possible to talk about container registries like Quay and DockerHub in general before focusing on DockerHub? Also, docker images can be directly built on Docker Hub if you have a Dockerfile on GitHub. Is this possible to add this bit or will it be out of the scope for the lesson?
• Examples of Using Container Images in Practice - I understand the lesson is Docker only, but is it possible to insert a link showing an example of how to run docker images using singularity on the HPC cluster?

[flor14]

I will finish my review by Oct 15th

[flor14]

Hello! Here is my review.
There is a lot of work in the lesson and has been easy to follow it. Congratulations to the authors for all your hard work 🎉
I have included comments but please feel free to use them to improve the lesson as you think is better.

Reviewer Checklist

Accessibility

• This item doesn't apply to the lesson, there aren't images in the lesson The alternative text of all figures is accurate and sufficiently detailed.
  • Large and/or complex figures may not be described completely in the alt text of the image and instead be described elsewhere in the main body of the episode.
• The lesson content does not make extensive use of colloquialisms, region- or culture-specific references, or idioms.
• The lesson content does not make extensive use of contractions (“can’t” instead of “cannot”, “we’ve” instead of “we have”, etc).

Content

• The lesson content:
  • conforms to The Carpentries Code of Conduct.
  • meets the objectives defined by the authors.
  • is appropriate for the target audience identified for the lesson.
  • is accurate.
  • is descriptive and easy to understand.
  • is appropriately structured to manage cognitive load.
  • does not use dismissive language.
• Tools used in the lesson are open source or, where tools used are closed source/proprietary, there is a good reason for this e.g. no open source alternatives are available or widely used in the lesson domain.
• This item doesn't apply to the lesson, Any example data sets used in the lesson are accessible, well-described, available under a CC0 license, and representative of data typically encountered in the domain.
• This item doesn't apply to the lesson, The lesson does not make use of superfluous data sets, e.g. increasing cognitive load for learners by introducing a new data set instead of reusing another that is already present in the lesson.
• The example tasks and narrative of the lesson are appropriate and realistic.
• The solutions to all exercises are accurate and sufficiently explained.
• The lesson includes exercises in a variety of formats.
• Exercise tasks and formats are appropriate for the expected experience level of the target audience.
• All lesson and episode objectives are assessed by exercises or another opportunity for formative assessment.
• Exercises are designed with diagnostic power.

Comments

• Motivation: It can be beneficial to create a concrete example to develop the exercises for the persona you are developing the lesson for.
  For example, you need a Docker container to reproduce a plot from an old article or to use a Docker container to run X scientific software that has a different OS than the one the researcher has installed on the computer. It is difficult to think of good examples but it can be more meaningful and motivating to place the student in a concrete situation. Only a plot or text appearing on the computer with a random phrase after running a container can make the lesson more attractive.
• There aren't solutions or answers for exercises in chapter 7.

Design

• Learning objectives for the lesson and its episodes are clear, descriptive, and measurable. They focus on the skills being taught and not the functions/tools e.g. “filter the rows of a data frame based on the contents of one or more columns,” rather than “use the filter function on a data frame.”
• The target audience identified for the lesson is specific and realistic.

Comments:

• Chapter 5: The objective 'Explain how the Docker Hub augments Docker use' is not clear to me.

Supporting information

• The list of required prior skills and/or knowledge is complete and accurate.
• The setup and installation instructions are complete, accurate, and easy to follow.

Comment:

• I couldn't find installation instructions (maybe it is me?). UBC-MDS maintains a set of installation instructions that include Docker for Mac, Linux, and Windows that maybe you can cite. Here is the example for Mac. Or maybe even linking the documentation will be enough.
• Probably you are aware of the multiple issues that the new Macs M1/M2 with the new arm64 processor can bring. Even if things are better now, this could be an issue.
  In my opinion experience, there are 3 moments when this could be a problem in this lesson:

1. If a Mac M1/M2 user wants to build or run a container available only for amd64 will have to add --platform linux/amd64 to the command. Read here.

2. If you are selecting an image in DockerHub, better to use one that is available for both architectures. This way, it would not be necessary to specify anything as Docker detects which one is necessary for the computer making the request. Last year this was an issue for rocker images but not for JupyterLab ones. In general, the popular and well-maintained repositories offer both:

3. Build multiplatform Docker images to push to DockerHub: I would add it as supplemental material. In the references section of this link, there are listed articles about how to create them link.
   Final note: this is based on what I read at the end of 2022, there could be more recent updates.

• No key terms are missing from the lesson glossary or are not linked to definitions in an external glossary e.g. Glosario.
  Comments:
• In the first chapter, I can read "virtual" and "pipeline". Wouldn't be better to link the glossary for a more exact description of those terms instead of quoting them?
• In Chapter 2 you start talking about DockerHub without defining it, you are going to talk more about it later. Maybe an analogy can help here?
• I used Docker container and Docker image instead of 'Container Image' for teaching because it seems less confusing for me. For example, the title of Chapter 6 'Creating Your Own Container Images' would be for me 'Creating your own Docker Images'. I know that some people use it your way, so whatever decision you make about this check that is the same in the Glosario. One option could be to read how these terms are referred to in the official documentation.

General comments

• I appreciate that you clarified the old and new syntax, which can be very confusing.

• All the analogies you used made the topics quite clear to understand.

• Chapter 3: The explanation of the command ls as well as your description of what means interactive in the context of the lesson is great, and will be useful for the learners.

• The video is a really interesting resource. Would you recommend the instructors to start the lesson with it or is it extra material?

• You explain very abstract topics. Do you think that any image of a diagram could help the students to learn? In the video suggested at the start, they are bringing some representations that could be re-used again in the lesson.

• The exercise time reported for some of the lessons is 0 min, even if you have exercises. This happens at least in chapters 1 and 2.

• You explain some abstract topics in this lesson. So you think that any image of a diagram could help the students to learn? In the video suggested at the start, they are bringing some representations that could be re-used again in the lesson.

• Chapter 4: Could be a good moment to check if the students recognized the difference between an image and a running container. I would ask directly in an exercise if to avoid memory issues they should delete the container or the image and which command they will use to do so.

• Chapter 7: "How can I make more complex container images?"
  This title isn't very specific. Probably you use the commands you explain there because you have I will propose questions at the level of detail of the examples/commands you explain in the chapter. The same for the title "More fancy Dockerfile options". The title could be slightly more informative.

** Workflow/Good practices?**
In chapter 6 there is a title that says 'Boring but important notes about the installation': I think this section is far more important than what is mentioned in the title. Basically, you are proposing a workflow to develop a Dockerfile.

• You introduce the flags --rm and --name in a previous chapter. Would you suggest using them as part of a Docker use workflow?

Reproducibility

• Chapter 9: Something that is interesting regarding Docker container's reproducibility is the use of tags for the base images. You mention how to use them and that is not mandatory as well, but it could influence from the reproducibility side. Another topic that could be important to mention regarding reproducibility is whether it is convenient to pin the version of the Python - R - .... packages they want to use inside the containers. I know that there are no guidelines about how to do this (you said that in the lesson), but at least mentioning some benefits or cons of doing it (or not) could help future Docker users make their own decisions regarding this topic.
  Also, using Docker images that are multiplatform can help with reproducibility as well.

What could be added as part of the topics you are covering?

• One concept that could be an obstacle for the students is the idea of Docker layer. The order of Dockerfile instructions matters. This is a source of confusion and probably a good topic to include as an exercise as well. For example, you can ask what is the OS of one specific Python image they are using.

[tobyhodges]

Thank you both very much for your reviews, @nanjalaruth and @flor14.

@sstevens2, @dme26, @jcohen02, @ChristinaLK, and @aturner-epcc: Reviews are now complete, and you can proceed with making changes and responding to the comments and suggestions whenever you are ready. To help you keep track of the improvements you can make based on these reviews, you may find it helpful to convert individual comments from reviewers, or groups of related comments, into issues on the lesson repository.

@nanjalaruth & @flor14: thank you again for volunteering your time to review the lesson. Please stay subscribed to this thread, so that you can check changes made in response to your review, and in case the lesson developers need to discuss any of your comments while they incorporate the feedback.

[jcohen02]

Many thanks @tobyhodges, and, indeed, many thanks @nanjalaruth and @flor14 for taking the time to review our Docker lesson. As suggested, we'll go through the review comments and feedback, grouping points as necessary, and open issues for the comments which we can then work to address.

[sstevens2]

Thanks @tobyhodges for facilitating this review! Also many thanks to the reviewers (@nanjalaruth and @flor14) for your thoughtful review! Looking forward to incorporating your suggestions into the lesson!

@jcohen02 Could you take the lead on convening the maintainers to make issues and co-work on them? We might need a couple sessions but could start with an hour to review and make issues?

[aturner-epcc]

@jcohen02 I know you have a lot on at the moment - do you want me to do this coordination?

[jcohen02]

Thanks @aturner-epcc and apologies, this is on my todo list but I got caught up with some other things. If you have the bandwidth available to take this on, that would be much appreciated. Equally, if things are very busy on your end at the moment too, leave it with me.