SC 2017 Best Practices in HPC Training

[jduckles]

I think it would be great for this team put in a submission for SC 2017 workshop on HPC training?

Due on August 1st.

https://sites.google.com/a/lbl.gov/hpc-training-best-practices/workshops/sc17

[ChristinaLK]

It would be great - but I'm not sure what we would talk about! We've already submitted an application for a BOF.

Maybe something like "capturing the novice mindset"? Or an invitation to come work on this lesson?

[dbrunson]

How about the invitation to join the discussion? Same basic material as the PEARC talk and what we'll do for the Community Speaks discussion. After all, every previous workshop for this group ends up talking about the Carpentries! (Full disclosure, I'm on the organizing committee.)

[maxim-belkin]

I think it would be great for this team put in a submission for SC 2017 workshop on HPC training?

I think it's a great idea! :)

It would be great - but I'm not sure what we would talk about! We've already submitted an application for a BOF.

One could describe the successes of SWC/DC, link them to carpentries' philosophies and in what ways these philosophies apply to HPC-related training, mention current goals as seen at the moment, and invite to the BoF. The deadline is not a hard one.

Maybe something like "capturing the novice mindset"?

I think one could announce the topics that will be touched upon in the BoF session instead. People in this session might not be aware (🙈) of SWC/DC (well, at least hypothetically 🙄)

Or an invitation to come work on this lesson?

Can I get one? 😅

[shwina]

Can I get one? 😅

At the moment, we have a skeleton for the lesson:

https://github.com/swcarpentry/hpc-novice/tree/gh-pages/_episodes

Feedback/discussion is welcome on the overall skeleton, or on the proposed topics under each. For example, #31.

If you'd like to start adding some actual lesson materials, that would be great too! PRs are welcome.

Edit: I realize this is not the issue to be discussing how/where to contribute. Please feel free to start a new issue or start a discussion on the mailing list (https://groups.google.com/a/carpentries.org/forum/#!forum/hpc-discuss) about this if you'd like!

[ChristinaLK]

@maxim-belkin I essentially gave a talk like you describe at PEARC that I could do at SC.
https://speakerdeck.com/christinalk/a-different-kind-of-carpentry

I'll look at the application tomorrow.

[psteinb]

I support contributing to this workshop as exchanging experience and best practices is always good. However, I personally couldn't provide any more resources than providing feedback to the slides. I also won't be at SC17.

[aturner-epcc]

Just been pointed at this discussion by Tracy Teal. We have put together a SC'17 BoF application to talk about HPC Carpentry and get as much community involvement as possible. People were keen to put together a submission for a presentation at Best Practices in HPC Training that mirrored the BoF and this has now been submitted. Would have asked here first but was unaware of this discussion - apologies for this.

It would be good to have other submissions on specific topics of interest to HPC Carpentry to the workshop. One thought that comes to mind, would be around technologies to support practical HPC learning - at EPCC we have played around with creating a Vagrant VM that contains compilers, libraries, batch system etc so people can take an environment home with them and I think there is useful discussion to be had here around whatever other people have done.

[psteinb]

@aturner-epcc thanks for letting us know. the idea of vagrant VMs is nice but hard to implement in practise in HPC (at least in my environment). Thanks for the encouragement to submit more.

[iamc]

For teaching purposes, you can create a Virtual Machine with Ubuntu as the base OS, and inside it instantiate a cluster made of docker containers. This allows for basic testing of the queue system, simple MPI applications, etc. Of course the learners should be able to run the VM (eg. a VirtualBox one), but it comes down essentially to be able to run a Ubuntu VM, as the containers overhead when testing basic things is very low.

This VM is very easy to setup using MCC (a tool that automates the process of creating virtual clusters whose nodes are containers) following the step by step tutorial here https://github.com/grycap/HPCmeetsDocker , just start inside the Ubuntu VM (you need Ubuntu as everything is based on LXD).

I have seen this setup in action for an HPC talk and it worked flawlessly. I'll point the author of both the tutorial and MCC, @dealfonso to this thread in case you want to ask something to him.

[dealfonso]

Hi all,

I think that, for teaching purposes, it would be better to use mcc. MCC is the short term for "My Container Cluster" and it is a collection of scripts that automate the creation of independent clusters whose working nodes are LXC containers. The only requirement is that you have LXD available. Currently it means that you run MCC in ubuntu. As @iamc said, it is easy to create an Ubuntu based Virtual Machine in which MCC is being installed.

MCC will make most of tasks for you: start the containers, create the private network for them, prepare a shared home folder, create easy and common names for the internal nodes, etc. And it even is able to provide access to the GPU and other features.

Feel free to ask if you need help on MCC.

Regards,
Carlos A.

[aturner-epcc]

MCC looks like a good option and I would be keen to try this out. I could try this on my system and look it in the context of the lessons already proposed and report back on how I get on.

[dbrunson]

Whenever possible, I prefer to have learners learn on the system they will use. For us, that would be our campus cluster. I can see how a virtual or containerized cluster could be helpful for conference workshops where the attendees may not all have accounts on the same physical cluster.

[aturner-epcc]

@dbrunson I think you are correct, this depends on the training perspective. For example, most of EPCC's training is not targetted at specific systems - we are usually training people to use HPC/HTC for them to go and use on a variety of systems.

Having a image-based environment is also useful as once the attendees have it they can take it away with them and continue to learn/explore in their own time.

[maxim-belkin]

Good job, @ChristinaLK! 👍 :)

[ChristinaLK]

thanks! It was good to talk with so many people about this project.

[psteinb]

I'd love to hear more about the event and what people are struggling with and where people want to go. is there any write-up available or a blog post?

[ChristinaLK]

I'm planning to do a write-up and distribute the notes after our BOF today.

[psteinb]

Very much looking forward to it. Have fun at SC17!