[Meta] Domain-specific material

[ahmadia]

[Update] Nov 2, 2013 - We are currently not accepting domain-specific material into the bc repository. Please see @gvwilson's post on Scope and this guiding statement:

Domain-specific tools for machine learning and computational fluid dynamics aren't built into LAPACK or NumPy: they're stored in separate repositories and released in their own packages. Similarly, domain-specific lessons don't belong in Software Carpentry. Instead, they should build on our lessons, just as computer vision libraries build on matrix and image libraries. Our examples should be drawn from biology, physics, economics, and similar disciplines in order to appeal to our audience, but everything we include must be accessible to anyone who's done freshman science.

Original issue for context

Per #110, I'd like to ensure that we have a consistent strategy with regards to adding domain-specific material to the bc repository.

Just as we have a consistent set of core SWC and THW lessons with some additional reproducibility and bioinformatics material, I would really like to have some domain-specific content in the bc repository that lives alongside the core SWC materials. There are strong advantages to introducing students to material that is flavored towards their own experiences and needs, and I want bc to be a place where we can collect suitably general, well-edited, lessons for explaining this.

I don't want to discuss this in #110, which is a specific PR, but I will hold off on merging anything in until we've agreed on this.

[rgaiacs]

I like the idea of domain-specific content in the bc repository.

We just need avoid add to much material that we couldn't maintain. So we could add domain-specific content related only to Python (or R) and keep the shell, git, db and web general.

[jdblischak]

I agree that domain-specific content is great for motivation. My suggestion would be to have one main general lesson file, e.g. python_functions.ipynb, that would be used by all boot camps. Then each discipline could have a separate exercise file, e.g. python_function_geosciences.ipynb, that would contain exercises with jargon and data specific to that audience. This way we aren't trying to maintain many different introductions to writing functions (and of course we can all benefit from our collective efforts to improve the material), but can also offer a domain-specific experience for each particular boot camp.

[ethanwhite]

@ahmadia thanks for starting this issue! Definitely an important discussion to have. I completely agree that domain specific content is valuable for motivating learners and I always teach with domain specific content (including shell, git, and databases).

I think there are two key questions to think about in terms of this discussion:

1. Scope: To me teaching core computational skills with domain specific context (e.g., looping over species vs. galaxies) is well within scope and if we can do this in a maintainable way that would great. What seems less clear to me is whether teaching domain specific skills is within scope. My feeling is that this has generally been out of scope for SWC, with SWC teaching the core skills so that folks can build their domain specific skills on top of them. Specifically with respect to bootcamps it seems to me that getting into really domain specific skills in 2 days when we're already covering shell, git, python/R, and testing seems like it would be difficult to do well.
2. Maintainability: As both @r-gaia-cs and @jdblischak wisely point out we need to think carefully about what we can maintain. I really like John's suggestion regarding a single core notebook and separate exercise notebooks. Ideally things would work in such a way that the examples in the core notebook could be programatically swapped out for properly structured domain specific examples, but I suspect that's technically a bit much to hope for at the moment.

So, to go back to my original comment, my quick read of #110 is that most of the material duplicates existing content (and not in a particularly domain specific way in most cases) and that the remaining material is primarily about domain specific skills rather than domain specific context for core skills. That said, I certainly could have missed something.

[ahmadia]

@ethanwhite - thanks for clarifying your position. I agree that there is redundant material in #110, and I need to make a clear case for what content in there goes into bc and why from there.

I really like his basemap examples for matplotlib, for example.

http://nbviewer.ipython.org/urls/raw.github.com/koldunovn/python_for_geosciences/master/05%2520-%2520Graphs%2520and%2520maps%2520%2528Matplotlib%2520and%2520Basemap%2529.ipynb

Obviously, this isn't going to be for starting-from-zero scientists, this material is targeted at scientists with some GIS visualization experience (climate, oceanography, environmental, geosciences) that are interested in seeing how to do something similar in Python.

It's not core material (we don't teach visualization as a core skill), but I think it would a perfect second-day lesson for a geosciences-themed bootcamp. Compare it to our existing THW material:

http://nbviewer.ipython.org/urls/raw.github.com/swcarpentry/bc/gh-pages/lessons/thw-matplotlib/matplotlib-learner.ipynb

[wking]

On Tue, Oct 29, 2013 at 06:55:59AM -0700, Ethan White wrote:

1. Maintainability: As both @r-gaia-cs and @jdblischak wisely point
   out we need to think carefully about what we can maintain. I really
   like John's suggestion regarding a single core notebook and separate
   exercise notebooks. Ideally things would work in such a way that the
   examples in the core notebook could be programatically swapped out
   for properly structured domain specific examples, but I suspect
   that's technically a bit much to hope for at the moment.

This is why I like @jiffyclub's approach from
swcarpentry/DEPRECATED-boot-camps@fda4a1a of
condensed instructor notes as the core content for live coding. Then
we can support as many stand-alone, domain-specific exercises as we
feel like maintaining. For example, see my restructured-modular-shell
branch 1. I haven't had time to translate this to my #102 framework
yet, but there's a start in 2.

This works because the condensed instructor notes aren't dropped in
front of the students; they just follow the instructor in a live-coded
notebook/shell. This means you can point them at any exercise you
like without having to alter a central notebook/lesson to bind the
domain-specific exercises. Of course, the #102 framework allows us to
support student-facing generic lessons or lessons that are tightly
bound to domain-specific exercises if folks want to maintain that. I
just don't think we have the time/energy to try and keep all that in
sync.

[ahmadia]

Sorry for starting this fire. @gvwilson and @wltrimbl have convinced me mostly off-list that we need to focus on core materials, and that domain-specific materials are not currently appropriate, particularly now that we are launching a second reorganization.

[wking]

On Sat, Nov 02, 2013 at 12:30:59PM -0700, Aron Ahmadia wrote:

@gvwilson and @wltrimbl have convinced me mostly off-list that we
need to focus on core materials, and that domain-specific materials
are not currently appropriate, particularly now that we are
launching a second reorganization.

If that's a transition from "I'm thinking of writing some new
domain-specific material" to "I'm thinking of helping assemble the
generic core material", then +1 ;). However, I see no need to drop
all of our existing domain-specific material just because it no longer
has a home in the core bc repository. As I understand Greg's scope
notes 1, he's happy linking folks to external affiliated
repositories/branches with domain-specific content that they can pull
in if their audience is up for it. That sounds like a call for
#102-style per-topic/excercise branching to me, so I think we should
spin off our existing domain-specific material into it's own
repositories.

[ahmadia]

Yes, the plan is to find homes outside of swcarpentry for domain-specific material right now.