Software Carpentry Workshop at the 225th AAS Meeting

Many of you have heard of the Software Carpentry Foundation, started some years ago by Greg Wilson, a non-profit organization whose members teach researchers basic software skills. The organization exists to teach fundamental software engineering practices to scientists, who in many fields spend as much as 40% of their time on programming, yet most are self-taught.  One of the organization’s most successful teaching methods is software boot camps, immersive, intensive workshops lasting one or two days whose content is customized to the needs of the audience.  These boot camps are taught by volunteers, generally scientists who have themselves attended a boot camp: to date, 81 instructors have taught 265 workshops for 9,000 learners. And the instructors themselves are usually not programming experts: they are people who have learned valuable skills at boot camps and want to pass this knowledge to others.

I attended my first boot camp at the 225th American Astronomical Society Meeting in Seattle. I have an interest in how to best educate astronomers in software engineering skills and on developing sustainable software, so I wanted to see first-hand the dynamics of how the boot camps work. And of course, I could pick up some new software skills.  The pic below shows us hard at work, being taught the essentials of Python by Erik Bray.

photo copy

The class took place over two days, Jan 3 and Jan 4, though I was only able to attend on Jan 3, and was taught by Azalee Bostroem, Erik Bray, Matt Davis, and Phil Rosenfield ably assisted by Pauline Barmby.  The topics covered were, with more detail on the workshop page at http://abostroem.github.io/2015-01-03-aas/:

  • Automating tasks with the Unix shell (Day One)
  • Building programs with Python (Day One)
  • Code Review (Day One)
  • Version control with Git (Day Two)
  • Python for Astronomers (Day Two)

Preparation was straightforward.  A few days before the meeting, we were asked to fill out a short questionnaire to allow the instructors to assess the skill level of participants, and to install required software on our laptops, including “all-in-one” scientific Python installer Anaconda.  I had some trouble installing all the Python packages, but was able to solve the problems quickly enough using the troubleshooting guide. The instructors were on hand one hour before class started to help participants sort out any installation problems.

The instruction was by design hands-on and interactive.  In general, the instructors showed us on the projector how to perform various tasks, and we followed along on our laptops. Then we carried out various exercises by ourselves or small groups, usually taking 5 to 10 minutes. The instructors circulated to help out anyone who requested assistance. Finally, we went over at least one solution on the projector.

A complete record of all activities was kept in an Etherpad file (not sure if this is or will remain public),  shared among all the participants, who took advantage of it to ask questions that were responded to quickly bu the instructors.  The Etherpad record, as well as all the class materials, allow us to go back and reproduce the exercises, or go back and do them for the first time when things went a little too quickly.

Matt Davis introduced the Unix shell is a powerful tool for automating operations on files. We broadly followed the on-line tutorial at http://software-carpentry.org/v4/shell/ and there is a Unix reference guide at http://abostroem.github.io/2015-01-03-aas/novice/ref/01-shell.html.

The Python part of the syllabus was intended to introduce us to the important concepts in programming. The lesson plan is on-line at http://abostroem.github.io/2015-01-03-aas/intermediate/python-review/index.html,  and the class notes and materials are at http://nbviewer.ipython.org/github/abostroem/2015-01-03-aas/blob/gh-pages/intermediate/python-review/python-full.ipynb.The lessons were carried out by constructing an iPython notebook. By the end of the session, we were able to

  1. Describe and distinguish the seven core elements shared by all programming languages.
  2. Use Python to write simple programs that use these core elements, using both the core library and scientific packages such as numpy.
  3. Make and save simple publication-quality plots using matplotlib (and APLpy, time permitting).
  4. Read, manipulate, and save data files in csv and text formats.
  5. Create standalone Python scripts that can be run from the command line.

These seven core elements referenced in item 1 are:

  1. Individual things (the number 2, the string ‘hello’, a matplotlib figure)
  2. Commands that operate on things (the + symbol, the len function)
  3. Groups of things (Python lists, tuples, and dictionaries)
  4. Ways to repeat yourself (for and while loops)
  5. Ways to make choices (if and try statements)
  6. Ways to create chunks (functions, objects/classes, and modules)
  7. Ways to combine chunks (function composition)

Matt Davis led us in a coder review session. We split up into groups of three or four to give our critiques of a common Python script, and compared notes at the end. The exercise was intended to show how even rapid code reviews by third parties can improve the quality of a piece of code.

So what did I learn?  I am not a Python expert, but I learned basic Python during the lesson, and I was impressed with the Python notebook for keeping a record of my efforts. The lesson on code review was instructive and I think participants would realize the value of them (as a manager of software projects, I am an advocate of code reviews). Chatting to participants afterwards, there was strong agreement that scientists can learn important software techniques through these immersive classes, and gain the confidence to learn more. If you want to get the most out of these classes, be prepared for a day or two of sustained hard work: the boot camp moniker is apt. Finally, I am impressed by Software Carpentry’s  business model for sustaining these classes: bootstrapping itself to develop a cadre of instructors and sustain its body of expertise.

Advertisements
This entry was posted in astroinformatics, Astronomy, Computing, cyberinfrastructure, education, informatics, information sharing, Open Source, programming, publishing, Python, Scientific computing, social networking, software engineering, software maintenance, software sustainability, user communities and tagged , , , , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s