On-line Resources for Software Development Practices Aimed At Scientists

A number of people have asked me if I know of on-line resources on the topic of software development practices, aimed at people who are scientists rather than expert software engineers. I am happy to say I know of several very excellent resources, so I am compiling them in this blog post. If readers know of others, please let me know and I will be happy to post them.

A recent paper by Aruliah et al entitled “Best Practices for Scientific Computing” is a very good starting point, and gives an excellent overview of best practices. I described the paper in a recent blog post.  One of the authors of that paper, Greg Wilson, founded the Software Carpentry web page. Version 4.0 of the lessons is a gold mine of information on practical information on everything from testing to version control to databases. Each lesson has several modules leading you through the processes. I particularly like the videos for each lesson. Here is a sample, from the lesson on Unit Testing module in the Testing lesson:

Short, intensive classes are an excellent way to learn the basics of programming and development practices – see the discussion on this topic from the .Astronomy 4 meeting. There are two sets of such classes that I know of. The Software Carpentry project itself now hosts Boot Camps, intensive 2-3 days classes that cover “… the core skills needed to be productive in a small research team: basic programming skills, version control, testing, using the shell, and relational databases. Short tutorials alternate with hands-on practical exercises, and participants are encouraged both to help one another, and to try applying what they have learned to their own research problems during and between sessions.”  The camps are free, and the project asks only that the travel costs of the instructors are reimbursed to them. We had a very successful bootcamp here at Caltech/IPAC, and I recommend them: get in touch with the Carpentry folks if you are interested.

A second “boot camp” is run by the Sci Coder project, managed by Demitri  Muna. This workshop is held each year in the New York area. It is designed ” …to introduce the young researcher to modern programming practices, languages, and tools as specifically applicable to scientific research. These skills will enable participants to write better code, write code faster, and leverage more tools than astronomers are typically introduced to or are aware of.”

Scicoder poster for 2012 scientific coding meeting

The SciCoder project itself is a site “that aims to bridge the gap between research scientists and modern programming practices.” It contains many useful articles on programming and practices. A recent article discussed Moving from IDL to Python, and another discussed Developer Tools on OS X 10.7 (Lion).

Finally, the Software Sustainability Institute  (SSI) has a wealth of on-line material on programming practices. The Institute itself is based in the U.K. and it works with researchers to help them develop software that can be maintained and sustained. The Guides page is particularly useful, and describes recommended practices for releasing code, managing dependency problems, validating code and so on. The information here complements that on the Carpentry and SciCoder site: it emphasizes practices, where the other two sites provide more hand-on programming information.

Section of the SSI Guides page

Section of the SSI Guides page

My recommendation for researchers wishing to improve their practices would be to read Aruliah et al.’s paper to understand the most important points, then go to the Software Sustainability Institute’s Guides to learn about practices in more detail, then go to the Carpentry and SciCoder pages to get down into the trenches. And think about hosting a local boot camp.

This entry was posted in astroinformatics, High performance computing, information sharing, programming, social media, social networking, software engineering, software maintenance, software sustainability, Uncategorized, user communities and tagged , , , , , , , , , , , , . Bookmark the permalink.

2 Responses to On-line Resources for Software Development Practices Aimed At Scientists

  1. Axel Marsh says:

    I had a look to the resources mentioned by you, at some level these practices can be helpful to scientists to be more efficient to the usage of tools. As the technology is advancing in the world of science, the tools and methods are too to which each and every scientist should be aware off atleast at some basic level.

  2. Useful resources are mentioned over here, at some level these practices can be helpful for us as well as for scientists to be more efficient to usage of tools and methods too.So, each of us and every scientist should be aware off these at least at some basic level.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s