This was the title of my talk at the “Innovations in Data Intensive Astronomy” Workshop, held at NRAO, Green Bank, WV from May 3-5 2011. I posted a short article about it last week. This week, I thought I would outline some of the ideas in the talk; some of them are subjects of blog posts, so I will supply links where appropriate.
Astronomy is a data intensive science. There are already 1 PB of public science data accessible from electronic astronomy archives, and this is growing at an accelerating rate of 0.5 PB/yr. Querying and analyzing these data will require a new business model of astronomical computing, as was recognized by many attendees at the meeting. Datasets will be so large that large scale transfer of data to astronomers machines will not be possible, and the processing will have to move to do the data.
So how do we go about evaluating technologies that will be of value to astronomers in this new era? There are, I think, several approaches needed. One of the most important will be to do careful cost and performance evaluations of different technologies to understand their strengths and weaknesses. I cited the examples of cloud computing and the use of GPU’s, both covered in this blog. We will also have to develop tools that help astronomers evaluate the best technology for their purposes; these don’t exist at the moment.
Having to move code to the data will have profound implications for code development. Code will have to consist of portable, scalable, environment agnostic components that can be plugged together to build new applications. I used the Montage mosaic engine as an example of how such code might look.
Finally, the culture of astronomy will change dramatically. Among the changes that I think need to take place are:
- Greater recognition of the role of software engineering: provide career-paths for IT professionals; and next generation software skills should be a mandatory part of graduate education.
- An on-line journal devoted to computational techniques in astronomy.
- Share computational knowledge from different fields and take advantage of it, through an organization such as the U.K. Software Sustainability Institute, a brain trust for software knowledge, if you will.