Astronomers are well aware of the explosive growth of public data in their field. The same is true of all the physical sciences. The well respected journal Science recently published a special on-line issue of 30 papers on managing data and working with them. The aim was to “provide a broad look at the issues surrounding the increasingly huge influx of research data by showcasing content that highlights both the challenges posed by the data deluge and the opportunities that can be realized if we can better organize and access the data.” The content is free to readers who register with the science website.
I particularly enjoyed Metaknowledge by J. A. Evans and J. G. Foster. They describe how proper harvesting of attributes of data sets can underpin powerful new faceted search algorithms to enable cross-discipline science.