The morning session on Day 3 was on the topic of emerging technologies. Steve Groom began festivities with a talk on Innovative ways of using database technologies. He described the techniques that are used at IRSA to ingest data into their database on a 24-hour turnaround. Data are ingested into partitions, which avoids having to re-index the entire database on a regular database. He also described the various indexing schemes in place, such as HTM, an internally developed “ChunkIndex” for accessing data in multiple positions, and R-tree based indices for extended data sets.
Dmitry Mishin talked about Data sharing and publication using the SciDrive service. SciDrive is a 100-TB cloud based, Dropbox-like system that provides access to astronomy API’s. It is under evaluation by groups of astronomers. The system supports automated extraction of metadata and publishing of data to the VO.
Greg Schwartz spoke about The challenges of data publishing: An AAS journals perspective. A workshop in early 2014 will discuss mechanisms for linking to data sets from AAS journals. This will involve discussing file formats, peer reviews of data, means of discovering data and so on. I was somewhat disappointed that he did not talk more about getting buy-in from scientists, such as convincing them that their citation rate will increase: see, e.g., this report in Science Daily: http://www.sciencedaily.com/releases/2013/10/131001091451.htm.
Mats Rynge spoke about the technologies used in computing a multi-wavelength map of the Galactic Plane in Producing an Infrared Multiwavelength Galactic Plane Atlas using Montage, Pegasus and Amazon Web Services. He described the tools and mechanisms used to compute this Atlas, and detailed the cost and performance. Performance is limited by the rate at which data are extracted from the archive, rather than network speed or processing rates.
Astronomy Computing Today 