From The Front Lines of ADASS – Day 3 Morning

The morning session on Day 3 was on the topic of emerging technologies. Steve Groom began festivities with a talk on Innovative ways of using database technologies.  He described the techniques that are used at IRSA to ingest data into their database on a 24-hour turnaround. Data are ingested into partitions, which avoids having to re-index the entire database on a regular database. He also described the various indexing schemes in place, such as HTM, an internally developed “ChunkIndex” for accessing data in multiple positions, and R-tree based indices for extended data sets.

Dmitry Mishin talked about Data sharing and publication using the SciDrive service. SciDrive is a 100-TB cloud based, Dropbox-like system that provides access to astronomy API’s. It is under evaluation by groups of astronomers. The system supports automated extraction of metadata and publishing of data to the VO.

Greg Schwartz spoke about The challenges of data publishing: An AAS journals perspective. A workshop in early 2014 will discuss mechanisms for linking to data sets from AAS journals. This will involve discussing file formats, peer reviews of data, means of discovering data and so on. I was somewhat disappointed that he did not talk more about getting buy-in from scientists, such as convincing them that their citation rate will increase: see, e.g., this report in Science Daily: http://www.sciencedaily.com/releases/2013/10/131001091451.htm.

Mats Rynge spoke about the technologies used in computing a multi-wavelength map of the Galactic Plane in Producing an Infrared Multiwavelength Galactic Plane Atlas using Montage, Pegasus and Amazon Web Services. He described the tools and mechanisms used to compute this Atlas, and detailed the cost and performance. Performance is limited by the rate at which data are extracted from the archive, rather than network speed or processing rates.

Advertisements
This entry was posted in archives, astroinformatics, Astronomy, Cloud computing, cyberinfrastructure, data archives, Data Management, databases, Grid Computing, image mosaics, informatics, information sharing, Parallelization, programming, Scientific computing, social networking, software engineering, software maintenance, software sustainability, Web 2.0 and tagged , , , , , , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s