The U. S. Virtual Astronomical Observatory project, funded jointly by NSF and NASA since May 2010, will close on September 30, 2014. The project held its close-out review on July 10 and 11, 2014, and you can download the report from the VAO web site.
Perhaps the most interesting part of the report is a discussion of the project’s impact on astronomical research, prepared by Bob Hanisch, and I am reproducing it here:
The impact of the U.S. VO programs on the virtual observatory overall can be seen in a number of ways:
- Significant contributions to at least 35 IVOA standards and documents, from the firstbasic standards and services (VOTable, Simple Cone Search) to sophisticated data models and advanced data access protocols (Table Access Protocol, ObsCore, Simple Image Access Version 2, etc.).
- Leadership of numerous IVOA Working Groups and Interest Groups, as well as lead- ership at the IVOA Executive level.
- A rich infrastructure for data discovery and access, with wide deployment and im- plementation at major data centers in the U.S.
- A robust operational environment in which distributed services are routinely validated against IVOA standards.
- A system of resource registries that enables discover of data and data services through the world.
- Exemplar science applications for data discovery, spectral energy distribution con- struction and analysis, and catalog cross-comparison.
- Desktop scripting tools, including a native Python implementation.
- Cloud-based data storage for collaborative research and simple data sharing with the research community.
- Creation of a “data scientist” position at the American Astronomical Society whose responsibilities include “to help process and manage the increasing volume of digital data and to integrate it within the Virtual Observatory.
- A repository of all VAO products: software, documentation, tutorials, videos, news- letters, etc.
However, it is not easy to measure impact quantitatively. Since the VAO is mostly about the deployment of software tools and infrastructure services, it can be challenging to at- tribute data accesses to the VAO as opposed to the underlying data services. Web applications are primarily entry points to VO services; scripting environments are needed for bulk processing. In the astronomy community at least, and probably in many other disciplines, new software can take many years to penetrate the community, and even then, there is not a strong culture of software citation. For example, we find that although some 22,000 peer-reviewed papers mention the VLA radio telescope, only 68 formally acknowledge the use of AIPS and only 59 acknowledge use of CASA, the two dominant reduction and analysis packages for radio interferometry data. Remarkably (or perhaps not, given the situation for software citation) of over 13,000 peer-reviewed publication in astronomy and astrophysics published in 2013, only 4% acknowledge use of the ADS, and the ADS is probably the most widely-used software system in the field. Thus, counting acknowledgments to VAO or VO tools is unlikely to reflect accurately on community take-up.
On the other hand, VAO usage logs indicate close to one million VO-based data accesses per month at U.S. data providers, and with~100 organizations who have published some 10,000 VO-compliant data services worldwide. VAO usage logs also show ~2,000 distinct users of VAO services in the past three months (April-June 2014). The ADS lists some 2,500 papers (about half of these peer-reviewed) citing “virtual observatory” in some context, and these papers are read as often and cited as often as other types of papers. Of course, without reading each and every paper one cannot be sure of the level of contamination in this sample (a paper saying “our observatory has photometry measurements of virtually thousands of stars” would count as a hit). We have tried to track re- search publications that are truly based on VO-tools and resources, but we do not have the time to do this with any level of completeness. The ~100 papers we are aware of are listed at http://www.usvao.org/support-community/vo-related-publications/.
The journal Astronomy and Computing (Elsevier) is publishing two issues dedicated to VO technology, and VAO team members are contributing a number of papers. The first issue is now in the editorial process with 14 articles in hand, and the second issue is ex- pected to be of comparable size. The VAO team intends to write an article for the second issue describing the accomplishments of the VAO project.
In order to maintain a U.S. national presence at the IVOA, and to help ensure ongoing cooperation between ground-based and space-based data centers in astronomy, we will be establishing a U.S. Virtual Observatory Alliance under the auspices of the AAS Working Group on Astronomical Software.
The VO concept has been adopted in numerous other fields, particular in space science (with seven VxOs within NASA), plus the Virtual Solar Observatory (NASA, NSO), Planetary Science Virtual Observatory (Europe), and the Deep Carbon Virtual Observato- ry (Renssalaer). The VO concept was recently endorsed by a panel of neuroscientists convened by the Kavli Foundation and General Electric as a means for improving access and interoperability to the vast data sets being collected in the European Brain Project and U.S. Brain Initiative. VAO and IVOA participants are now playing leading roles in the international Research Data Alliance and the newly formed U.S. National Data Service.
Needless to say, I disagree on good portions of the above. Case in point the Registry has THOUSANDS of records that point to dead/unreliable links, standards have been developed in isolation from the computer science community, etc. The results are obvious: it was an experiment (started and funded well before 2010) that was only marginally successful. The archives will now have to pick up some of the pieces and hopefully rebuild some interfaces and applications from the ground up listening to their user base this time. This should have perhaps been their role from the start.
My $0.02