This is the title of a fine presentation given by Arvind Gopu at the SPIE Meeting in Montreal, Canada on June 22nd. The One-Degree Imager (ODI) is installed in the WIYN 3,5m telescope at Kitt Peak (Tucson, AZ). It contains 13 Orthogonal Transfer Array (OTA) detectors, and has been in full science operations since March 2013, and produces 0.25 TB to 1 TB of data in a 5 night run. These volumes are expected to double in 2015 with the installation of a 5 x 5 array of detectors.
In the interests of space, I will restrict this post to a discussion of the underlying architecture. Arvind also described the end-user services, and I will talk about these in a later post. The ODI is a collaboration between WIYN partners, NOAO and the Indiana Pervasive Technology Institution (IU PTI).
ODI-PPI is a framework of frameworks to archive data, and incorporate analysis and reduction tools and make data and services accessible through web interfaces. The diagram below summarizes the architecture:
The essential feature is that distributed, lightweight services are tied together by the Advanced Message Queuing Protocol (AMQP) based message bus. Core services are deployed on RedHat 6.5 Linux KVM based virtual machines, and the system uses a MySQL Replicated Cluster, Rabbit MQ Cluster, NSF disks as well as compute nodes. The project takes advantage of Indiana University Compute Resources such as Petaflop-scale processors and a 42 PB archiving tape system.
At the backend, data are transferred from Kitt Peak to Indiana. A Python based data ingestion system stages data and extracts metadata for ingestion into the archive. The system uses object oriented code using a Model/View/Controller (MVC) design pattern, incorporated in a PHP/Zend framework.
Finally, Tier 2 (processing) components are incorporated into a Tier 2 framework, shown below, that allows integration of any application that can be executed on the Unix command line. Functionality available in this framework includes stacking and source extraction, along with quick look calibration.
