Category Archives: Parallelization

Scientific Workflows for Science and the Science of Scientific Workflows

Posted on June 30, 2017 by astrocompute

In this era of compute intensive astronomy, tools for managing complex workflows are becoming a crucial part of our science infrastructure. This video offers a fine introduction to the Pegasus workflow manager by the team leader, Ewa Deelman (ISI, USC). … Continue reading →

Posted in astroinformatics, Astronomy, Cloud computing, computer modeling, computer videos, Computing, computing videos, cyberinfrastructure, Grid Computing, High performance computing, informatics, information sharing, Parallelization, programming, Scientific computing, software engineering, software maintenance, software sustainability, Uncategorized, workflows | Tagged , , , | 1 Comment

Best Practices for HPC Software Developers

Posted on May 9, 2017 by astrocompute

This is a series of seven videos describing best software engineering practices. I would recommend it to all  HPC software engineers. The series was the result of a collaboration between the IDEAS project, Argonne Leadership Computing Facility, NERSC, and Oak … Continue reading →

Posted in astroinformatics, computer modeling, computer videos, Computing, computing videos, cyberinfrastructure, High performance computing, informatics, information sharing, Parallelization, programming, Scientific computing, software engineering, software maintenance, software sustainability | Tagged , , , | Leave a comment

The Pegasus Workflow Manager and the Discovery of Gravitational Waves

Posted on February 19, 2016 by astrocompute

We have all heard so much about the wonderful discovery of Gravitational Waves – and with just cause! In today’s post, I want to give a shout-out to the Pegasus Workflow Manager, one of the crucial pieces of software used in … Continue reading →

Posted in astroinformatics, Astronomy, Black Holes, Computing, cyberinfrastructure, Gravitational waves, High performance computing, informatics, LIGO, Operations, Parallelization, programming, Scientific computing, software engineering, software maintenance, workflows | Tagged , , , , , , , , , , , , , , | Leave a comment

Exploring the dark universe with supercomputers: article in symmetry magazine

Posted on January 19, 2016 by astrocompute

The other day, I accidentally came across the on-line magazine symmetry (yes, it’s lower case!) which posts articles in the fields of particle physics and related topics, such as dark matter, dark energy, neutrinos, all written for the educated layperson.  I was … Continue reading →

Posted in astroinformatics, Astronomy, computer modeling, Computing, cosmology, cyberinfrastructure, dark energy, DES, galaxy formation, GPU's, Grid Computing, High Energy Physics, High performance computing, informatics, LSST, Parallelization, programming, Scientific computing, simulations, software engineering | Tagged , , , , , , , , , , , , , | Leave a comment

The 8th Extremely Large Databases Conference and Workshop

Posted on July 17, 2015 by astrocompute

These annual workshops that discuss ” …the real-world challenges, practical considerations, and nuts-and-bolts solutions in the realm of managing and analyzing extreme scale data sets.”  The attendees include Big Data users from industry and science, developers, researchers, and providers. This year’s … Continue reading →

Posted in astroinformatics, Cloud computing, computer modeling, computer videos, Computing, computing videos, cyberinfrastructure, Grid Computing, High performance computing, informatics, information sharing, Parallelization, programming, publishing, Python, R, Scientific computing, social media, social networking, software engineering, software maintenance | Tagged , , , , , , , , , , , | Leave a comment

What’s the Difference Between Cluster, Grid and Cloud Computing?

Posted on June 2, 2015 by astrocompute

After my last post on introductory videos on cloud computing, I was asked if there were videos that explained the difference between Cluster, Grid and Cloud Computing. Here is a very good one by Prof. Ajit Pal, Department of Computer Science … Continue reading →

Posted in astroinformatics, Cloud computing, computer videos, Computing, computing videos, cyberinfrastructure, Grid Computing, High performance computing, information sharing, Parallelization, programming, Scientific computing, Uncategorized | Tagged , , , , , , | Leave a comment

The Palomar Transient Factory: High Quality Realtime Data Processing in a Cost-Constrained Environment

Posted on May 5, 2015 by astrocompute

This is the title of a paper by Surace et al. (2015) currently available on astro-ph and presented as a paper at ADASS XXIV in October 2014. The Palomar Transient Factory (PTF) is an example of a cost-constrained project that is … Continue reading →

Posted in astroinformatics, Astronomy, astronomy surveys, cyberinfrastructure, data archives, Data Management, High performance computing, informatics, information sharing, Observatories, Operations, Parallelization, programming, Scientific computing, software engineering, software maintenance, software sustainability, Uncategorized, user communities | Tagged , , , , , , , , , , , , , | Leave a comment

Tools For Astronomical Big Data

Posted on March 12, 2015 by astrocompute

This week, I attended the Tools for Astronomical Big Data Workshop, held in Tucson from March 9 -11 and sponsored by NOAO. Over 100 practitioners gathered to “survey the present state of the art in addressing the challenges of conducting … Continue reading →

Posted in astroinformatics, Astronomy, astronomy surveys, Catalogs, Cloud computing, cyberinfrastructure, Data Management, Data mining, databases, DBMS, Grid Computing, High performance computing, image mosaics, informatics, information sharing, Montage, Observatories, Parallelization, programming, Scientific computing, SDSS, social networking, software engineering, software maintenance, software sustainability, Uncategorized, user communities, visualization, XSEDE | Tagged , , , , , , , , , , , , , , , , , , | Leave a comment

The Front Lines of ADASS 2014 – October 5 2014

Posted on October 5, 2014 by astrocompute

This week, I will be posting about the technical sessions at the Astronomical Data Analysis Software and Systems  XXIV meeting (ADASS, http://adass2014.org) in Calgary, Alberta. This annual meeting ” … is a forum for scientists, developers and programmers working in … Continue reading →

Posted in astroinformatics, Astronomy, astronomy surveys, Catalogs, computer modeling, Computing, cross-matching catalogs, cyberinfrastructure, data archives, Data Management, High performance computing, informatics, information sharing, metadata management, Parallelization, programming, Scientific computing, social networking, software engineering, software maintenance, software sustainability | Tagged , , , , , , , , , , , , , , | Leave a comment

SciPy2014 videos on-line

Posted on August 7, 2014 by astrocompute

This year’s annual Scientific Computing with Python conference, SciPy2014, was held in Austin ,TX, July 6-12 2014.  This conference “… allows participants from academic, commercial, and governmental organizations to showcase their latest Scientific Python projects, learn from skilled users and developers, … Continue reading →

Posted in astroinformatics, Astronomy, computer modeling, computer videos, Computing, computing videos, cyberinfrastructure, Data mining, High performance computing, informatics, information sharing, Parallelization, Python, Scientific computing, SciPy2014, social media, social networking, software engineering, software maintenance, software sustainability, user communities | Tagged , , , , , , , , , , , , | Leave a comment