So Just What Is Cloud Computing?

I have written some blog posts about running science applications on the cloud, such as What Types Of Applications Are Best Run On The Cloud? and Why Don’t You Use The Cloud?

Someone pointed out that I had never really defined cloud computing. Well, there are many answers to this question, and this is perhaps what confuses people. Perhaps appropriately, the answer is a little nebulous … (sorry, that was bad). I have always thought of it from an end user’s perspective as simply buying compute resources as you need them from a third-party vendor.  But there is more to it than that. For one thing, many people can now build their own clouds and not have to pay any usage charges. After a spot of Googling, possibly the best definition I found comes from the SearchCloudComputing web site. They define cloud computing as “… a general term for anything that involves delivering hosted services over the Internet.”

There are, I think, three characteristics of cloud computing:

  • Services are made available on demand and, for a commercial cloud, users only pay for what they use
  • The cloud is “elastic,” in that resources are made available as they are needed,   and
  • The service is managed by the provider; end users simply access it from their desktop machines via the internet.  The provider may offer to its customers any or all of the following:
  1. Infrastructure  – a complete system such as a system team manages locally for an organization
  2. Platforms – an environment you would use as you would a desktop, or
  3. On-line software service  – such as access to a data processing tool, or a search engine.

What makes all of this possible?  There have been many improvements in distributed computing in the past decade, but the key to cloud computing is virtualization. This is the same technology that you can use on your desktop to run, say, Windows from your Apple computer: it involves the complete simulation of a platform in a machine that is created and stored as a file on the host platform. The creation of virtual machines, for instance, is what allows resources to be allocated on demand.

This is a very simple description. If you are interested in a more technical description, see the Wikipedia page on cloud computing, or download the article “Cloud 101” by Allisa Taylor.

Advertisements
This entry was posted in astroinformatics, Cloud computing, cyberinfrastructure, High performance computing, Parallelization and tagged , , , , , . Bookmark the permalink.

6 Responses to So Just What Is Cloud Computing?

  1. Steve B says:

    It is worth noting that the notion of a compute cloud is also often used interchangeably (perhaps recklessly) with a grid, the shift in nomenclature resulting from namespace pollution of the latter term during the early 2000s. As well, many organizations have what they term internal clouds, where internal users can access a variety of resources (often matched to or augmented with external resources as well). As an example, consider a lab (e.g., NOAA) with significant internal resources — a ‘traditional’ computing cluster (racks of Intel- and/or AMD-based compute nodes, serving work under a scheduler), some GPU resources (for rendering, visualization, or computing, and perhaps also available via a scheduler), large SMP machine (say, 256-processor), and perhaps another cluster with fast (e.g. Infiniband) interconnects for specialized tasks. These resources are available to staff via a hierarchical queue system, but if additional resources are needed, NOAA might contract with Amazon EC3 or MS Azure for additional computing resources, hence joining the internal “cloud” with the external. Among interesting developments here are workflow mappers (such as Pegasus, which Bruce has experience with) that provide a common abstraction above these diverse resources, and enable desk jockeys to simply define the types of computational horsepower (“I need 100,000 Infiniband-linked cores”) needed, rather than the specific resources (“I need all of Jaguar for a year”).

    Like the term “grid”, however, cloud computing is also beginning to suffer namespace difficulties, especially in industry, and thus I applaud readers in their effort to find suitable, working definitions. Clouds and grids share many common features, with an important distinction being the funding model providing those resources: a user or organization pays a cloud provider directly, while grids often operate under cooperative agreement under the aegis of a funding agency, like Teragrid. That’s not to say one can’t budget cloud resources into your grant proposals — on the contrary, it is perhaps advisable to do so, especially for postdocs or faculty at smaller institutions without access to large HPC installations.

  2. astrocompute says:

    Steve:

    Thank you for this thoughtful comment – you managed to almost write my next post for me! I approach the definition of grids, clouds from the point of view of how I can use them, mainly because I am end-user rather than an architect. I am thus not sure I completely agree with the mechanism of payment as the distinction between grids and clouds. Rather, I think that elasticity is a defining characteristic of clouds. They are good for workflow applications in particular. Grids I think of in the classical sense: a cluster that uses software to manage jobs and resource allocation, and “finds” and allocates unused resources to run a job. They are very powerful for running “tightly-coupled” applications.

    • Steve B says:

      Interesting comment Bruce! I agree with you, but would extend your definition of elasticity a bit. Grids (even in the ‘classical sense’) are elastic, allowing a user to potentially stretch their computational domains significantly using non-local resources. However, they certainly the rapid response to changes in project requirements that clouds can provide; the computational “spring constant”, if you will, is much larger for clouds than classic grids.

      One other note to make is in regard to storage clouds, which can be useful tools under certain circumstances. A recent article in Network World tests several cloud storage providers, giving some cost details as well:
      http://www.networkworld.com/reviews/2010/082310-cloud-storage-test.html

  3. astrocompute says:

    Yes, interesting point – perhaps “responsive” might be a better term than “elastic.” I tend to think that with clouds, I can get resources when I need them, literally on-demand. Grids do indeed involve more shenanigans to allocate more resources.

    The link to storage costs was interesting, and maybe I should go back and look at storage costs again. When we did our cost benefit study in 2009, cloud storage was prohibitively expensive for I/O bound applications. It wasn’t just storage, it was storage and moving data. See my earlier post on this topic at
    http://wp.me/pU7SH-Y.

    Have providers made their storage costs more attractive recently?

    • Steve B says:

      With a few more competitors, prices are starting to drop slowly, but I don’t believe that they are much lower than they were, and companies are starting to explore new business models. Of course, the current models charge a premium for getting one’s data *out* of the cloud!

      In your previous post, and the article it references, I didn’t see a cost analysis relative to hosting these resources internally, say over a archetypal three-year lifespan of HPC hardware under normal usage, including costs for maintenance, replacement parts, and human-hours. It’s an analysis that some organizations need not worry about, though!

  4. astrocompute says:

    I think slowly is the operative word! And yes, for Montage the transfer-out costs were the killer. Actually, I do have a cost analysis, and I will make it the subject of a blog in week or so – it would be to much to put in a comment.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s