Hosting & Cloud Computing: Numbers Don't Lie

There's lots of chatter in the market today regarding the value of using outside data centers, hosting services and cloud computing.  But listening to pundits/analysts trying to objectively predict true value left me hollow.

While I'm not an investment professional, I do know that the stock market doesn't lie.... so instead, I thought I'd look at a bundle of stocks from publicly-traded data center companies in the data center space, and compare against a market benchmark.

I chose companies on publicly-traded markets in both the US as well as in Europe. My criteria were somewhat subjective, but basically the companies had to have a primary business operating data centers. I also excluded Telcos because it is difficult to separate their carrier revenues relative to hosting revenues. So, my initial "virtual fund" consists of 12 companies: Digital Realty Trust; DuPont Fabros; Equinix; Internap; Iomart; Macquarie Telecom; Navisite; Rackspace; Savvis; Switch & Data; Telecity; Terremark.

I also took a 5-company subset of these public companies that had significant offerings in the cloud computing space (Equinix; Navisite; Rackspace; Savvis; Terremark). I labeled this "virtual fund" a cloud-only index.

The chart at right is my best attempt to (a) tabulate historic end-of-month closing price of each stock; (b) calculate month-to-month percentage gains for each; (c) create "virtual funds" where $100 would be invested equally across each vehicle (initially $8.33 in each of the 12 hosting stocks, and initially $20 in each of the 5 cloud-related stocks). The benchmark I used is the Nasdaq index, also assuming an initial $100 investment.

Not surprisingly (for me, anyway) both "indexes" are outperforming the Nasdaq -- perhaps proving the thesis that datacenter operation and application outsourcing is indeed a growth market (or at least a speculative growth market?) as compared to the general technology market. What would be equally useful (but not an analysis I've done) is to chart gross revenues for the Index companies. This would be a telling barometer of actual business.

I'll continue to update this index at the end of each month. Comments, additions and suggestions welcome!

Emergence of Fabric as an IT Management Enabler

Last week I attended Gartner's annual Data Center Conference in Las Vegas. Four days packed with presentations and networking (of the social kind). Lots of talk about cloud computing, IT operations, virtualization and more.

Surprisingly a number of sessions directly referenced compute Fabrics -- including "The Future of Server Platforms" (Andy Butler), "Blade Servers and Fabrics - Evolution or Revolution" (Jeff Hewitt), and "Integrated Infrastructure Strengths and Challenges" (Paquet, Dawson, Haight, Zaffros). All very substantive analyses of what fabrics _are_... but very little discussion of why they're _important_. In fact, Compute fabrics might just be the next big thing after OS virtualization.

Think of it this way: Fabric Computing is the componentization and abstraction of infrastructure (such as CPU, Memory, Network and Storage). These components can then be logically re-configured as-needed. This is very much analogous to how OS virtualization componentizes and abstracts OS and application software stacks.

However, the focus by most fabric-related vendors thus far is simply on the most fundamental level of fabric computing, which is simply virtualizing I/O and using a converged network. This is the same initial level of sophistication when the industry believed that OS visualization was only about the hypervisor. Rather, we need to take a longer view of fabric computing and think about higher-level value we create by manipulating the infrastructure similar to how we manipulate VMs. A number of heady thinkers supporting the concept of Infrastructure 2.0 are already beginning to crack some of these revolutionary issues.

Enter: Fabric as an Enabler

If we think of "fabric computing" as abstraction and orchestration of IT components, then there is a logical progression of what gets abstracted, and then, what services can be constructed via logically manipulating the pieces:

1. Virtualizing I/O and converging the transport

This is just the first step, not the destination. Virtualizing I/O means no more stateful NICs and HBAs on the server; rather, the I/O presents itself to the OS as any number of configurable devices/ports, and I/O + data flow over a single physical wire. Transport can be Ethernet, FCoE, Infiniband, or others. In this manner, the network connectivity state of the physical server can be simplified and changed nearly instantaneously.

2. Virtual networking

The next step is to define in software the converged network, its switching, and even network devices such as load balancers. The result is a "wire-once" physical network topology, but with an infinitely reconfigurable logical topology. This permits physically flatter networks. Provisioning of the network, VLANs, IP load balancing, etc. can all be simplified and accomplished via software as well.

3. Unified (or Converged) Computing

Now things get interesting: Now that we can manipulate the server's I/O state and its network connections, we can couple that with creating software-based profiles of complete server configurations -- literally defining the server, its I/O, networking, storage connections, and even what software boots on it. (Software being either a virtual host, or a traditional native OS). Having defined the entire server profile in software, we can even define the entire environment's profile.

Defining servers and environments in software allows us to provide (1) High Availability: With a hardware failure, we can simply re-provision a server configuration to another server in seconds -- whether or not that server was running a VM host, or a native OS. (2) Disaster Recovery: we can re-constitute an environment of server profiles, including all of their networking, ports, addresses, etc., even if that environment hosts VMs and native OS's.

 4. Unified Management

To achieve the ultimate in an agile IT environment, there's one remaining step: To orchestrate the management of infrastructure with the management of workloads. I think of this as an ideal Infrastructure-as-a-Service -- physical infrastructure that adapts to the needs of workloads, scaling up/out as conditions warrant, and providing workload-agnostic HA and DR.  From an IT agility perspective, we would now be able to abstract nearly all components of a modern data center, and logically combine them on-the-fly as business demands require.

Getting back to the Gartner conference, I now realize one very big missing link -- while Gartner has been promoting their Real-Time Infrastructure (RTI) model now for some time, they have yet to link it to the coming revolution that will be enabled by fabric computing.  Maybe we'll see some hint of this next year.

Infrastructure Virtualization: The Next Logical Step

2010 will be an interesting year for virtualization - but not from the perspective you're probably thinking. It will be the year of the virtual infrastructure, not of the virtual machine.
Yes, the O/S virtualization market is maturing as it transforms how servers and applications are managed. The major vendors all offer hypervisors and management to accomplish server consolidation, live migration, HA, lifecycle management, lab management, and more. And they're even offering higher-level tools for DR and cloud computing... Read more on VMBlog.com

Infrastructure 2.0 – A Virtual Analogy

Is OS virtualization an end in itself? Is it both necessary and sufficient for all things Cloud and IaaS? Is it the panacea IT Operations has been looking for? From where I see it, abstracting the OS is certainly a great start, but it’s actually only 50% of the goal.

To a degree, OS virtualization is the “shiny metal object” de jure in that it’s captivating everyone’s attention. It is of course very valuable, and is causing an important inflection point in datacenter operations and economics. But there is a less-visible, less sexy side to datacenter operations and economics that lies “below” the CPU in the stack...

Read more on the Infrastructure 2.0 Blog

Differing Target Uses for IT Automation Types

One of the most oft-repeated themes at this year's VMworld was that of "automation." Everybody claimed they had it, but on closer investigation it had any number of poorly-defined meanings.

A specific angle I want to address here is that of infrastructure automation; that is, the dynamic manipulation of physical resources (virtualized or not) such as I/O, networking, load balancing, and storage connections - Sometimes referred to as "Infrastructure 2.0". Why is this important? Although automation of software (such as provisioning & manipulation of VMs/applications) usually captures attention, remember that there is a whole set of physical datacenter infrastructure layers that IT Ops has to deal with as well. When a new server (physical or virtual) is created, much of this infrastructure also has to be provisioned to support it.

There are 2 fundamental approaches to automation I'll compare/contrast: Let's loosely call them "In-Place" Infrastructure Automation, and Virtualized Infrastructure Automation.

Confession: I am a champion of IT automation. The industry has evolved into a morass of technologies and resulting complexity; the way applications (and datacenters) are constructed today is not the way a greenfield thinker would do it. Datacenters are stove-piped, hand-crafted, tightly-controlled and reasonably delicate. Automating how IT operates is the only way out -- hence the excitement over cloud computing, utility infrastructure, and the "everything-as-a-Service" movement. These technology initiatives are clear indications that IT operations desires a way to "escape" having to manage its mess.

At a high-level, automation has major top-level advantages: Lower steady-state OpEx, greater capital efficiency, and greater energy efficiency. And, automation also presents challenges typical of paradigm changes: distrust, organizational upheaval, financial and business changes. The art/science of introducing automation into an existing organization is to reap the benefits, and mitigate the challenges.

As infrastructure automation moves forward, it appears to be bifurcating along two different philosophies. Each is valid, but appropriate for differing types of uses:

• "In-place" infrastructure automation: (distinct from run-book automation) Seeks to automate existing physical assets, deriving its value from masking the operational and physical complexity via orchestrating in-place resources. That is, it takes the physical topology (servers, I/O, ports, addressing, cabling, switches, VMs etc.) and orchestrate things to optimize a variable such as an SLA, energy consumption, etc.
• Virtualized Infrastructure automation: Seeks to first virtualize the infrastructure (the assets as above) and then automate their creation, configuration and retirement. That is, I/O is virtualized, networking is frequently converged (i.e. a Fabric), and network switches, load balancers, etc. are virtualized as well.

Each of these two approaches has properties with pros and cons with which I'm familiar -- having worked for companies in each space. I'll try to elucidate a few of the "high points" for each:

"In-Place" Infrastructure Automation:
Examples: Cassatt (now part of CA), Scalent

• Automates existing assets: Usually, there is no need to acquire new network or server hardware (although not all hardware will be compatible with the automation software). Thus "in-place" assets are generally re-purposed more efficiently than they would be in a manually-controlled scenario. Clearly this is one of the largest value propositions for this approach - automate what you already own.
  
• Masking underlying complexity: A double-edged sword, I suppose, is that while "in-place" automation simplifies operation and streamlines efficiency, the datacenter's underlying complexity is still there - e.g. the same redundant (and sometimes sub-optimal) assets to maintain, same cabling, same multi-layer switching, same physical limitations, etc.
  
• Alters security hierarchy: Since assets such as switches will now be controlled by machine (i.e. the automation SW automatically manipulates addresses and ports) this architecture will necessarily modify the security hierarchy, single-point-of-failure risks, etc. All assets fall under the command of the automation software controller.
  
• Broad, but not complete, flexibility: Because this approach manipulates existing physical assets, certain physical limitations must remain in the datacenter. For example, physical server NICs and HBAs are what they are, and can't be altered. Or, for example, certain network topologies might not be able to be perfectly replicated if physical topologies don't closely match...or, if physical load balancers aren't available, servers/ports won't have access to them. Nonetheless, if properly architected, some of these limitations can be mitigated.
• Use with OS virtualization: This approach usually takes control of the VMM as well, e.g. takes control of the VM management software, or directly controls the VMs itself. So, for example, you'd allow the automation manager to manipulate VMs, rather than vSphere.
  
• Installation: Usually more complex to set up/maintain because all assets, versions, and physical topography necessarily need to be discovered and cataloged. But once running, the system will essentially maintain its own CMDB.
  

Virtualized Infrastructure Automation:
Examples: Cisco UCS, Egenera, Xsigo

• Reduction/elimination of IT components: The good news here is that through virtualizing infrastructure, redundant components can be completely eliminated. For example, only a single I/O card with a single cable is needed per server, because they can be virtualized/presented to the CPU as any number of virtual connections and networks. And, a single virtualized switching node can present itself as any number of switches and load balancers for both storage and network data.
  
• Complete flexibility in configuration: By abstracting infrastructure assets, they can be built/retired/repurposed on-demand. e.g. networking, load balancing, etc. can be created at-will with essentially arbitrary topologies.
  
• Consistent/complementary to OS Virtualization models: If you think about it, virtualized infrastructure control is pretty complementary to OS virtualization. While OS virtualization logically defines servers (which can be consolidated, moved, duplicated, etc.), infrastructure virtualization similarly defines the "plumbing" and allows I/O and network consolidation, as well as movement/duplication of physical server properties to other locations.
• New networking model: One thing to keep in mind is that with a completely virtualized/converged network, the way the network (and its security) is operationally managed changes. Organizations may have to re-think how (and who) creates and repurposes network assets. (Somewhat similar to coping with "VM Sprawl" in the software virtualization domain)
  
• Use with OS virtualization: This approach is usually 'agnostic' to the software payload of the physical server, and is therefore neutral/indifferent to the VMM in place. Frequently the two can be coordinated, however.
  
• Installation: Usually relatively simple. Few components per server, few cables, especially in a 'green field' deployment. Installation of software/BIOS on physical servers is probably not what you're used to, though.
  

Ideal use of these two approaches differs too. Obviously, "In-Place" Infrastructure Automation is probably best-suited for an existing set of complex datacenter assets - especially in a Dev/Test environment. As you'd expect , a number of existing lab automation products out there target this market. On the other hand Virtual Infrastructure Automation can certainly be deployed on existing assets, but its real value is for new installations where minimal hardware/cabling/networking can be designed-in from the ground up. Most of these products are designed for production data centers, as well as cloud/utility infrastructures.

My overall sense of the market is that adoption of "in-place" automation will be driven primarily by progressive IT staffs that want a taste of automation and service-level management. Virtualized Infrastructure Automation adoption, on the other hand, will tend to ride the technology wave driven both by networking vendors and OS virtualization vendors.

Stay tuned for additional product analyses in this space...

A real-world cloud user shares his findings

I subscribe to a number of mailing lists from my alma mater. A few weeks ago, an alum "John" posted a request for recommendations for a cloud computing vendor for his small investment firm. What follows is his email to the group following responses he received.

This is an incredibly illustrative peek inside of the "real world" of cloud computing, and what prospective SMB users are looking for and concerned about. As well as what's "Good Enough". I've not edited anything....


I had many requests to share our findings so I figured I would share with the group. I appreciate all of the input I received. It has been really helpful.

~ John


Having looked into cloud computing solutions for our small investment firm over the past few months, we have learned a lot about the growing movement towards remote data storage and accessibility. Our goal has been to find a cost-effective solution for our IT needs that would make it convenient for employees of our company to access our shared network (documents and emails) all over the globe without much hassle, difficulty, or expense. While the cloud computing landscape is still relatively new, what is already available is exciting. Both Google, Microsoft, and other companies have products available such as Google Apps and Microsoft Office Live, but neither has fully come to the point of being able to handle our business needs. We are currently in the process of setting up a Google Apps trial period, through a consultant, to try out business e-mail and calendar via Google’s Gmail and Google Calendar. We will do this test while retaining our current Microsoft Exchange server.

There have been many issues to consider as we have been speaking with various consultants and researching all of the available alternatives. First, since we are an SEC-registered investment adviser with lots of confidential and sensitive information on our hands, issues regarding the security of our electronic files – both in terms of disaster recovery as well the integrity of the company with whom we are entrusting to house our data – are paramount. This also ties in with the issue of record retention, which is equally important to us. In terms of data storage and backup – our current system is not ideal. We need to retain copies of all e-mails and files for at least seven years, if not more, and this information needs to be secure and easily accessible. There seem to be some progress in this area (Google Postini and Amazon S3, for example), but as of yet, there is not yet one system that can do all of these things in the way we’d require.

Second, since we currently are not pleased with our current remote network access - we would like an easy and inexpensive way to access email and our network drive from any computer with Internet access. We have discovered that while web-based, unlimited e-mail and calendar storage are currently available from multiple providers, a solution for mass file storage that would essentially replicate our shared network drive and allow large files for multiple software applications to be stored/backed up in the cloud does not yet exist at an attractive price. In particular, a system where we could modify docs in the cloud without having to download and upload/re-save the file each time it needs to be edited.

One interesting product we discovered during our search is called Dropbox. You download Dropbox to one computer, save any type of file you would like to a “Drop Box drive”, and it syncs up automatically with the Web. Then, when you are at home or traveling, you can access those docs through a web browser... or you can download Drop Box onto another computer anywhere and you can edit the docs directly in Drop Box. The only glitch is that Dropbox does not yet have file storage capacity for a company with over 200GB of data to store and seems to be geared more for individual users. Word on the street is that Google will be coming out with a new product soon that has similar features to Dropbox, but on a much larger scale that would be useful for businesses.

In terms of cost and ease, Google Apps seems to be the best solution for us right now (it comes out around $50/user/year), at least for the e-mail and archiving component. Microsoft’s upcoming 2010 Web Apps platform seems appealing as well, particularly because we might be able to edit complex Excel documents directly in the cloud from anywhere.

Bottom line, what we have learned is that this rapidly-developing option for IT is not yet 100% ready to cover all the bases our business needs, but it will probably get there sometime in the next year or two. For the time being, we are going to see how the e-mail works and go from there.

Alternative Recommendation for DCeP "Service Productivity"

Back in February of this year, The Green Grid published a paper listing proposed Proxy measures for data center productivity, specifically Data Center Energy Productivity (DCeP).

This paper followed a much earlier output from the group in 2007 - which helped define the now much-used PUE and DCiE metrics which I wrote about back then. Those metrics were (and are) nice if what you care about are "basic" efficiencies of a data center -- simply how much power is getting to your servers relative to all of the other power being consumed by infrastructure systems (e.g. lighting, power distribution, cooling, etc.). But the shortcomings are they don't quantify the "useful output" of a datacenter vs. power input. So, for example, you could have a fantastic PUE... but with a datacenter full of idle servers.

Again, enter The Green Grid to take analysis to the next level. The excellent paper published in February details 8 "proxy" approaches (i.e. not necessarily metrics) that could be used by data center operators to begin to baseline efficiencies based on "useful output". The Green Grid also set up a survey where they have been soliciting feedback from users regarding the appropriateness, usefulness, etc. of these proxies.

Why 8 approaches? Because not everyone agrees on what "useful work output" of a datacenter really is. Should it be Bits-per-kWh (proxy #4)? Weighted CPU utilization (proxies #5 & #6)? Compute units delivered per second (proxy #7)? Each has its pros and cons. Fortunately, the Green Grid recognized that nothing's perfect. Says the paper: "...The goal is to find a proxy that will substitute for a difficult measurement, but that still gives a good-enough indication of useful work completed."

In addition, the Data Center Knowledge blog pointed out:

The new goal is to develop a simple indicator, or proxy, rather than a full metric. The Green Grid compares the proxy to EPA mileage ratings for new cars, which provide useful data on energy efficiency, with the caveat that “your mileage may vary.” The proposals “do not explicitly address all data center devices and thus fall short of a complete overall measure of data center productivity,” the group says.

To this end, the issue was also recently dealt with extremely eloquently in Steve Chambers' ViewYonder perspective on datacenter efficiency - and has the right idea: Why not base efficiency on the service provided (as opposed to CPUs themselves, or some abstract mathematical element). This approach is very similar to what I proposed a year ago February, Measuring "useful work" of a Datacenter"

In short, the proposal is to compare the data center Service's SLAs with the power the overall datacenter consumes.

Why use the "SLA" (Service Level Agreement)? Two reasons. (1) The SLA is already part of the vernacular that datacenter operators already use. It's easily understood, and frequently well-documented. (2) The SLA encapsulates many "behind-the-scenes" factors that contribute to energy consumption. Take this example: Not all 1,000 seat email services are created equal. One may be within a Tier-I data center with a relatively low response rate requirement and allowing users only 500MB of storage per mailbox. Another enterprise with the same email application may be operating in a Tier-III datacenter environment with a rigorously-controlled response rate, a full disaster-recovery requirement, and 2GB of storage per mailbox. These two SLA examples are quite different and will therefore consume different power. But wouldn't you now rather compare apples-to-apples to see if your particular instantiation of these 1,000 mailboxes was more efficient to another enterprise with the same SLA?

How would such a proxy/measurement be accomplished? The approach is somewhat analogous to the Green Grid's proxy #1 ("Self-assessment reporting"), coupled with peer-reporting/comparison of data as is done with the DOE's DC-Pro tool.

Thus, data centers would
1) quantify the number of Services and SLAs for each,
2) measure overall power consumed,
3) upload these numbers to a public (but anonymized) database.
After a while, there would be statistically-significant comparisons to be made -- say a "best practice" energy efficiency range for a given Tier-III email application with 2GB storage and disaster-recovery option.

I'm open to other suggestions of how to pragmatically apply application SLAs vs Watts to gauge overall datacenter energy efficiency - again, my earlier proposal of this is here. But it seems that the SLA encapsulates all of the "output" related service metrics, while being agnostic to the actual implementation. Seems elegant, if you ask me.