Converged Infrastructure. Part 1

Since joining Egenera, I've been championing what's now being termed Converged Infrastructure (aka unified computing). It's an exciting and important part of IT management, demonstrated by the fact that all major vendors are offering some form of the technology. But it sometimes takes a while for folks (my analyst friends included) to get their heads around understanding it.  So I'm going to take a stab at a multi-part Primer on the topic.
 
Part 1: What is Converged Infrastructure, and how it will change data center management

Converged Infrastructure and Unified Computing are both terms referring to technology where the complete server profile, including I/O (NICs, HBAs, KVM), networking (VLANs, IP load balancing, etc.), and storage connectivity (LUN mapping, switch control) are all abstracted and defined/configured in software. The result is a pooling of physical servers, network resources and storage resources that can be assigned on-demand.

This approach lets IT operators rapidly repurpose servers – or entire environments – without having to physically reconfigure I/O components by hand—and without the requirement of hypervisors.  It massively reduces the quantity and expense of the physical I/O and networking components as well as the time required to configure them. A converged infrastructure approach offers an elegant, simple-to-manage approach to data center infrastructure administration. 

From an architectural perspective, this approach may also be referred to as a compute fabric or Processing Area Network. Because the physical CPU state (i.e. naming and configuration of I/O, networking and storage naming) is completely abstracted away, the CPUs become stateless and therefore can be reassigned extremely easily creating a “fabric” of components, analogous to how SANs assign logical storage LUNs.  And, through I/O virtualization, both data and storage transports can also be converged, further simplifying the physical network infrastructure down to a single wire.

 The result is a “wire-once” set of pooled bare-metal CPUs and network resources that can be assigned on demand, defining their logical configurations and network connections instantly.

BTW, there is another nice resource -- a white paper commissioned by HP (!) executed by Michelle Bailey at IDC. In it she defines what is a converged system:

"The term converged system refers to a new set of enterprise products that package server, storage, and networking architectures together as a single unit and utilize built-in service-oriented management tools for the purpose of driving efficiencies in time to deployment and simplifying ongoing operations. Within a converged system, each of the compute, storage, and network devices are aware of each other and are tuned for higher performance than if constructed in a purely modular architecture. While a converged system may be constructed of modular components that can be swapped in and out as scaling requires, ultimately the entire system is integrated at either the hardware layer or the software layer.

Converged Infrastructure and Software Virtualization

A Converged Infrastructure is different from—but analogous to—hypervisor-based server virtualization.  Think of hypervisors as operating “above” the CPU, abstracting software (applications and O/S) from the CPU; think of a Converged Infrastructure as operating “below” the CPU, abstracting network and storage connections. However, note that converged Infrastructure doesn't operate via a software layer the way that a hypervisor does. And converged Infrastructure is possible whether or not server virtualization is present.

Converged Infrastructure and server virtualization can complement each other producing significant cost and operational benefits. For example, consider a physical host failure where the entire machine, network and storage configuration needs to be replicated on a new physical server. Using Converged Infrastructure, IT Ops can quickly replace the physical server using a spare “bare metal” server.  A new host can be created on the fly, all the way down to the same NIC, HBA and networking configurations of the original server.

A Converged Infrastructure can re-create a physical server (or virtual host) as well as its networking and storage configuration on any “cold” bare-metal server.  And in addition, it can re-create an entire environment of servers using bare-metal infrastructure at a different location as well. Thus it is particularly well-suited to provide both high-availability (HA) as well as Disaster Recovery (DR) in mixed physical/virtual environments – eliminating the need for complex clustering solutions. And in doing so, a single Converged Infrastructure system can replace numerous point-products for physical/virtual server management, network management, I/O management, configuration management, HA and DR.

Converged Infrastructure - Simplifying Management for “The other half” of the Data Center

In the manner that server virtualization has grown to become the dominant data center management approach for software, converged infrastructure is poised to become the dominant management approach for “the other 50%” of the data center – its infrastructure.

However adoption will take place gradually, for a few reasons:

• IT can only absorb so much at once. Most often, converged infrastructure is consumed after IT has come up the maturity curve after having cut their teeth on OS virtualization. Once that initiative is under way, IT then begins looking for other sources of cost take-out.... and the data center infrastructure is the logical next step.
• Converged infrastructure is still relatively new. While the market considers OS virtualization to be relatively mature, converging infrastructure is less-well understood.

But there is one universal approach that can overcome these hesitations -- money.  So, in my next installment, I'll do a deeper dive into the really fantastic economics and cost take-out opportunities of converging infrastructure...

IT Industry Analysts - Falling Into the Bond Rating Agency Trap?

One of the leading causes of our recent economic melt-down was that "independent" credit rating agencies had a conflict-of-interest with the firms they were supposed to be watching.The very firms tasked with objectively gauging risk were also being paid by the firms they were evaluating...And in the end, the big losers weren't either of them... it was the public.

Well, beware that some of the same could be happening in the IT space.

I'll change the names to protect the innocent -- but let's say that I recently attended a day-long IT analyst event, one where all of the senior analysts trot-out their recent research. And to be honest, most of it was of very high quality.

But in one session which focused on an up-and-coming trend in IT, the analyst only cited the major IT vendors (think: HP, CSCO, IBM, Dell etc.) as the leading innovators and players in the space. It was complete Bunk. Of the four "leading" vendors mentioned, only one of them had any significant innovation in the space. Two others so coated their offerings with "marketecture" that real innovation was tough to discern.  And the final crime was that 2-3 smaller vendors I know who actually pioneered the space weren't mentioned at all. And they're the ones providing *real* products with real value today.

Yes, the analyst had a responsibility to his customers (IT end-users) to watch the big players in the industry. And to be sure, the big vendors dominate most market spaces. But the analyst also has a responsibility to truly master his market space and to report-back on the true leaders, innovators, and visionaries. Instead, I believe he unwittingly fell prey to the big vendors that pay much of his firm's bills in order to stay in the analyst's limelight. The failing here is industry-wide, and the IT consumers of the analyst's information are the real losers. Innovation isn't recognized, and therefore value isn't really transferred. And nearly all large industry analysts are guilty of this at some level.

In contrast, another friend of mine is a technology industry analyst with a major international financial institution. When he interviews me on my industry, company and product, he's clear that his reports are not commissioned by vendors, nor even by his bank's clients. There cannot be so much as a hint of conflict-of-interest in his work. Think about it.

---

Major IT Industry analysts have been my friends for years. I've worked for IT vendors small and large, and IT analysts have been (and mostly still are) great sounding boards for new ideas, helped identify market opportunities, and have added lots of marketing value if/when they approve of your product. And IT analysts add value on the IT consumer side too -  by identifying trends, pointing-out leading vendors, and recommending best-practices.

But sometimes these folks fundamentally fail at what they're "paid" to do. My advice: Always get a second opinion.

Hosting & Cloud Computing market index: Update

This month's updates to my original index indicate that the hosting market - and particularly those companies that are in the cloud hosting market - are doing quite well at holding their own against the falling NASDAQ index. Although the NASDAQ component was down ~ $7, my broad hosting component was down less than $1, while my cloud computing component was actually up ~ $7.

One other point of note: Apparently Merriman Curhan Ford also believed that coverage of this space was now warranted:

"We believe Cloud Computing represents a fundamental shift with regards to how IT organizations manage and source data center computing resources.  Companies such as Terremark, Rackspace, SAVVIS and NaviSite are at the forefront of this development," said Alex Kurtz, senior vice president and technology equity research analyst of Merriman Curhan Ford.  "Our core differentiator in covering this space is leveraging our expertise within our existing coverage of IT systems vendors, who are competing for the same IT budget dollar and impacted by the same macro trends as a Terremark or a Rackspace."

If you think Converged Infrastructure & Fabrics are niche, guess again

A few weeks ago, I Tweeted about an analyst conversation where it was looking like the market for Fabric Computing / Unified Computing would be growing rapidly in the foreseeable future.

Another analyst friend of mine quickly commented back – sarcastically – that the market was sure to be in the billions of dollars.

I was feeling a little unsure about this market until a few weeks later when I was shown a technology report from Thomas Weisel Partners. Although the market definition for converged infrastructure (also known as Unified Computing) was still forming, TWP felt that sales of Converged Infrastructure solutions could rise as high as $15 billion by the end of 2014.  Billion with a “b”?  Right-on…

Then there is a report by Gartner Research on fabric-based computing… which estimated that by the end of 2012, roughly 30% of the world’s top 2000 companies would have some form of fabric-based computing architecture. (Under the heading of “fabric” falls Unified Computing as well as Converged Infrastructure).

So, why is the market (for fabric computing, converged infrastructure, unified computing) still considered so new in the market, yet forecast to be so booming in 2-4 years?

First of all, what we’re talking about here are systems like Cisco UCS, Egenera PAN Manager, HP VirtualConnect, IBM Open Fabric Manager, and a few others. At the heart of each system is technology (sometimes HW, sometimes SW, sometimes mixed) that virtualizes I/O and leverages converged networking.

And why are vendors all chasing this approach? For a number of reasons --

1. It’s incredibly complementary to virtualization: in the same way that the hypervisor changed how SW is abstracted, provisioned, managed and migrated, Converged Infrastructure changes how IO/networking/connectivity is assembled and managed. This gives vendors a valuable set of new offerings, and can tie management of infrastructure to management of VMs – yielding end-to-end abstraction of the entire data center. Roughly as much $ is spent managing infrastructure as it is managing software… to the TAM is huge here.
2. It changes how availability is delivered: By manipulating IO addressing, networking and connectivity, Converged Infrastructure Management can re-provision failed hardware – either in the form of physical servers, or indeed, entire environments. Thus, Converged Infrastructure has the potential to displace a big chunk of traditional clustering software… (nearly a $ billion, if you follow IDC’s estimates)
3. It changes how networks are physically wired and managed: Converged Infrastructure uses fewer IO components (either a single LOM or a single CNA), converged network protocols, fewer cables, and generally fewer switches. This yields a lower CapEx investment, and a commensurate lower OpEx to manage. The opportunity to sell alternative approaches to each of these technologies is immense.
4. Converged Infrastructure is highly complementary to shared storage: the pervasiveness of SAN storage is a major enabler of a more virtual/flexible data center. As physical/virtual servers move, migrate and scale, storage simply follows.  An increasing ratio of servers – especially blades – are being shipped with HBAs, indicating that SAN use is on the upswing.
   

As to evidence that this market is shaping-up, we need only look to the magnitude of investment that Cisco, Egenera, HP, IBM – and even Emulex and Qlogic – are pouring into this market. Methinks we’ll see the hockey-stick shortly.

Hosting & Cloud Computing market index: Update

Last month I proposed that another way to measure adoption of cloud computing (or, at least, expectations of adoption) was to look at the stock market performance of a bundle of publicly-traded service provider companies.

Since then, I've expanded the list, and carried the range back 24 months.

The total list (the "broad hosting index") consists of: Digital Realty Trust, DuPont Fabros, Equinix, Internap, Iomart, Macquarie Telecom, Navisite, Rackspace, Savvis, Switch & Data, Telecity, and Terremark.  I also baselined my "virtual" fund against the NASDAQ index. I created another virtual fund (a subset list of the above) consisting of Equinix, Navisite, Rackspace, Savvis and Terremark - representing service providers with substantial businesses in the Cloud hosting space as well.

Here are some interesting observations of the value change of US$100 invested equally across each index:

Since Jan 2008:

• Nasdaq:  + ~14%
  
• Broad Hosting index: + ~90%
  
• Cloud Subset: + ~50%

But, I also looked at the change since the market bottomed-out in March of 2009. Since then, the picture is a tad different:

• Nasdaq: + ~ 55%
  
• Broad Hosting index:  + ~135%
  
• Cloud Subset:  +~ 115%

These numbers tell me that the performance (or at least, expectations of performance) in the hosting space far exceeds the broader NASDAQ technology sector.  Interestingly, the "cloud index" under-performs the broader index. No explanation here other than the fact that we're dealing here with a statistically low number of companies, and a few "high performers" in the broader index seem to be lifting it above the cloud index.

I'll plan on updating this periodically. Comments, additions, etc. welcome!

Hosting & Cloud Computing: Numbers Don't Lie

There's lots of chatter in the market today regarding the value of using outside data centers, hosting services and cloud computing.  But listening to pundits/analysts trying to objectively predict true value left me hollow.

While I'm not an investment professional, I do know that the stock market doesn't lie.... so instead, I thought I'd look at a bundle of stocks from publicly-traded data center companies in the data center space, and compare against a market benchmark.

I chose companies on publicly-traded markets in both the US as well as in Europe. My criteria were somewhat subjective, but basically the companies had to have a primary business operating data centers. I also excluded Telcos because it is difficult to separate their carrier revenues relative to hosting revenues. So, my initial "virtual fund" consists of 12 companies: Digital Realty Trust; DuPont Fabros; Equinix; Internap; Iomart; Macquarie Telecom; Navisite; Rackspace; Savvis; Switch & Data; Telecity; Terremark.

I also took a 5-company subset of these public companies that had significant offerings in the cloud computing space (Equinix; Navisite; Rackspace; Savvis; Terremark). I labeled this "virtual fund" a cloud-only index.

The chart at right is my best attempt to (a) tabulate historic end-of-month closing price of each stock; (b) calculate month-to-month percentage gains for each; (c) create "virtual funds" where $100 would be invested equally across each vehicle (initially $8.33 in each of the 12 hosting stocks, and initially $20 in each of the 5 cloud-related stocks). The benchmark I used is the Nasdaq index, also assuming an initial $100 investment.

Not surprisingly (for me, anyway) both "indexes" are outperforming the Nasdaq -- perhaps proving the thesis that datacenter operation and application outsourcing is indeed a growth market (or at least a speculative growth market?) as compared to the general technology market. What would be equally useful (but not an analysis I've done) is to chart gross revenues for the Index companies. This would be a telling barometer of actual business.

I'll continue to update this index at the end of each month. Comments, additions and suggestions welcome!

Emergence of Fabric as an IT Management Enabler

Last week I attended Gartner's annual Data Center Conference in Las Vegas. Four days packed with presentations and networking (of the social kind). Lots of talk about cloud computing, IT operations, virtualization and more.

Surprisingly a number of sessions directly referenced compute Fabrics -- including "The Future of Server Platforms" (Andy Butler), "Blade Servers and Fabrics - Evolution or Revolution" (Jeff Hewitt), and "Integrated Infrastructure Strengths and Challenges" (Paquet, Dawson, Haight, Zaffros). All very substantive analyses of what fabrics _are_... but very little discussion of why they're _important_. In fact, Compute fabrics might just be the next big thing after OS virtualization.

Think of it this way: Fabric Computing is the componentization and abstraction of infrastructure (such as CPU, Memory, Network and Storage). These components can then be logically re-configured as-needed. This is very much analogous to how OS virtualization componentizes and abstracts OS and application software stacks.

However, the focus by most fabric-related vendors thus far is simply on the most fundamental level of fabric computing, which is simply virtualizing I/O and using a converged network. This is the same initial level of sophistication when the industry believed that OS visualization was only about the hypervisor. Rather, we need to take a longer view of fabric computing and think about higher-level value we create by manipulating the infrastructure similar to how we manipulate VMs. A number of heady thinkers supporting the concept of Infrastructure 2.0 are already beginning to crack some of these revolutionary issues.

Enter: Fabric as an Enabler

If we think of "fabric computing" as abstraction and orchestration of IT components, then there is a logical progression of what gets abstracted, and then, what services can be constructed via logically manipulating the pieces:

1. Virtualizing I/O and converging the transport

This is just the first step, not the destination. Virtualizing I/O means no more stateful NICs and HBAs on the server; rather, the I/O presents itself to the OS as any number of configurable devices/ports, and I/O + data flow over a single physical wire. Transport can be Ethernet, FCoE, Infiniband, or others. In this manner, the network connectivity state of the physical server can be simplified and changed nearly instantaneously.

2. Virtual networking

The next step is to define in software the converged network, its switching, and even network devices such as load balancers. The result is a "wire-once" physical network topology, but with an infinitely reconfigurable logical topology. This permits physically flatter networks. Provisioning of the network, VLANs, IP load balancing, etc. can all be simplified and accomplished via software as well.

3. Unified (or Converged) Computing

Now things get interesting: Now that we can manipulate the server's I/O state and its network connections, we can couple that with creating software-based profiles of complete server configurations -- literally defining the server, its I/O, networking, storage connections, and even what software boots on it. (Software being either a virtual host, or a traditional native OS). Having defined the entire server profile in software, we can even define the entire environment's profile.

Defining servers and environments in software allows us to provide (1) High Availability: With a hardware failure, we can simply re-provision a server configuration to another server in seconds -- whether or not that server was running a VM host, or a native OS. (2) Disaster Recovery: we can re-constitute an environment of server profiles, including all of their networking, ports, addresses, etc., even if that environment hosts VMs and native OS's.

 4. Unified Management

To achieve the ultimate in an agile IT environment, there's one remaining step: To orchestrate the management of infrastructure with the management of workloads. I think of this as an ideal Infrastructure-as-a-Service -- physical infrastructure that adapts to the needs of workloads, scaling up/out as conditions warrant, and providing workload-agnostic HA and DR.  From an IT agility perspective, we would now be able to abstract nearly all components of a modern data center, and logically combine them on-the-fly as business demands require.

Getting back to the Gartner conference, I now realize one very big missing link -- while Gartner has been promoting their Real-Time Infrastructure (RTI) model now for some time, they have yet to link it to the coming revolution that will be enabled by fabric computing.  Maybe we'll see some hint of this next year.