Fountainhead

It Will Be a Data-Centric (Cloudy) World

2012-01-19T16:45:00.000-08:00

(Or, Where Tomorrow's Clouds Will Form)

Move Mohammed to the mountain, or the mountain to Mohammed?

In the context of data, applications and cloud computing, this question takes on a new perspective - and the role of Mohammed and the Mountain may soon reverse.

In the traditional application-centric (and static infrastructure) world, the Application is the the immovable "mountain". Like a magnet, the app is permanently-located, attracting to it local data stores and peripheral support apps. Administrators dote around it like worker bees around the queen.

But for some uses and applications this may all change - altering with it the how-and-where compute and community clouds form.

Observation #1: Apps are becoming mobile

With increased use of a virtualization layer, migration tools, shared storage, fat network pipes, and virtual I/O and switching, we are all now realizing that where the executable application code resides is becoming far less important. Everything is just data - and can be moved/migrated. In mature virtual environments, VMs typically move between between servers because of maintenance windows, because of capacity adjustments, etc. But when VMs move move between physical data centers (separated by many miles or more) there is often a data movement as well. But there's no denying that the application is becoming more mobile.

We also have the emergence of large data arrays and analytics appliances that embed internal servers that speed queries and analysis. VM's typically run on top of these servers being migrated in-and-out of the arrays as workloads and queries change. Hang on to this visual... we'll come back to it later.

Observation #2: Big Data is becoming bigger

When we start talking about hundreds of Terabytes - or even Petabytes - of structured/unstructured "big data", moving that data becomes increasingly physically difficult. Where it's generated is generally where it stays. Think about financial stock exchanges; retail data warehouses; medical imaging; geologic or climatological data. These stores are now becoming big and immovable.

So, enterprises are now locating these data stores within critical data centers - within which they are co-locating the applications that require frequent access to that data. Sometimes that proximity is sufficient, and sometimes the analytics may even move within the array. But any way you look at it, those data stores are becoming the center of attention, around-which the applications now congregate.

An interesting shift. But wait, there's more...

Outcome: Where Tomorrow's Clouds Might Form

So let's expand this model from enterprise data centers to public clouds. Or even to "community clouds".

Take the example of financial exchange data - NYSE’s Capital Markets Community Platform about which I blogged last year. Here we have a special-purpose, "community cloud" - optimized for financial institutions, wherein they can locate trading and analytics applications. (Imagine a 3-person hedge-fund startup needing infrastructure). Operationally it's fit-for-purpose, with a high performance, low-latency compute backbone, with a common security/compliance envelope. But it's got another trait: At its core is a historic data warehouse of every tick for every trade. Now that's big data.

If you think of it, the NYSE data store has become the "Mountain" around which applications (supplied by the cloud tenants) now congregate.They run their algorithms and analytics against the local data store. The data within the community cloud has become the anchor, the magnet. The apps are moved to be near the data.... not the other way around. Remember that data array that had embedded VMs? Well, think of this model as that array on steroids.

So, what might this mean if you want to build a differentiated cloud computing resource - say, targeting a specific industry vertical? It says to me that the world will shift to a data-centric model. Focus on amassing and maintaining massive high-value data, all (presumably) requiring a similar security/compliance model. And then build a business by allowing tenants access by co-locating their applications in the same cloud as the data resides.

I could see this transforming both the cloud service provider ecosystem, as well as entire industry groups. Consider new Cloud Service Provider models: What if NOAA formed the Weather and Atmospherics Community Platform? If healthcare companies created federated Medical Records Community Platforms? If the USGS formed the World Geologic Community Platform? If other brokerages created equivalent capital markets platforms?

Cloud computing is shifting lots of conceptual IT models these days. But while you're considering what Cloud makes possible for applications, spend some time wondering what data makes possible for the Cloud.

Other References

My Cloudy Nerdy Playlist for 2012

2012-01-02T10:38:00.000-08:00

Political movements have their anthems. Starbucks has its background music brand. But I thought it might be fun to collate a playlist that specifically appeals to the techie/cloudy group.

So here goes. Some of these are pretty funny... others leave a bit to be desired. And if you know of more (or have favorites of your own) send them my way and I'll add them...

Songs about Clouds

Get Off Of My Cloud - The Rolling Stones
Clouds - The Jayhawks
Clouds - Django Reinhardt (awesome rendition w/a Spanish guitar)
Clouds - The Go-Betweens
Clouds - The Submarines
In The Clouds - The Cult
Death Cloud - Cloud Control
Cloudy - Average White Band (not the greatest audio/video)

Other (mostly funny) songs about IT

Nerd Lust - Schaffer & The Darklords
She Blinded Me With Science - Thomas Dolby -
Nrrrd Grrrl - MC Chris. Hi-grade lyrics. Low budget production.
(Do You Wanna Date My) Avatar -The Guild
The SysAdmin Song - Wes Borg - A classic.
IT Administration Song – To the tune of “A few of my favorite things”
The Computer Song – Dare you to memorize all of these acronyms. And this was just 2007.
Cookie Computer Song – Classic Sesame Street. Primitive. 1974!
Happy System Administrator Day - (one of my fave Animes) Features “The SysAdmin Song” above
Networking Tips – Don’ts - Sheryl Nicholson (a short Hip-Hop style interlude)
Object-oriented Programming Song - #7 - from Design Minstrel (great lyrics, horrific delivery)
The Computer Scientist - Code Play (awesome indie creation. but delivery also leaves something to be desired)
The Computer Error song - A fun sampling/mashup... or listen to the remix. (Windows listeners only) Maybe this'll be the V0geball psyche-up tune next year.
The Day the Routers Died – Had to end with this one from RIPE 55 conference, 2007. Freakin' brilliant tribute to IPv6. Not sure who sang, tho. But picture a room full of geeks singing the final stanza...

Techy Poetry

Horton Hears Hadoop - Joe Onisick
Horton Hears a Hadoop - Tom Keske
(I need more here... )

Even MORE Links...

Steve Jobs in 1980... and Cloud + Big Data Today

2011-12-19T13:29:00.000-08:00

In 1980, Steve Jobs gave a talk (YouTube) about the early days of Apple.

It's fascinating because it describes events that lead up to the first commercially-available Apple computer, and presaged the movement toward "canned" programs (commercial software) rather than everyone writing programs themselves.

However, around the 12:00 mark, Steve made a really incredible observation regarding the true goal of Apple (at the time) and about how to use the new level of computing power available to Apple computer users:

"As we move into the '80's, the amount of computational power - the amount of raw horsepower - we can get into a small box for a reasonable price is staggering.... One of the things people always ask me [is] 'what we've got right now is just fine; VisiCalc runs fast enough. Some of the database stuff runs fast enough. What are we going to do with this extra awesome power?'

"The answer to that is that we're going to put it [computational power] into applying/solving that problem again: In other words, we're going to start chewing up power specifically to help that 1:1 interaction go smoother. And specifically not to actually do the number-crunching and database management and word processing. We're actually going to apply that power specifically into removing that barrier....

This statement struck me because, in essence, Steve was saying "it's not about faster, it's about easier". He was pointing to all that could be possible if the clunky interaction with the technology was relegated to the background and made invisible.

In today's enterprise context, my interpretation of this is "What could be possible if all of that data center technology could be relegated to the background and made invisible?" In essence I thought, what if we could mask the granular time-consuming operational efforts of managing servers, I/O, applications, networking, storage, security etc. and get back to why the data center is there in the first place? Today, IT operations for 'keeping the lights on" consumes ~ 75% of IT's budget today, and only ~ 25% is left over for innovation and serving the business.

The goal of IT Transformation should be to "solve the problem again" and put the computational horsepower on autopilot. Let's get to the point where our interaction with the data center is the ability to ask for the resources we need, and in response, we instantly get fast, scalable, secure services.

If we can get to the point where 25% of the cost is to keep things running, and 75% of the cost is used for innovation about the data, what would be possible for business?

In my opinion, IT is still largely in the Dark Ages. We are obsessed with speeds and feeds, tuning and tiering. The purpose of the Data Center is the Data - manipulating and analyzing it for the business. If we found a way to direct 75% of our computational horsepower to THAT, what would be possible?

That needs to be the goal of IT Transformation. That is why Cloud is such a critical enabler.

Thanks once again, Steve.

Other Resources:

The Death of Why (Blog by EMC's Bill Schmarzo)

Predictions: What You WON'T See in IT For 2012

2011-12-15T13:50:00.000-08:00

While everyone is jumping in with 2012 predictions ('Tis the season) I want to impress on everyone that a dose of sobriety is in order.

IT vendors - and even analysts - are understandably eager to see new technologies and operating models (cloud) adopted quickly. But let's acknowledge the High Tech adage: Not very much happens in 2-3 years, although massive changes occur over a decade.

At any rate, I wanted to highlight things I believe we won't see happen in 2012. Perhaps because we're just being a bit over-eager, or perhaps other enablers have to precede them. Here goes:

An Instant Cloud
Despite claims from vendors, and a plethora of tools and technologies, you still won't see an "instant-on" private cloud solution in 2012. And you probably won't for years to come. I know many folks (mostly vendors) will vehemently disagree with me, but let me challenge you all with this: Cloud is an operational model, enabled by technology. Simply implementing a tool (BTW, most of which are still only tenuously integrated with each other, as well as with hardware and networking platforms) won't solve the problem for you. And definitely not for an enterprise-ready level of availability

Building infrastructure is one thing; knowing how to operationalize it, integrate it into your enterprise, and how to re-structure your service delivery processes are very different.

However, I do believe that in the next 3 years or so, mainstream enterprises will come to realize that the issue is only partly technology-based. And I hope that change-management and organizational design models will mature to the level of technology models so that "turn-key" process change and skill development will accompany the product sale.

One Dominant Public Cloud
Despite growing use of AWS, Google and Azure, I believe that none of these will be a runaway dominant leader for 2012. Or ever.

Although there will likely be 2-5 very large public cloud leaders who compete on economies-of-scale, I believe the invisible hand of the market will instead cultivate many more "special purpose" community clouds. These players will develop based on knowing their specific market requirements - i.e. competing on security, compliance, use habits, special-purpose applications, performance, etc. Take for example financial markets (NYSE's Capital Markets Community Platform), Healthcare (varied provider solutions), or even Federal, State, and Local Government (varied initiatives).

The next 2-3 years will definitely see more specific examples of these special-purpose cloud computing initiatives - and perhaps even the emergence of a few 'dominant' community clouds in selected markets. But in 2012, we won't see the community cloud market held back by the presence of large public clouds.

Broad Use of IT Chargeback
Financial chargeback (and showback) have been discussed for years, and are being implemented in greater numbers lately. And although I am a proponent of IT Financial Transparency, we won't see the broad-based use of chargeback unfortunately still won't go mainstream in 2012.

In my opinion, implementing financial transparency tools are second-level initiatives. They don't make sense to implement until and unless the IT department first has other financial controls and metrics in place. And they certainly don't make sense unless IT and the lines-of-business agree on what they're trying to achieve with better IT financial transparency in the first place. Is it really cost-recovery, or merely better knowledge of variable costs and consumption? Is it an attempt by IT to become more 'competitive' and to measure itself against external providers? Is it an effort by the CIO and CFO to gather better build-vs-buy decisions? The enterprise has to ask these questions before forging ahead with a chargeback program.

So, while 2012 may not be the year of chargeback, it might be the year when IT begins to take a more evolved approach to measuring its variable cost, to metering consumption, and to implementing the goals and strategies it will need to begin these initiatives. Broad-based use of chargeback may still be a few years off, but I hope that IT financial maturity begins soon.

IT is Elevated to a Strategic Business Enabler
Unfortunately, in 2012, we'll still see the vast number of enterprise IT groups continue to report up to the CFO, to be pressured to keep-the-lights-on with less, and to simply be considered a cost center by the organization. As much as IT should be treated as a core enabler of the business, this just won't be so in 2012. It takes time.

But a transformation of IT is taking place, slowly. Saavy CIOs are thinking of themselves as "internal SPs", and beginning to relate to partnering with lines-of-business in a formal manner. To make this transformation, IT first has to adopt new models for services consumption, operations and technology.

In the new context of Business Enabler, IT partners with Lines-of-business to ensure that (a) services are quickly made available to support the top-line revenue needs, (b) IT works with business managers to educate them about potential new services and top-line opportunities, (c) IT adopts a 'consumerized' mindset whereby it supports an "any device" approach to endpoints, and (d) IT is as comfortable with brokering external services as it is with generating its own - doing whatever it takes to support the needs of business users.

But for 2012, let's push IT - and the business - to begin planning for this transformation.

Customers Catch Up to Vendor Vision
As a long-time marketer in high-tech, I've seen the tendency of vendors to push customers to adopt the Next Big Thing. And that N.B.T. is frequently disruptive (or at least discontinuous) with respect to the "legacy" approach to doing things.

To be sure, there are always customers who are leading-edge in their technology adoption. But the mainstream customer adopts technology *incrementally* and rarely if ever discontinuously. This is a byproduct of (a) the human tendency to mitigate risk, and (b) business' tendency to plan change - and budgets - incrementally.

So, for 2012 - and for the foreseeable future - this trend won't change. Hopefully vendors will be more clear about what's "vision" or what's "for early-adopters", and maintain a healthy dose of sobriety about selling high-brow discontinuity to the mainstream market.

As this relates to Cloud computing, it's clear that more of the mainstream market is adopting virtualization, and has bought into the concept of cloud computing initiatives. But let's be clear: In 2012 the average IT infrastructure won't be completely re-built into a private cloud. However, I believe that the majority of medium and large IT shops will all have begun their progression toward the private cloud eventuality.

Other (prediction-related) links

IT's Blind Spots for 2009 (Vol I) Fountainhead Blog
IT's Blind Spots for 2009 (vol II) Fountainhead Blog
Prediction 2009: The Future of Virtualization Fountainhead Blog
10 IT Predictions for 2012 Chuck's Blog

Celebrating Innovation at EMC

2011-12-04T09:06:00.000-08:00

Can innovation be fostered or facilitated? Can it be measured? Is it a formal or informal process? Can you have an innovation competition? Is it limited to Engineering?

These were not the topics I was expecting at last week's EMC Innovation Conference. Rather, I thought it was going to be a geek fest with lots of greasy-haired, propeller-hat-wearing attendees drinking diet-Coke and speaking mostly PERL, C++ or Greek.

Instead, the event was attended by a few hundred local Silicon-valley EMC-ers, with hundreds more globally watching simulcast. It was business-casual with execs and technologists alike, and kicked off with none other than EMC's COO Pat Gelsinger (and former Intel CTO) and MC'd by SVP/CTO Jeff Nick. Rather than a siloed techno event, the importance of innovation at EMC was supported at the very highest levels of the company. Now that's refreshing.

Guest stars attended too - one Scott McNealy - in his usual humorous, politically- and sports-charged manner.

But First: Did the COO "Get It"?
Resounding yes on this one. Pat opened with a real command of what innovation means to the company - Innovation can mean disruption... and even destruction of old ideas. And that's good (if you don't agree, read the Innovator's Dilemma). But Pat also pointed out that without being coupled with commercialization/monetization, innovation doesn't count for much. And knowing how to commercialize an innovation can be just as challenging as coming up with the idea in the first place. More on that later.

Pat was also insightful in that innovation isn't just about technology. You can have organizational, marketing, and process innovation, in addition to product-based innovation. And innovation can be top-down and/or bottom-up (EMC does both). To help accomplish systematic, company-wide innovation, EMC uses the Golden Triangle: Organic innovation, University research/relations, and EMC Ventures. More on these later, too.

Can Innovation Be Measured?
Both Jeff Nick and Steve Todd (Director of EMC's Innovation Network) focused on this question, and there are many perspectives. Patents, for one, are a useful metric for organic innovation, but as Pat pointed out, they don't necessarily indicate that an idea has been monetized. Nonetheless, there is a rough norm of patents per employee per year that the company is striving towards. Intellectual Property in-and-of itself can be valued, as we've seen from recent acquisitions of IP by IBM, Google, and others.

Jeff also pointed out that another indicator of innovation is of course the monetization itself. What's the ROI on an organic development group? Did an acquisition recoup its cost? Did a a venture investment create a useful return, or even better, might it have been a profitable acquisition?

While measuring innovation might still be tricky, knowing how to facilitate indicators gets part-way there. So when Steve pointed out that another indicator tended to be when small groups work together to solve a problem, there was an "aha" moment. How to constantly encourage groups to form, and information to flow? Therein lies his "day job"...

Can you Compete for Innovation?
Thus, Steve realized that encouraging groups to form with a common purpose might spur organic innovation. And, for the past few years, he's been conducting an annual innovation contest within and across the company. Participation has been growing at a healthy clip. Topics can be many - some were geeky, but some were clearly for social betterment.

To his credit, Steve also used a dose of data analytics to map every individual and every project across the company. He confirmed, in fact, that finalists tended to work in groups or "Cliques".Which begged the question, how to facilitate and accelerate this behavior in the future.

Formal or Informal?
The other aspect I really appreciated about the day was the intentional mix of formal and informal innovation the company strives to maintain. As I mentioned, the company emphasizes the concept of the "Golden Triangle". In addition to internal/organic innovation, we also work informally with external universities and research... and of course, formally with many prominent VCs and startups.

Mark Lewis, EMC's Chief Strategy Officer and leader of EMC's Venture group, gave a fantastic overview of how we regularly look at the startup community, and place measured investments with them. The ideas is to discover great ideas both ahead of the competition, and early enough such that we don't have to pay huge premiums to acquire should we deem them valuable.

The strategy behind these investments is broad, but what I heard was a 2-pronged approach:
(a) Pursuing the technologies that are complementary to EMC's product set - technologies and ideas that are net-new to the company, yet extend our leadership in the data center market
(b) Pursuing products that might eventually replace or cannibalize existing EMC products. Like the Innovator's Dilemma points out, better to cannibalize your own products than have someone else do it for you.

Limited to Engineering?
Capping off was another surprising panel... of marketers. Could Innovation extend into how we market EMC and our products? Absolutely.

Two fascinating points stood out, and hopefully made a huge impression on the audience and EMC as a whole. First, the panel chosen consisted entirely of recently-acquired marketing leadership located locally in the Bay Area. Scott McNealy had pointed out that "Silicon Valley DNA" was somehow different of that from the rest of the country - and EMC (based in Hopkinton MA) was making a concerted effort to graft that DNA into the rest of the company.

Second, was the attitude of corporate marketing. Do things differently. Use metrics. Understand the trends of the "new" buyers... (Hint: they don't pay attention to marketers using traditional email, snail-mail or traditional TV). Innovation here had to do with how social marketing, new/viral media, and fast-paced imagery was the way to both get attention as well as to change the face of the company brand.

For me, this day definitely served to change how I thought of the CTO's office, how I related to the concept of innovation, and how I think of corporate development. As EMC further pursues these paths, hopefully other leading companies will as well -- it's good for them, for careers and personal development, and frankly for the economy as a whole.

More Resources
Jeff Nick (CTO): About Article
Mark Lewis: About Blog
Steve Todd: Twitter: @SteveTodd Blog: Information Playground
Chuck's Blog: The Challenge Of "Innovating Innovation" In Large Corporations

A Marketing Lesson For IT

2011-11-08T14:56:00.000-08:00

In an earlier post of mine, "An Image Makeover for IT" I shared about a surprising meeting I'd had with our IT department -- where they were looking to learn how to better market their services within the company. I felt that this was a harbinger for the "New IT", and I think I was right...

Then last week I was given a draft Data Sheet by EMC's IT department.Whoa, you say. A Data Sheet? Isn't IT supposed to operate down there in the bowels of the data center keeping the "lights on"?

As you know, EMC's IT group has been progressive, from their virtualization initiative to building an internal Private Cloud for the company, to organizing for success, to marketing themselves as if they were a competitive service provider. Which is just the way more IT groups will have to think.

Why would IT Want To Market Themselves?
If IT is to think more "competitively" then they need to organize and act competitively, including driving demand/consumption for their services. This isn't about internal politics (validating their annual budget) but rather about driving awareness, demand, and preference for their services. Because IT's new competition is "shadow IT", the advancing commercialization of IT. More simple-to-use, aggressively priced external services are being offered to users who are accustomed to the convenience. So IT finds itself having to do the same.

And there is the education component too... because while the enterprise consumer might want to compare internal vs. external services purely based on price (think: AWS, DropBox, MobileMe), IT has to show that there are hidden costs to these: Risk exposure, inability to prove compliance, cost of securing data, etc. etc. So IT turning to traditional marketing devices isn't all that odd in the final analysis.

A Data Sheet for IT...
Back to the Data Sheet I received. The IT dept did their homework - Like all good product marketing summaries, it included components to project leadership and to drive demand:

Service components: In this case, it is an IaaS style service, replete with the value proposition. It defined the service, the pain-points it alleviates, and why internal EMC engineers should consider using it.
Overall Value: IT also did a great job of illustrating the *total value* of the service - not just economic value, but the value of agility (time-to-provision) and convenience.
Value of Risk Avoidance: Some marketers like to market to the paranoid... in this case, IT illustrated the risks of not using an IT-sourced service.Security, compliance, SLAs, etc.
Competitive comparison: In particular, I like the fact that the internal private-cloud based service full-up cost was compared to that of popular public cloud providers. It illustrated that the actual cost of external services isn't always just the bare-bones hourly cost on your credit card...
Availability roadmap: I also like the fact that IT set expectations with its customers. What's available now, and what new services/features will be available when.
And a little bit of Esprit de Corps:Yep, good old pride in what the company is doing, and how the IT department is helping drive the top-line business.

...And Enabled By New Roles, New Skills
If IT is to succeed as a "competitive" provider to the enterprise, simply publishing a Data Sheet is just one deliverable in a larger story. The organization needs to think, act, and hire like an internal service provider - and ask itself "How do successful SP's drive business demand?"

The process begins with a cultural and leadership shift to a desire to act not as an internal (monopolistic) cost center, and more like a competitive business catering to the varying demands of its internal customers. This is a CIO-level decision, often done in partnership with LoB's and even with the CMO.

And who executes on this new approach? What new functions need to sediment into IT? Well, it's not about technology skills anymore. Rather, IT will need to acquire traditional business and marketing skills. A few suggestions:

Develop In-bound and out-bound Marketing: This refers to classical Inbound product management (listens externally to customer needs, and helps determine product features, plans, pricing - and classical product marketing, helping drive external awareness, preference and demand. In both cases here, Inbound/outbound functions work as closely with internal lines-of-business to understand their needs, roadmaps and futures.
Initiate client "relationship management": Somewhat akin to a "sales" function, these individuals are assigned to sit closely-aligned with business users of IT, and work closely with in-bound product management. These folks detect leading indicators of how IT can better serve the business, and look for ways to add value either with competitive technology, services or analysis.
Include Pricing and Financial Management: Pricing and costing of services is a shift away from tallying-up gross capital and operational expenses, with a move instead towards activity-based costing. The ability to discern the true usage-based cost of an IT service means that the organization can better align supply with demand, and make better buy-vs-build decisions. This "financial transparency" allows the CIO, CFO, and Lines-of-Business to make better overall decisions regarding using IT to support the enterprise.

If IT is to be competitive, it has to act competitive. Marketing of IT is far from the only shift that needs to be done - and arguably, it probably follows shifts in IT technology and IT operations. But marketing IT is nonetheless an illustrative example of what is to come with a larger IT transformation initiative.

Act Now! Top Opportunities for IT Transformation Services...

2011-11-02T10:45:00.000-07:00

It's become obvious of late that cloud computing is not a technology end-in-itself, but rather an enabler of broader IT transformation. I've heard it from CIOs, CTOs, and even EMC's own IT department. All said, it's clear that IT transformation affects the entire organization - its consumption model, operations, organization, finance and more.It's not just about products anymore.

IMO there is a massive opportunity for leading change-management organizations to work with IT organizations to help them assess and execute this transition. Technology alone will not make IT more competitive.

So I'm a bit shocked that this era isn't a windfall for services providers who will help shepherd IT organizations through the change management process. It should be. To me this seems like thre's a huge investment opportunity to develop service practices. And, it's a huge opportunity for IT organizations to take pause and assess what their structure, operational model and business model will be a few short years from now. And how they plan to get there.

Follows are a few observations - and opportunities - I've made in the hope that a rising tide lifts all ships:

No Show Offs?

Where are the services providers, consultants and change-management professionals at trade events?

Given the tons of IT, cloud, and tech related trade shows, I'm continually amazed at how 99% of floor exhibitors are hardware or software firms. The show managers occasionally include special floor sections for "cloud" or "security" and such, but never for services providers or consultants who would assist the attendees in implementing the technology and change management needed to leverage technology.

I'd like to see an "IT Transformation Services" section in the large shows (i.e. VMworld, Cloud**, etc.) including consultants, organizational design firms, HR firms, IT training and talent management organizations, certification boards. etc. This would be where firms could hawk their services and compete for the non-product enablers for the CIO. Show producers ought to offer incentives to capture these firms and the value they provide.

I'd also like to see more intentionally-produced tracks at the shows that talk about real-world IT transformation successes... without inclusion of product or technology. Topics? Financial transparency. Organizational design. Service-oriented design. Line-of-business customer focus.

Where Are the Big Guys?

Next, why aren't Accenture, Deloitte, CSC, Cap Gemini, Accenture etc blazing the IT change management trails? Perhaps they have IT change management practices. But I read next-to-nothing from these folks regarding philosophy, approach or successes. I'd like to know more about what they're doing and/or planning.

From where I sit, most System Integrators and management consultants either have 'low-level' practices where they thrown technologists at a problem, or high-level management consultants who produce reports and strategies. I'd like to see the middle-ground, with practices that work with the CIO and their lieutenants, to assess what type of transformation(s) would make the IT organization most valuable to the lines of business.

Where are the books, keynotes, articles and methodologies developed to show-off the IP I *hope* these folks are developing?

And What of Professional Certifications?

No, not technology certifications; I mean IT Skill certs. Cloud Architect. Cloud Engineer. IT Services/Product Management. IT Marketing (inbound, outbound). IT Business Liaison. IT Financial Services Management. Not all of these are traditional IT roles, but they will be in demand shortly as IT re-tools itself to act and operate as the equivalent of an internal Service Provider.

And while we're at it, I'd *love* to see more business and MBA programs offer technology leadership and change management for CIO's style coursework. Technology leaders will no longer need to be technologists themselves.

And in closing...

At EMC's recent IT Leadership council, it was clear to attendees that Transformation of the IT org was not about the technology anymore. It was about skills, roles, organizations, culture.

I am continually impressed by what EMC's IT department has done in most if not all of these domains. And I know that our own EMC Consulting arm has deep bench strength in this area. And as a company, we're going so far as to offer professional certifications in these emerging domains.

But as I said above, a rising tide lifts all ships.... I'd like to see broader activity in all of these areas from partners and competitors alike. The best way to make the new cloud computing paradigm successful is to make the IT departments successful in their transition. So, where is the help?

Post-Blog Updates

Recent announcement from Deloitte for Cloud Service Providers

IT Leadership: Two Days with Leading CIOs

2011-10-20T23:52:00.000-07:00

I'm writing this at Logan airport as I fly home after two full days in Boston, attending EMC’s IT Leadership Council. At it, senior EMC staff were able to spend nearly two full days with a room full of IT leaders from some of the world’s largest and most respected companies.

What made the time unique was the quality of interactions as well as heady topics. EMC’s own leaders and practitioners interacted with our leading customers about IT Transformation, IT as a Service, managing IT change, and cloud computing futures. But the conversations were rarely about technology, and did not touch on product at all. Rather, they focused on challenges facing IT today and on IT’s renewed role to support the business.

This was the first time EMC has attempted something so ambitious with such a senior audience. And the results were fantastic -

For our customers: EMC was able to share our experience in transforming our own IT, helping to validate many of their approaches. Our customers got to interact directly with our internal consultants and practitioners, and exchange views on best practices, pitfalls and works-in-progress. And I believe we also transformed our customer’s view of EMC's knowledge and capabilities as an enterprise vendor/partner.
For EMC: We had a unique chance to get out of the “weeds” of technical details, and to focus on the high-level business drivers that are really facing (and affecting) our customers. This was a chance to hear the unfiltered voices of the customers first-hand, for two days. Our thanks to them cannot be overstated.

It’s hard to summarize all of the themes from all of the sessions – including dozens of breakouts on various topics – but follows are my personal takeaways and “high points” from many individual and group conversations:

IT transformation isn’t about technology: Almost everyone was in agreement on this. The technology problems can be solved for. But the real barriers to IT reinventing itself lie in the area of new operational and organizational models, evolving roles and skills, and new financial models. Often-heard was “My technology is ready. My people are not”.

IT leader’s focus: To support business agility: Yes, IT agility and infrastructure agility were still points of conversation. But more important was providing business agility – the ability to help lines-of-business be more productive, more profitable and more competitive. Linking the business case between cloud, IT investment and LoB top-line is becoming an increasingly important strategic conversation for the enterprise.

IT will compete for business: This theme was becoming more prevalent. Users are turning to external service and cloud providers because of pricing and/or convenience. Sometimes termed “shadow IT”, internal IT now has to think of itself as having to “win the business” from lines-of-business. It has to reinvent itself as a competitive internal Service Provider (and/or service broker) to the business. IT is now rarely the only game in town.

Public Cloud isn’t (always) the panacea: There were more than a few customers – mainly banks, government contractors and the like – for whom the public cloud is simply a non-starter, usually due to regulations and compliance needs. But private cloud remained appealing. They were eager to learn more about private clouds and the IT transformation needed to make them productive.

IT must move away from a “hero” culture: IT heroes used to embody all of the organization’s tribal knowledge, and could parachute into a problem and solve it at any time of the day or night. But this SWAT culture has to make way for the “new guard”, consisting of IT staffs trained as generalists, who can work with increasing levels standardization, automation and shared infrastructure. Many practitioners agreed that entirely new staff rewards systems needed to replace those that awarded those with superhero powers.

Marketing? In IT? As IT shifts to becoming an internal service provider that competes for business, it’s also faced with acting like a business unit – replete with marketing functions. And these are skills that are somewhat alien to the organization. They include outbound marketing: publicizing and actively marketing their services to business units to drive demand, and inbound marketing: working alongside customers to identify needs and requirements for future product/service development.

Financial transparency more important than ever: To compete, to forecast, and to model, IT has to know its per-unit costs, whether or not chargeback/showback is implemented. Knowing capital, operational and incremental costs helps the organization make better buy-versus-build decisions, and allows the rest of the enterprise to make similar decisions about how/where to apply infrastructure. And most important, if/when IT chooses to price services, they do so with detailed information.

Overall, IT is having “new and unfamiliar” conversations: IT is talking with lines-of business regarding business agility. They are learning about inbound and outbound marketing skills. They are discussing competing against consumer service - and/or brokering them. They are being asked to support any and all employee-provided devices. They’re entertaining buy vs. build for application services. They are being asked to meet or exceed demand for services… rather than to limit them. And they’re shifting from structured data and heavyweight GUIs to unstructured data using throw-away apps.

Much debate was had over whether IT is going through a Transformation or an Evolution. My answer? Evolution takes an awfully long time.

Event Agenda

Practitioner’s View: IT Transformation and GreenPages’ Solutions Architect

2011-10-18T19:44:00.005-07:00

Last week I had the chance to speak with EMC Velocity Partner GreenPages – specifically with Trevor Williamson, their director of Solutions Architecture. Trevor regularly speaks with CIO’s from Enterprises and SMBs about cloud computing and IT transformation. He has a refreshingly balanced perspective on technology… and on its impact on business. Follows are some paraphrases from our conversation late last week.

Lots of Talk About Cloud But…

Not surprisingly, Trevor noted that 80% of his customers are talking about cloud computing (who isn’t?) but only really 20% are doing something about it.

Why: He’s noticed lots of reasons – having to do with resistance to change, the demographics of IT leadership, and more. Trevor’s analogy was to that of an automobile: Traditional IT looks like a basic car with a basic engine and basic drive train. Yeah, it basically keeps running for ever. But if you wanted to change something, good luck. There wasn’t much room for upgrading without changing the whole thing.   Next, enter virtualization… akin to a car with Fuel Injection. It’s an upgrade – and the engine will run more efficiently. But in the very grand scheme of things, it’s still a point-solution (although quite valuable). Now finally, enter the concepts of automation and Cloud Computing: where the car’s components become completely different and independent – no piston engine, an electric drive train, etc. So it’s much easier to upgrade components without needing to alter others. Simply increasing the battery gives better range. Changing the electric motors gives better performance. Changing the engine might speed battery recharge. But components now operate more distinctly and can be optimized independently.

On IT Finance, and Why It Might Be the Key to Change

Trevor also had a terribly pragmatic perspective on a simple but ingrained factor holding-back IT’s efficiency.   In the traditional (read: old) IT financial model, the IT Project Number was the key to all implementations. Any change to a specific system got charged to that number. And the dollar size of that number got watched under a microscope. However, bounding the infrastructure’s financial model with a single project number reinforced the technological stove-pipe-ness of the assets themselves. The financial model did not allow for any sharing of assets between systems – because each new asset of course had to be assigned to a single project number. So, one result was the wasteful sizing of each silo - and resulting horrific utilization of each silo. Ergo, Capital inefficiency became a necessary result of a limited financial model.

Enter: Shared infrastructure that can meter incremental IT usage, and cost-accounting that can keep track of (and assign) incremental costs. With these two tools, IT could now implement a shared (and, needless to say, virtualized) infrastructure. The improvement in utilization, and resulting improvement in capital efficiency, was and is a boon to IT.

So, Who Calls the Play?

When an IT department decides to go down the path of a private cloud, who makes the call?

Well, there are usually two models Trevor and GreenPages frequently see – either top-down (by decree), or organic (bottoms-up). More often-than-not, Trevor believes most initial deployments are bottoms-up. The typically the initial application is internal to IT, such as QA/Dev/Test – also because these use case customer needs are very dynamic and fast-moving. Giving the engineers a self-service portal and ability to spin-up tons of cookie-cutter environments is a great fit and a great win. And doing so internal to IT (in contrast to line-of-business apps) minimizes the number of approvers… also speeding along the project.

Most initial bottoms-up automation and cloud initiatives are relatively new implementations, *not* system refreshes – which surprised me. But for good reason: Refreshes usually involve a cross-application and cross-domain initiative. Which means more approvers and multiple teams.

Now, there are times of course when IT leadership “decrees” building a private cloud – and mostly the impetus is either to reduce costs, or to cater to the LoBs who are looking for agility that will result in making more money. But when these initiatives are by leadership decree, they’re usually bigger (and slower) than their internal organic cousins. In fact, the top-down initiatives are usually corporate-scale, even to the point where they’re ‘branded’ and ‘marketed’ cross-org. But reading between the lines, all of the hoopla doesn’t make the top-down initiatives any faster than the internal approach.

Is building an Internal Cloud Bad (for IT Jobs)?

Trevor gets this question a lot. He’s not yet seen an instance where virtualizing and automating infrastructure has meant a reduction in jobs. Instead, he’s found more the case that it’s good for the *existing* jobs. Building a private cloud helps IT personnel climb out of day-to-day fray, and frees-up their time for more interesting, higher-value work for the company.   We both noted that IT should be focused on extracting value from company data and partnering with the lines-of-business…. Rather than “keeping the lights on” laboring as a monotonous workhorse.

And Finally…. On Keeping IT Competitive

“Why does business go around IT?” I asked. Trevor was pretty firm on this one: Only when IT can get out of its own way and move at “market speed” will internal customers will choose them. He sees lots of use of external services by lines-of-business, and lots of IT organizations racing to keep up with them.

How to achieve market speed? Part of the answer is re-engineering IT as an internal service provider / service broker. IT must think of itself as a service bureau, right down to marketing its services to users in the enterprise. From an infrastructure perspective, IT also has to become more agile. It has to “stop plugging holes with humans” and start by leveraging automation as part of its DNA. Lots of point systems can be automated, e.g. provisioning OS, virtualization platforms, I/O, networking. And only then can Orchestration be layered across those automated systems to attain the speed and agility needed. Take it a step at a time, with the goal being competitive and customer-focused.

A Leading Indicator of "Consumerized IT"?

2011-10-10T07:27:00.000-07:00

Let me start with a game: Try to guess the type of customer I met with last week...

Roughly 5,000 users with ~ 25% annual turnover
Pressured to support multiple user-provided devices (Smart phones, iPads, etc.)
Significant "Shadow IT" (e.g. users turning to external web services w/out IT's knowledge)
Inquiring into a VDI environment to increase manageability, reduce costs
Increasingly huge storage needs, esp. for video (on-demand media, surveillance, etc.)

Give up? Well, this customer was a High School district, with thousands of students and hundreds of faculty. And they shocked me with their challenges, ones which I assumed only large enterprises had. I have to admit I was pretty surprised at their level of sophistication - and their vision.

And then it dawned on me: These guys actually more of a "leading indicator" than most their commercial enterprise cousins. Their internal customers are nearly all Millennials, arguably leading users of consumer technology, and sometimes the most demanding. The student base requires support for "any device, any time", and doesn't hesitate to use external commercial services rather than IT (think: kids using DropBox, Evernote, Mobile Me, etc.). Also, users (students) without their own personal "smart" devices will typically log into multiple alternative devices a number of times each day, requiring virtual display and authentication technologies.

So, think about it: These students - and their appetite for "consumerized" technologies, will be your workforce of tomorrow.

What that high school IT department is facing is what most IT will be facing during the next few years. So consider carefully who (and what) you'll need to support in the coming years. And consider the organizational, technological, and governance transitions you'll need to implement as well.

In a recent Forrester Research report, "Shifting from Rules to Guardrails" there are some good observations of how IT (and the CIO) have to transform from being technologists to becoming brokers, vendor managers, and providers of technology governance:

"We see a blurring boundary between business and IT. The rise of self-service technologies, driven by empowered employees, provides new opportunities for IT to improve business responsiveness by enabling greater autonomy... To do this successfully, IT must evolve new governance approaches that empower the business with guardrails and education, reserving strict technology control for only the most critical technology assets. For many, this will be a radical change.

So, my lesson was that the places to look for Leading Indicators of IT transformation aren't necessarily the analysts and brainiac IT leaders... Instead, perhaps I should focus more on the needs of the 9th grade entering class.

I hope to stay in touch with the IT staff at the school district, and see what priorities (and solutions) they see emerging in the coming months. As I exited the meeting, one of the IT leaders took me aside and also mentioned that IT has also experienced - not surprisingly - hacker break-ins as well. As I said, perhaps the youngsters are where to look...

Other Resources

Fountainhead: New models for Consumption, Operations, and Technology
Fountainhead: IT-as-a-Service: IT competing for business vs. "Shadow IT"
Chuck's Blog: Learning to Compete - The real challenge of ITaaS
Forbes: Consumerization of IT raises new security challenges
CIO.com: Consumerization of IT taking its Toll on IT managers

Documenting IT Transformation Since 2005

2011-09-19T17:38:00.000-07:00

If you haven't looked at how EMC's IT department has transformed itself over the years, you should - it's amazingly illustrative. Our CIO, Sanjay Mirchandani is visible and articulate. Our VP of infrastructure, Jon Peirce, is equally eloquent about how IT is transforming - and was a major force behind EMC's recently dedicated state-of-the-art Datacenter. And of course Chuck Hollis has been doing an amazing job of chronicling the meaning of IT transformation, consumption models and organizational changes.

First, show me the money

But where does that leave us regarding exactly what we achieved? What metrics illustrate the real benefits of all this work? Well, it turns out that EMC IT has been documenting their process since about 2005. While not all data points are completely audited, they tell a powerful story.

In the conceptual 3-phase "Journey to Your Cloud" we speak about the trend toward increased virtualization - and how it's applied. In Phase I, IT virtualizes "internal" IT-owned apps saving primarily on cost. In Phase II, IT virtualizes production applications, benefiting primarily quality-of-service and availability. And in Phase III, IT is operated as a service, primarily improving overall business agility.

But along a pragmatic journey, the benefits occur gradually, sometimes in parallel.

EMC's IT group has been virtualizing its infrastructure - and documenting the results - since 2005. At right, the chart depicts how virtualization toward a cloud infrastructure undeniably results in improved infrastructure economics AND improved IT agility. And their trend continues.

The other benefit from virtualization is the ability to re-construct an environment should failure occur. This impact on quality of service is available anywhere virtualization is present - so you'd expect the pervasiveness of Highly-Available environments to roughly track the degree to which virtualization is implemented.

Sure enough, as EMC virtualizes its entire infrastructure (and we're not entirely there yet) we're pushing nearly 97% of applications with High Availability.

Finally, as IT infrastructure is virtualized and consolidated, utilization rises, and (presumably) physical data center space required (not to mention power) is reduced.The EMC IT group's statistics clearly show this trend as well.

In fact, EMC's recently-christened data center is not larger than our existing data center (as it was mainly developed to economize on power cost, and to create a physically-distanced site from EMC's HQ) so there is still lots of room to grow.

In the coming months, EMC, our IT department, and our consulting arm, will be continually elaborating on our own journey through transforming our IT... not just the technology, but the organizational models, operational models, consumption models, and more.

I personally have to say that the degree of rigor that Sanjay & team have put into this transformation is impressive. They have a very clear vision of where they want to be with respect to enabling the EMC business, as well as setting a clear example of what is possible for EMC customers.

Other References

An Image Makeover for IT

2011-09-13T13:31:00.000-07:00

Wow. I just have to share notes about a meeting today - one that might have been unthinkable a year or so ago.

I've been working closely with EMC's CIO and internal IT groups lately, helping understand how they're delivering IT as a Service. EMC's IT is going through a major transformation, where it's becoming more competitive, more business-centric, more like an internal Service Provider.

What blew me away was a conversation I was invited into because I was in marketing. Could I help? IT wanted to begin investigating how to market its on-demand/ITaaS services internally to EMC lines-of-business. We discussed audience segmentation, how IT would like to be perceived (read: Brand Essence), and the services/value IT provides to assist the business.What should a marketing and roll-out plan look like? Graphic designs? Should IT instead be called BT (Business Technology)? How would the value of Service Manager roles be highlighted?

The discussion - while well from over - I believe is a harbinger for the "new" IT, where IT provides value and competitive advantage to other lines-of-business, where it acts like a strategic partner/vendor, and competes for business. It's so much more than technology.

We agreed that another harbinger of change was when the CIO spends more time with the CMO, rather than with the CFO. And I can attest to being present to one such meeting.

So, in the process of IT elevating its value-add, IT will enhance how it caters to the business. It will actively promote its new service orientation, its new operating model, its new mindset.

And I am more than happy to help.

Related

IT-as-a-Service: Models for Consumption, Operations, Technology

2011-08-30T15:06:00.000-07:00

In a week where products surely dominate the news (it's VMworld, in case you live under a rock) I want to share some non-product insights I've made while working with EMC's IT group regarding running IT as a Service.

First, running IT as a Service doesn't mean outsourcing IT. It's about transforming corporate IT into thinking of itself as an internal service provider. Catering to internal Line-of-Business customer needs. Providing pricing and rate sheets. Offering financial transparency. Even marketing against and competing against (external) services.

IT as a Service isn't just "more virtualization"

The market is equating virtualization with cloud, and worse, equating cloud with IT Transformation. And by IT Transformation, I mean the re-casting of the IT department to operate like an internal service provider, and to act like a business. In my opinion, Cloud (use whatever definition you prefer) is really just a critical enabler for IT transformation.

In both EMC's and VMware's similar 3-step "Journey to Your Cloud" models, at first glance you'd think that the end-goal is more virtualization and more automation. But what we at EMC (and a number of customers) have found is that the end-state is where the technology revolution winds down, and the organizational evolution begins to spin-up. Just having the technology in place is necessary, but not sufficient, for a transformation of how IT operates. So, as I've said, Cloud is the means, but not the end.

Our IT department has over 79% of services virtualized and running on standard platforms. And arguably the virtualization initiative is beginning to be completed. (Aside: The new VMware 5.0 product line supports even bigger VMs and leverages CPUs with more cores - allowing for production-scale databases to now be virtualized). Manpower is now being diverted into the NEW areas needing resourcing: Business Transformation of IT, and the services packaging, marketing, and management that comes along with it. Another way of putting it: Now that the technology is posing less of a problem to design and manage, more time is being spent working with EMC's Lines-of-Business to cater to their needs and enhance their business agility.

3 Tenets of IT Transformation

After conversations with our CIO Sanjay Mirchandani and our VP of Infrastructure, Jon Peirce, it's clear that this final phase of the "journey" is only partly comprised of technology. At the core, there are 3 broad areas of focus when transforming IT: Consumption models, Operations models, and Technology models.

IT Consumption Models
Essentially this describes how supply is separated from demand, and that service capacity is created just-in-time. Services are generated from an approved inventory stored in a service catalog (self-service, if you're in IT), with each service having clear pricing, SLAs etc. The pricing could be variable and "consumption based" that is, it's not only metered, but based on both true cost as well as opportunity cost for access to the infrastructure. But expect new challenges for IT finance. Lastly, the consumption model can also include services brokering - that is, it can include services generated from outside IT as well as those generated within/by IT. Both sources are equally valid, so long as IT still provides common governance, access, pricing and secure delivery to internal LoB customers.
IT Operations Models
Another non-technological shift is how IT operations changes, morphs and grows. One would expect that the operations mechanisms become more automated with less human intervention. But the real shift in operations is the *mindset* of IT, shifting from a "technology builder" to a "service delivery" organization. This fundamental shift focuses on delivering services (internal, external etc.) to the internal LoB customer to meet their needs/requests. A business mindset might also mean that IT will have to grow roles that sound like "service manager" "service product manager" and "client marketing manager" skill that IT will need eventually to acquire/develop. Which ultimately implies a good dose of Change Management will be necessary - that the IT organization, skills, roles, goals, etc. will shift over time. (a very excellent blog by Chuck Hollis dives into this topic)
Technology Models
Last but not least is technology - but this part of the story isn't so much about new technology so much as it is about how technology is used... and by whom. Necessary, but not sufficient, is the use of a virtualized, automated, and converged infrastructure (cloud componentry). Sans jargon, I mean an infrastructure that is virtual and pooled, so that it can be composed on-the-fly as business conditions require. Since this departs from the traditional vertical stack model where IT personnel *skills* are specialized, new skills will be in demand. We'll need more IT generalists, IT staff with horizontal *services delivery* skills, not point-product skills, and with comfort around automation. Oh - and the technology *roles* and *organizations* will change too to be consistent with the new model.

And finally, the need for Change Management

No, I don't mean Change Management in the context of ITIL/ITSM. Rather, it's in the context of organizational design and development (think: Management Consulting jargon). With all these new models, Change Management is critical to orchestrating the *non-technical* shifts in IT. Our own EMC IT department employs a number of these types (ex-BCG, McKinsey, etc. folks) who help our staff develop new skill-sets, morph org structures, and create new engagement models for the business owners.

IT transformation does not simply happen as a result of new technology. Rather, all of the products you buy are simply enablers to help you get to the really Hard Work. But the payoff is enormous.

Cloud Is Not The End - It's The Means.

2011-08-09T15:56:00.000-07:00

Once you get a new tool, you ask: What can I build that I couldn't before?

Cloud computing, at its core, is an operational transformation - mostly focused on infrastructure. If you're an enterprise IT shop, you're right to think that such a change will (ultimately) simplify your world. But simplification, faster time-to-provision, and on-demand capacity aren't the end points of IT transformation. They are the new tools that are the Means to a more aspirational end.

What every business craves is the ability to respond to new ideas (innovation) and to market pressures (competitive, consumers) faster and more completely - Agility. A recent McKinsey study shows that the top 3 metrics of "agility" were centered on revenue growth rather than cost reduction - signalling that companies value growth over expense-cutting. And IT is the chief approach to enabling revenue growth for many enterprises.

But merely having faster infrastructure is necessary but not sufficient for an enterprise to achieve real business agility.

My thesis is that Cloud computing - whether Private cloud, or a mix of Private/Public (Hybrid) - is really the means to a bigger end: Enabling IT to serve and enable the business, rather than simply respond to technology requests. Think of IT as an internal Service Provider (ITaaS) - developing, marketing, pricing and refining technology to meet the specific needs of Line-of-Business users. Forrester research even makes the valid observation that IT (information technology) needs to undergo the conceptual transition to BT (business technology).

The Goal is IT (I'll continue to use the term for now) that is structured and goaled to serve the business. And that is a whole lot more than just providing a virtualized cloud infrastructure.

A great overview of this ITaaS restructure was recently written in Chuck's Blog, where he outlines the skill, organizational, and financial transitions that must necessarily accompany the infrastructure transformation.

In his blog, Chuck cites an oft-used slide by John Peirce, EMC's VP of IT Infrastructure and Services.

This is a great analogy to how IT was traditionally (and will be) built and operated. In effect, IT will continue to progress toward an on-demand, if not cloud-like resource that business can tap into.

But consider the corresponding change to how the IT organization itself will be built, run, and operated. In fact, consider how IT skillsets will need to change, and how it will need to partner with the lines-of-business to ensure that they're equipped with the right technology at the right time. In fact, if the "new" IT does its job right, it will even work with the business owner to understand their business better, and suggest new tools (think Big Data analytics, etc.) that might add even more value.

So I then got to thinking, we need a companion slide: That ITaaS transformation isn't about infrastructure only... it includes how IT works with the business as a Service Provider - at times actually competing for business against "Shadow IT" from external sources. There are for basic facets to think about this transition

From "monopoly" to "market": Rather than IT being "the only game in town", Shadow IT is causing indirect competition - where IT will have to offer and price services in a manner (and speed) that will cause internal customers to want to purchase from them.
From vertical to horizontal: where the organization shifts from stack-focused to service-focused. Literally, the orgcharts and skills-sets change over time. Chuck also does a great job of explaining EMC-IT's transformation over time)
From "enterprise tax" to consumerized pricing: Rather than the fixed-price (frequently capital-expense) based pricing for standing-up a stack, IT will shift to a variable-priced model based on consumer needs and competitive pricing
From IT as a cost center to IT as a center of value, where IT teams with the LoB to create and offer services that move the business - whose job it is to generate revenue - forward.

These 4 areas don't necessarily assume there is a cloud infrastructure in place - and indeed, don't require one. But to get the value out of a cloud infrastructure, you do require to implement them.

The desired end-game is for the enterprise to be more competitive, responsive, and agile. Cloud is an enabler - but don't overlook what needs to be paired with technology to get the full effect.

In future Blogs I will plan to go deeper into each area, exploring best practices, as well as how our own EMC IT department is faring on their own journey.

More Info

IT Operations in a Cloudy World (James Urquhart)
From Silos to Services (Chuck Hollis)
EMC-IT on the Cloud Journey

IT-as-a-Service: IT Competing for Business vs. “Shadow IT”

2011-07-27T14:28:00.000-07:00

As I begin to sink my teeth into the realities of IT Transformation and the operational change to IT-as-a-Service (ITaaS), it’s becoming shockingly clear that adoption challenges aren't technology issues.

Although debate continues over what cloud computing means, clarity is beginning to take shape as public/commodity cloud, private cloud, and hybrid cloud models evolve.

So if we now know how to build clouds, where does that leave our IT operations? What of our IT organization, skill-sets and CIO’s? How does the technology map to enable lines of business? How will infrastructure change the game for the enterprise?

IMHO, these are the questions we still must answer in order for “cloud” to be the next successful model for IT.

Enter IT-as-a-Service

Much the way that the internal combustion engine was the technology transformation catalyst for new forms of transportation and resulting commerce, cloud technology is transforming how information infrastructure impacts organizations and business models. The Technology is the enabler of The Services. But the automobile alone didn’t alter the landscape. It needed infrastructure, customization, and even rules for safe operation. Same goes for IT technology.

EMC’s own IT department, led significantly by Jon Peirce, VP of EMC’s IT and Private Cloud Infrastructure & Services, thinks of the infrastructure relationship this way:

IT as a Service is a delivery model leverages cloud infrastructure to enable business users to be more agile through readily-consumable IT services that have transparent prices and service levels. While it is built on technology, ITaaS isn’t a technology. It is an operational model that transforms our traditional approach to IT into a services-based world.

Good timing. Because there is another trend afoot: The emerging external set of services – from public cloud service providers – to attempt to compete for the same attention. And dollars.

Competing with “Shadow IT”

John observes that IT’s days as a “monopoly” on technology are gone because

Users are global, mobile and social, with impatience for having information at their fingertips. They’ll instantly use any alternative if it’s accessible. IT needs to plan for this – or have a competitive alternative
Access: iPads and other edge devices are pervasive. The days of “IT-approved” access devices (the corporate-issued laptop) are numbered. Users will demand their own type/style of devices.
Public clouds are clamoring for developer’s attention and $. Essentially developer with a credit card has the potential to release corporate IP to the outside. IT needs a model to deal with this… and an attractive alternative.
SaaS alternatives are courting business managers. And worse, IT isn’t necessarily informed when business managers use these services. Governance and access models need to be created, since there will always be external SaaS options.

So, as users and LoB’s turn outside the company, this “Shadow IT” phenomenon arises : the use of external IT resources. Appealing because of their on-demand nature, yet dangerous because of their security porosity, lack of usage governance, and lack of financial transparency/control.

So IT finds itself in a competitive position vs. Shadow IT.

John then asks a question this way: If our internal line-of-business customers had a choice, would they use us (Enterprise IT)? When IT was the only game in town, it didn’t matter what they charged or how good the service was because LoB’s had no choice. But now there is. So we have an unavoidable imperative to be more competitive.

Unavoidable Implications for the New IT

As I think about IT Transformation to IT-as-a-Service-for-the-business, there are two implications that are inevitable and unavoidable.

IT cannot resist this transformation. It will be forced upon them because of the use of, and competition from, Shadow IT - as well as from the increased demands from LoB. So IT needs to be better-acquainted with the competition, their services, their SLA’s, their pricing. Like any competitive situation, IT needs to do *external* benchmarking in all of these areas. Because if they don’t their CFO will do it for them.
IT needs to think competitively. IT orgs need to think in terms of winning the internal business by actively selling and creating demand for products (services). This is opposite from how they’ve been conditioned to behave – so IT has to develop basic business skills and even organizations to operate in a competitive business environment. These include product marketing, product management, financial management, and even competitive analysis and sales skills.

This is an exciting time for IT. And while most are focusing on the technology, I urge you to look at the business and operational aspects of this change. While any change can be scary at first, it also can provide a brand new set of competitive opportunities for the business.

For more info:

Is Shadow IT or Rogue IT Fueling the Cloud? (Richard Ingram)
Cloud DevOps and Shadow IT (James Urquhart)
Cloud: The Ultimate Shadow IT (HP Grounded in the Cloud)
Organizing from Silos to Services (Chuck Hollis)
The Cloud Concierge: The new CIO - Creating IT as a Platform (Brian Gracely)

A Morning With State Government... Talking IT Transformation

2011-07-13T22:31:00.000-07:00

Think your enterprise is challenged as it struggles to move toward IT-as-a-Service and a shared IT services model? It seems that state and local governments are also trailblazing as well.

Earlier this week I had the honor of spending a morning in EMC's Executive Briefing Center with various members of a state legislature - and members of their IT staff - looking to learn more about their investment in a Vblock, and how it could enable a shared services infrastructure that could save them $ millions while upping services to citizenry.

This was not a technical crowd in the least. These were state representatives with constituencies who cared about things like better services and lower-cost government. But they wanted to know that they'd chosen the right horse, the right technology.

But what was fascinating was nobody wanted to drill into the technology... or even really get educated about it. Rather, they were simply acutely aware of the opportunity to save money while upping service. They knew that government agency data centers were siloed. They realized how long it took to deliver new IT services. They acknowledged how un-integrated inter-departmental state data was. But they all wanted to be part of the solution, to get the rest of the legislature to a point of appreciating the opportunity before them.

The notes I took might sound familiar:

Where do we start? VDI sounds like a shoe-in. But after that, which departments, offices and data centers should become part of the shared-services model? [What workload migration and ROI model should they adopt?]
If we do end up saving money, there's the risk that the savings will be taken away from us - how do we ensure it's plowed-back into innovating and creating higher-level services? [How to meter IT costs? What higher-level services could be proposed to the lines-of-business? How to facilitate IT educating departmental management in what new opportunities are available?]
Every office and department feels like they have a "special IT need" that only their own data center can provide. Is that really true? [How to illustrate the versatility of a cloud environment? How to guarantee differentiated SLA's?]
With a shared infrastructure, how do we ensure that sensitive information (e.g. the Highway Patrol department) is kept secure from prying eyes of other parts of IT... and indeed, other parts of the state government? [How to illustrate multi-tenancy? security? auditability?]
How can we ultimately simplify the government experience for citizens? e.g. Reduce paperwork for driver's licensing? Work permitting? Unemployment applications? [How to go about merging and analyzing structured and semi-structured data from diverse sources?]

While this was not the forum to solve the problems, by the end of the morning we were all happy to have the issues laid-out on the table for discussion. And to have educated the policy-makers and users that they can in fact operate their state government IT as a 21st century infrastructure.

The other good news is that this state is not the first to make this transition. A very good initial resource, for example, is from the Center for Digital Government - their paper on IT-as-a-Service for State and Local Government which gives a number of very good examples of state governments taking the right steps for the right rationales. There is also an excellent paper published by the US Department of the Interior and their IT Transformation plan.

Stay tuned on more of what IT Transformation makes possible, and how to migrate to a service-based IT organization.... ITaaS.

Real-World Financial Services Cloud

2011-06-01T16:56:00.000-07:00

Cloud computing has been made very real today.

As I subtly alluded-to in March, The New York Stock Exchange (NYSE Euronext) today launched their Capital Markets Community Platform (CMCP) along with partners EMC and VMware. It's essentially a high-performance, low-latency, special-purpose cloud IaaS, replete with customers and roadmap. Register Coverage FIN Alternatives Coverage

This is a very notable event for a few reasons:

Cloud is not a commodity: Unlike general-purpose public clouds, NYSE has constructed a high-performance, low-latency infrastructure to meet the specific needs of trading firms. From these perspectives alone, use of a public cloud (AMZN, RAX, etc.) would never meet the stringent performance requirements. My belief is that we'll see even more of these industry-specific clouds arise. Differentiators will likely vary based on needs for performance, privacy, security, scale, etc.
Cloud is highly reliable: A lingering question has been whether the cloud - and associated automation controls - was reliable enough for mission-critical applications. NYSE is no stranger to Financial-Markets levels of reliability, and has clearly taken great pains to ensure that their experience carries-over to the CMCP.
Cloud is highly secure: Ditto to above. the CMCP is accessible to customers only via a highly-secure network and only to pre-validated users.
Cloud enables new forms of business: For me, this is the most exciting aspect. NYSE's cloud now allows small firms (picture 3 hedge fund managers and their dog in a garage) to take advantage of enterprise-grade hardware and data... say to test and run new trading algorithms. Access to resources such as this would have been far outside of the reaches of the small firm.
The Cloud + Big Data story is real: What's also nifty about NYSE's implementation plan is that it allows users to create DB's on demand, and will allow users to access massive data in the form of market play-backs. This DBaaS will obviate the need for tenants to replicate TB or even PB of data as they test algorithms against historic market data.

NYSE's partnered with EMC and VMware to construct the cloud, and VMW has also posted an excellent Blog on the topic. A few excerpts:

"So, why is this better, and why is NYSE Technologies the right organization to deliver? For hedge funds and other buy-side firms, their value isn't in integrating compute, storage, networks and security -- it's in analytics, trading strategies, algorithms, application strategies and other proprietary expertise. The NYSE service means those IT organizations no longer have to struggle with integrating data dumps and feeds into their infrastructure and operations. Trade execution speed can be critical, so physical location and proximity to the market matters. NYSE's experience in operating large scale, mission-critical VMware-based infrastructure -- the NYSE and Euronext exchanges -- is unquestionable....

"...NYSE represents an alternative cloud future: one that contains a vibrant ecosystem of clouds, both internal IT departments and external cloud service providers, with unique understanding and focus on customer needs, married with the ability to deliver through scalable, on-demand and trustworthy IT services. What internal IT organizations and cloud providers like NYSE share is a rejection of the concept of an inflexible cloud monoculture. Instead, they choose to build high performance, secure and scalable infrastructure because it meets critical business needs. They obsessively focus on value delivered to the customer and never confuse that with cost of service.

And that's it. Cloud is now about Value, even more than it has ever been about cost reduction.

Hosting/Cloud Index: Update

2011-06-01T15:57:00.000-07:00

Back in December, 2009, I proposed an index of hosting and cloud providers, and posited that these sample portfolios would be an excellent gauge of the market's perception of the state of the business. I also made an update in December of 2010.

I thought I'd make an update again, and review the state of the business. Also, it would seem that others are adopting the same idea, as I was recently reminded by Software Advice. Good to see that these types of metrics are being adopted...hopefully they cut through some of the vendor hype.

Now, on with the statistics.

First, I created an index of a superset of publicly-traded hosting providers, some of whom also provide cloud computing resources. My only litmus test was that these were *not* SaaS providers, and that they provided hosting/IaaS services as their primary business. The performance was compared against the NASDAQ composite, and has been quite positive. The large increase in index price was primarily due to Verizon's acquisition of Terremark for a hefty premium.

Next, I took a subset of the providers who solely claimed to provide cloud-computing services.

As I had suspected, this index has vastly out-performed the NASDAQ - my supposition being that the market is placing a higher premium on any business conducting "cloud"-related business.

The market would appear to still be going strong, and I'll continue to update progress every few months.

It's All Just Data to Me

2011-05-29T09:20:00.000-07:00

Now that I’ve been with EMC for a few months, my relationship to storage, computing, and networking has once again shifted. And, in the context of the cloud computing operations model, my relationship to the physical location of data - and processing of that data - has shifted too.

My new perspective starts with Computer Science 101: Where, at its heart, computing is simply data and instructions (stored on similar media) which are combined on a device (CPU) and produce an output.

Since computing began, this model was consistent – but as the data and instructions grew in size and abstraction, the media changed to the point where instructions (code) and data, were each stored in physically separate locations.

Until recently the data and instructions would be transported (over the network) to individual physical CPUs (with their own sets of OS) where they would be combined and executed. And then, the resulting data generally was transported back to its place of residence.

Servers are Just Bits

Now, enter the Virtual Machine. At the heart of it, it's simply another file (e.g. VMDK) – in other words, just more data.

So in the modern virtualized data center, what we have – at the extreme – is a model where not only the data and instructions are bits… but the servers are bits too. All they require are physical CPUs to execute.

In the ‘traditional’ model, the data and instructions were brought to where the physical servers and O/S were. But today, with pervasive farms of generic physical servers, we have the situation where *either* the data bits can be brought to the server, or the server bits can be brought to the data.

Some of the implications you’ve probably already thought of – such as vMotion of a VM from one physical server to another, or using a DRS-style control to re-locate VMs from failed physical compute resources elsewhere.

But consider another situation that’s happening with increasing frequency: The need to work with “Big Data” – such as running analytics on unstructured bits that could be on the Terabyte to Petabyte scale. Here is a case where it makes sense to send Mohamed to the mountain than the other way around… To literally re-locate the servers (which are, after all just data themselves) closer to, or co-incident with, the data.

Or, consider a “follow-the-moon” strategy for data center energy efficiency: where the most energy-efficient (and least expensive) physical servers are chosen to handle workloads. Once again, the data (which includes the virtual server, data and instructions) is simply transported to the optimal set of physical processing resources.

Cloud Infrastructure and Data Management

From where I sit, the importance of data storage, data management and data portability suddenly becomes paramount. It can reasonably be argued that physical servers are now merely execution platforms for the VM data bits, and that the network is simply becoming flatter and fatter.

So the future data center and cloud model might be thought about as a data management problem. Where and how to locate bits, back-up bits, scale bits, operate on bits. True, this is a data-centric view of the world. But it's also a healthy perspective from which to view the renewed importance of data and its dynamics, versus the other more static components of the data center.

Cloud's Transformation: The Softer Side

2011-04-24T18:29:00.000-07:00

After having spoken to numerous customers and vendors, it's clear to me that cloud computing's operational transformation necessarily triggers structural changes in the IT organization - as well as in the rest of the enterprise.

Overheard at a conference late last year, an analyst I was briefing illustrated it this way: A Converged infrastructure requires a converged organization to operate it.

I'm convinced we'll see significant internal transformation in the future - not of technology, but of people, roles, skill-sets, and organizations. As evidence, just take a look at the organizational transformation EMC's IT department has gone through in the past 3 years (HT to Chuck's Blog)

Consider this:

The Role of the CIO: Today the CIO is orchestrator of technologies, if not a technologist him/herself. Governance of the technologies/vendors is perhaps secondary because "keeping the lights on" is such a dominating task. In the future, the role will shift from technologist to where the CIO (and IT overall) will become a service portfolio and governance manager... Regardless of whether the services are generated internally or externally. Implication: CIO's will need new skills, policies, processes.
IT Organizations: Referring again to Chuck's blog (and excellent illustrations therein) the IT organization will shift from siloed / distinct organizations to a set of unified service organizations leveraging a common services infrastructure. Implication: change management, goal changes, departmental funding changes.
Individual Skill-sets: Today's IT skills (esp. in larger organizations) are specialized around applications, servers, networking, backup, etc. each which aligns with the organizational structures, above. However, in the future many of these functions will either become more automated and/or combine with (be embedded within) other service management functions. Implication: new skills training, certifications, processes.
Supporting Services: As IT transforms, so will adjacent organizations and services - like finance, lines-of-business, legal/compliance, vendor/partner management. How IT is measured and accounted-for, related-to as a business partner, and how it dovetails with external partners/providers will necessarily shift. Implication: need for change management and new organizational design.

Looking forward, if these transformations occur even at a modest level, I would expect too see other broader-scale industry-wide changes in these and related areas.

CIO roles will shift to governance & vendor management (perhaps even modeling supply-chain management)
Organizational & change-management resources (firms facilitating change specific to IT transformation) will be in higher demand
IT skills development will re-invent itself; new training and certifications (e.g. cloud architect) will become the norm. Fewer special-purpose technologists will be needed, in favor of a new breed of "converged" technologists
Entirely new categories for job recruitment will emerge to find and place this new talent
IT financial management skills development, training etc. will be in further demand as IT shifts from being a high-dollar capital expense to becoming an on-demand business resource/enabler.

In the future I'll continue to reflect and blog about what I'm hearing in the market. But we should all be keenly aware of the non-technical impacts of the IT technology shift.

And, if you know of examples today, do share!

Marketing the Cloud

2011-04-04T12:29:00.000-07:00

Of all the marketing, marketing "The Cloud" has all the makings of a real challenge: The concept is new, the technology is disruptive, buyers are skeptical, hype abounds, and the terminology (just what is "cloud"?) is murky. So, when recently asked how do I "market the cloud" this Blog idea arose.

For me, marketing is far more than making "buzz" in the market. It's about matching seller and buyer: First, ensuring that the seller's product specifically targets one or more needs in the market (and adjusting as-needed), and second, ensuring that the buyers understand the product and its fit-for-their-purpose (and adjusting the buyer segments as-needed).

So, where a nascent concept, confused buyers, and evolving definitions are concerned, I turn to basics of new product introduction: (a) understanding customer problems/opportunities, (b) clearly defining the product/solution, (c) addressing objections, (d) helping customers through the adoption cycle.

Focus on specific issues the cloud addresses, not the Cloud itself: Before I recommend "cloud" as a solution, I ask myself what problems will customers really try to solve? They've heard “cloud” and it likely interested them, but for what reason? Getting to the need point is critical: It is cost? agility? keeping-up-with-the-Jones’? New business enablement? You have to first ask the business need question, not try to force-feed a solution. Usually the cloud model is compelling on nearly all levels - but the customer first needs to understand - and want to pursue - the opportunity. Good marketers ensure customers self-select into the solution, even if it's an extremely broad one. Also, an exercise I sometimes pursue is to avoid using the term "cloud" altogether during this phase. Instead, I focus on the attributes of cloud computing, and wait to hear whether they resonate with the customer's needs. Sometimes they might not.

Get clear on definitions - and use lots of adjectives: The next question to ask is: What cloud? Too often marketers of the cloud model don't modify the noun Cloud with an adjective like Private/Internal, Public, Hybrid, etc. causing even more confusion. It's alphabet soup. Many buyers usually start by thinking the only cloud is the public cloud. Once buyers are clear about the operational cloud model you're both talking about, you can have a more meaningful marketing action.

Know your buyer's technology maturity, and technology appetite: Different markets, segments and customers will have different technology appetites and be at different technology maturity states. So, as much as vendors want buyers to take a big step and buy all-new stuff, there has to be a spectrum of offerings to fit buyers at different stages of the maturity curve. (See "It's A Journey", below...)

Be pragmatic - identify resistance areas and objections: I say pragmatic, because everyone has their own list of objections and concerns. They might be trust/security/governance issues; economic models to justify the investment; the risk of moving to new operational models; dealing with change management (a change in IT will necessarily impact changes in related orgs); the list goes on. Make sure you've listened carefully to all objections, and thought-through responses. I've unfortunately seen wonderful products fail - not because they don't work, but because when it comes to implementation, all of the pot holes and speed bumps haven't been identified and addressed.

Be pragmatic - it's a Journey: Few buyer segments adopt all-new models - especially cloud - in their entirety on day-1. So marketers need to be prescriptive about where to start, what to do when, and how to help buyers with a roadmap that accelerates them down the path. Most cloud buyers (with the exceptions of folks like service providers) make incremental changes to infrastructure – so marketers have to help recommend the incremental changes (and products/services) they’ll need in the coming years.

Educate: Finally, I believe a rising tide lifts all boats. The more the market is educated about cloud computing models - and how to get there - the faster the market will mature. It's our job to help provide education tools, models and success stories. And to draw distinctions between here-and-now vs. futures vs. vision.

The opportunity we have with cloud is also a danger: There is an inordinate amount of hype in the space. So, as we move down the hype cycle, we need to get pragmatic about the value the cloud model offers, the journey customer take to implement, and the opportunities it creates.

A Community Cloud: Real-World Example

2011-03-24T22:58:00.000-07:00

When I first heard the term "Community Cloud" I shuddered. I thought: Just what we need... another cloud definition.

But I had a peek at one yesterday speaking with an established services customer (who must remain anonymous for the moment). They got their start building a co-location facility for companies in their specialized and highly-regulated industry. But it became obvious that they could add more value as a service provider than just supplying a cement slab, cooling and electrical outlets.

So they've set out to create a raw cloud IaaS infrastructure, but with some attributes that are specific to the community/ecosystem that they serve:

Security: Access to the cloud is granted only after a trusted validation of identity (required by regulating bodies) - and certain out-of-band management functions can only be made over hardware VPNs.
Availability: Cloud resources are available at roughly a five-9's level (or better) including complete fail-over and DR sites - this is uber-Enterprise-Grade availability.
Performance: Because of the specialized industry, the processing and networking performance of the cloud is optimized for high transaction rates and extremely low-latency.

Most other cloud properties, such as elasticity and metering are as you would expect.

Because of the special attributes, the company aims to become a special-purpose Cloud Service Provider to its industry - something that a generic AWS, Google or Rackspace could never be. And many other firms in the industry -- large and small -- will likely find both economic and performance advantages to host in its infrastructure.

Then, things really got interesting...

In addition to the raw IaaS they'll provide, they also plan to provide a special-purpose PaaS to tenants. For example, most clients will tend to use a common set of "Big Data" - ranging in size from Terabytes to Petabytes. If each tenant maintained their own instance of this data, it would be massively costly, inefficient and complex. So instead, the company will host a single, on-site shared instance of the data, charging for its access and use by users of the cloud. And they expect to offer a wide range of such PaaS services in the future.

What does this example say to me? That (as many predict) the market may in fact only support a very few number of generic IaaS providers who compete almost solely on cost and economies-of-scale. But, assuming this example is even partially successful, there will be room in the market for countless "community clouds" serving the special needs of enterprises and ecosystems globally.

I'd be interested to know if you're aware of opportunities (or instances) of other real-life specialized community clouds in your area of business. The era of cloud has only just begun.

Cloud Attributes Apply Across the Stack

2011-02-19T10:08:00.000-08:00

My “aha” moment here at EMC came during my first week when I was asked to describe the generic attributes of cloud infrastructure. Here I was, in an organization that’s made billions on storage, and I was about to talk about cloud attributes solely from a compute perspective.Was I missing something?

I then realized that I’d always related to storage as a “big, fat, dumb disk in the sky”, and assumed that it was merely subservient to the compute stack.

Well, not exactly.

My re-think was that attributes of Compute, Storage, and yes, Network, all had to be reconsidered in the context of a holistic cloud-based infrastructure.

Cloud Attributes:

Most will agree that the following attributes describe the operational profile of a generic cloud: (HT to IDC)

Shared, standard service. Built for a market (public), not a single customer
Solution packaged. A “turnkey” offering, integrates required resources
Self-service. Admin, provisioning; may require some “onboarding” support
Elastic scaling. Dynamic and fine grained
Usage-based pricing. Supported by service metering
Accessible via the Internet. Ubiquitous (authorized) network access
Standard UI technologies. Browsers, RIA clients, and underlying technologies
Published service interface/API. Web services and other common Internet APIs

I’ll add a few functional attributes as well:

Consolidation: ability to make optimal use of lower-level resources
Automation: ability to self-configure to provide the required service
Self-healing/failover: ability to correct for failure with little or no service interruption
Multi-tenancy, Multi-tiered-SLA: ability of resources to securely house individual services & service-levels across a shared infrastructure
Global availability: ability to provide a shared service across multiple availability zones

Attributes in a Storage Context

The first assumption most make is that these traits apply exclusively to the compute layer (physical servers, VMs and the like). But pause and consider the storage (and network) facilities need to embody most, too.

But consider this: In a virtualized world, servers are files, and files are just data.

So, when we talk about cloud-related scaling, service migration, server fail-over etc., we must also implicitly speak of managing data dynamics, data replication and data mobility. When we talk about automation, self-service provisioning and service elasticity, we’re implicitly talking about dynamic data/storage provisioning and expansion. When speaking of multi-tenancy and tiered SLAs, we’re also speaking of shared storage facilities performing identical functions in lock-step with the compute facilities.

From a broader perspective , begin to consider implications of global availability and hyper-scale. The terabytes of data that embody virtual servers and their data might need to be migrated to (or duplicated in) multiple hemispheres- not a trivial task from an integrity and latency perspective. We can know (or hope) that the physical servers will be there… but it’s the bits that still have to travel.

The next idea these observations triggered was the need to keep compute, network and storage stacks in lock-step when rolling-out cloud services. The answer (not surprisingly) is converged infrastructure... An approach where the desired cloud attributes are assigned to the 3 stacks simultaneously. More about that in a future Blog.

But I'm now encouraging everyone to view storage of bits in a completely different light – one where the functional and operational attributes of storage must be architected to embrace the core attributes of cloud computing. For without the bits, there can be no servers, no data, and no services. More about that in a future post as well :)

Why Utility Computing Failed (But Cloud Computing Didn't)

2011-01-24T17:35:00.000-08:00

Since about 2006 I’ve been involved with data center IT automation. Back then I started with Cassatt, one of the first companies trying to automate infrastructure components in the data center. Rob Gingell, the CTO, had a design principle of “service-level automation”, where the variable monitored and maintained was the service, not the server. That was a revolutionary thought.

The technology behind this was a combination of orchestrating physical and virtual devices, which automatically composed appropriate infrastructure stacks to keep the service SLA within pre-defined bounds. And it absolutely worked! The best market description we had for this technology was “Utility Computing,” and drew from the analogy of electrical utilities: No matter what the draw (load), the supply would always be generated/retired (elasticity) to keep up with it.

But selling the Utility Computing model, and service-level automation technology, was hard, if not impossible. We’d frequently have successful POC’s, and demonstrate the product, but the sales inevitably stalled. The reasons were many and varied, frequently tied to the ‘psychographic’ of the buyers. But overall, we could point to a few frequent problems:

Automation was scary: The word “Automation” frequently scared-off IT administrators. They were accustomed to complete control of their hand-crafted infrastructure, and visibility into every layer. If they couldn’t make and see the change, they didn’t trust that the system actually worked.
Lack of market reference points: Peers in the market hadn’t tried this stuff either – and there was no broad acceptance that utility computing was being adopted
Inflexible Process: The use of ITIL and ITSM procedures were designed to govern manual IT control, and had no way to incorporate automatic approaches to (for example) configuration management.
Organizational fear: There was usually the un-stated fear that the utility computing automation systems would obviate the need for certain jobs, if not entire IT organizations. Plus, the systems spanned multiple IT organizations, and it was never clear which existing organization should be put in charge of the new automation.
Multiple Buyers: Because Utility Computing touched so many IT organizations, the approval process necessarily included many of them. Getting the thumbs-up from a half-dozen scared organizations was hopeless. Even if the CxO mandated utility computing, implementation was inevitably hog-tied.

Enter Virtualization

Somewhere around 2007, OS virtualization began to go mainstream. And its value proposition was simple and uncomplicated: Consolidate applications, reduce hardware sprawl. It was a no-brainer.

But just below the surface, virtualization had an interesting effect on IT managers: It began to make them more comfortable to break the binding of physical control and physical management of servers, transitioning instead to being more at ease with logical control of servers.

As consolidation initiatives penetrated data centers, additional virtualization management tools followed. And with them, more automated functions. And with each new function came IT’s incremental comfort with automating logical data center configurations.

And Then, Commercial Examples

At just about the same time, Amazon Web Services had begun to commercially offer these virtual machines in their EC2 – Elastic Compute Cloud. This could be had for the use of a credit card, and charged-for on an hourly basis. IT end-users now had simple – if sometimes only experimental – access to a truly automated, logical infrastructure. And one where all “hands-on” aspects of configuration were literally masked inside a black box.

Now the industry had its proof-point: There were times when full-up IT automation, without visibility into hardware implementation, worked and was useful.

Use of EC2 (initially) lay outside the control bounds of IT management and IT’s organizational boundaries. Developers and 1-off projects could leverage it without fear of pushback from IT – usually because IT never even knew about its use.

Once IT management acquiesced that EC2 (and similar services) was being used, they finally had reason to look more closely. And the revelations were telling: How was it that the annualized cost basis for a medium-sized server was lower than an in-house implementation could possibly hope to achieve? How come configuration and tear-down was so simple? Finally IT had to look in the mirror at the fact that this thing called cloud computing might be here to stay?

Looking Back

While it’s clear that the concept of cloud computing isn’t new, some important industry changes – more psychological and organizational than technological – had to take place before widespread adoption would happen. And even then, it took some simple commercial implementations to prove the point. Too bad these weren't around a few years earlier during the "utility computing" era.

Watching this unfold, lessons *I* learned – or at least some explanations of this effect I’ve examined have been

Psychology/Attitude shifted: the more broadly OS virtualization was adopted, the more IT’s attitudes became accepting of automation and of logical control.
Technology change was replaced by operational change: The new approach is more a change to the operational approach than a technology upheaval. The way users interacted with the cloud was appealing and nearly viral.
Value was Immediate: The “new” cloud economic evidence was/is usually so compelling that it has forced IT to take a second look. This started with simple consolidation economics, but has expanded well beyond that.
Broad availability accelerated adoption: Even only a few commercially available cloud providers helped provide immediate proof-points that the new model was here to stay. And purchasing this technology was as simple as entering a credit card number

Going forward, I would expect these 4 (perhaps more) “pressure points” will continue to help accelerate the use and adoption of internal clouds, public clouds, etc. In future Blogs I’ll begin to look at how to further mainstream Cloud (and automation) adoption, as it serves to accelerate improvements to business’ bottom line.

Hosting/Cloud Index outperforms NASDAQ Nearly 4x

2010-12-21T14:18:00.000-08:00

Exactly a year ago, in 2009, I started looking into whether the market for Hosting and Cloud Computing could be measured in the stock market - with my first Blog on the topic. I revisited again in January 2010, and later in February 2010.

I created a number of model portfolios based on leading public hosting companies. The list included: Digital Realty Trust; DuPont Fabros; Equinix; Internap; Iomart; Macquarie Telecom; Navisite; Rackspace; Savvis; Switch & Data; Telecity; Terremark. During 2010, this index has gained 45% to the Nasdaq's 17%

I also created a sub-group of those companies with explicit cloud computing offerings. That included Equinix; Navisite; Rackspace; Savvis; Terremark. During 2010, this sub-index has gained 57% to the Nasdaq's 17%

I'm not surprised by either outcome... Assuming the market doesn't lie because it takes into account valuations and expectations, I draw a few personal conclusions:

More enterprises are turning to outsourcing their IT. Whether or not it involves cloud computing, I suspect enterprises find it advantageous to hand-over IT (management, or at least co-location) to businesses where this technology is core. It may also indicate the skyrocketing consumption of computing power by enterprises.
It would appear that hosting firms with Cloud Computing offerings are being valued higher than their counterparts. I haven't looked at cashflows or balance sheets (yet) to determine whether this is actual value, or speculative value.

What's next for 2011? I'm going to guess more of the same, if not an acceleration as more companies move to outsource non-core IT operations. I'll also be watching consolidation of data center operators, as recently evidenced by Rackspace acquiring cloudkick, and Cologix acquiring Navisite. 'Guess I'll have to update my portfolio companies...

IO Virtualization: The “Hypervisor” for Your Infrastructure

2010-12-13T08:40:00.000-08:00

An Explosive Technology, But Don't Treat as a Standalone Product

More than ever in 2010, IO Virtualization (IOV) has been showing-up in products, written about, spoken about. Because I’ve had a few years’ experience with this technology, I wanted to give a very brief explanation of the concept, and focus more on why it will be increasingly important.

In particular, I want to draw an analogy where you should view IOV as a critical enabling feature of future IT Management… but not as a stand-alone product. Why? It's similar in concept to how the hypervisor is an enabler (but usually not used as a stand-alone product) of data center management services.

This blog is related to my 2009 installment on Fabric as an IT Enabler.

What is IOV?

Today's Physical Infrastructure

IO Virtualization is an approach whereby physical IO components such as Network Interface Cards (NICs) Host Bus Adaptors (HBAs) and Keyboard/video/Mouse ports (KVM) are reproduced logically rather than physically. In other words, a physical IO port (Ethernet, Infiniband, PCI, etc.) might logically represent itself to the O/S as different configurations.

Clearly this is convenient because it (a) eliminates multiple costly IO devices that also consume power and installation time. But it’s also convenient because IO – and it’s associated addressing such as IPs, MACs, Worldwide Names, etc. – can be instantly configured with a mouse.

The other consequence of IOV is that a single physical port means a single physical cable. In essence, a server’s logical IO is consolidated down to a single (physical) converged network which carries data, storage and KVM traffic. So this means that no matter how many logical IO devices you configure for a server, there is still only a single cable out the back. So IOV yields the ideal “wire-once” server environment that’s still infinitely re-configurable.

The overall value of IOV becomes clear fast: Fewer physical IO devices to buy, fewer cables to install, zero re-cabling, fewer physical ports to buy, and instantly re-configurable IO.

Differing Implementation Approaches

Infrastructure With
IO Virtualization

In brief, there are a few differing approaches to IO virtualization:

Existing on-board Ethernet with new IO drivers: (e.g. Egenera)
Converged Networking Adapters (e.g. Qlogic, Emulex)
Appliances + high-throughput IO devices (e.g. Xsigo)
Existing physical IO but with address hardware-based mapping/virtualization (e.g. HP VirtualConnect)

Putting IOV in Perspective

You should think of IOV using the following analogy: The way in which the hypervisor abstracts software in the application domain, IOV abstracts IO and networking in the infrastructure domain. (However, to be clear, IOV is not a software layer as-is the hypervisor)

This analogy leads to a few more observations:

Where the hypervisor added software portability in the software domain IOV will do the same for the infrastructure domain. Higher-order services like HA and consolidation were made possibly by the hypervisor. Similarly, HA, DR and migration can be accomplished with IOV. And what’s more, a hypervisor is not required for IOV, so you can use IOV with native applications too.
The hypervisor used to be the focus, but now it’s merely an enabling feature embedded within higher-level IT management products. Those products leverage the hypervisor to perform tasks such as migration, fail-over and consolidation. You should view IOV similarly: it is an enabling feature that will allow for analogous IO consolidation, migration and fail-over.
Where hypervisor implementations and performance used to be hotly-debated, nobody really cares anymore. Today the real *value* is not in the hypervisor, but in the management tools surrounding it. Similarly, IOV should be judged less on how it is implemented, and more on the management tools and automation which manage it.

Forrester analyst Galen Schreck made a similar observation recently:

….Aside from benefits like reducing cabling and switch ports, I think the most interesting aspect of virtualized IO is the ability of a physical server's personality to be moved to any other server in the data center. In addition to the underlying network technology, the thing that makes this possible is integrated management of the server and data center fabric. In most cases, this won't be a stand-alone product that you acquire (though you can build your own solution from InfiniBand and PCI Express products on the market). This capability will most likely be an integrated part of whatever server and network environments you select, but now is the time to begin planning how you'll tie it in with the rest of your system management environment.

IO Virtualization in the IT Management Landscape

How might IO virtualization be used as part of the IT ecosystem in an integrated manner?

In much the same way that the hypervisor has since been embedded in tools like VMware’s vCenter, IOV can (and has been) embedded with higher-level management tools.

Taking an example I’m rather familiar with, Egenera’s PAN Manager Software surrounds IOV technology with facilities such as integrated with converged fabric networking, server boot control and storage connectivity. When used alongside these and other services, IOV enables:

Server High Availability– In the case of hardware failure, a server’s infrastructure state (IO addressing, storage naming, network topology and workload) can be re-instantiated on another bare-metal server. This provides a ‘universal’ style of failover that doesn’t require clustering software. And what’s more, the failed-over server workload could be a native OS, or a VM host. IOV is agnostic to the workload!
Disaster Recovery – expanding on the example above, if an entire domain of servers fails, the entire group of server IO states, networking states, etc. can be recovered onto another domain (assuming shared/replicated storage). This approach to DR is elegant because it fails-over not just workloads but the entire logical server/environment configuration as well.
Scaling-Out – where a series of server profiles can be instantly replicated into an instant cluster. Workloads, NICs, HBAs, networking addressing and storage connections (complete with fabric-based load balancing) can all be cloned… starting with the IO and networking profiles, made possible through IOV.

In future blogs I’ll dive more deeply into how software-based IOV operates as part of the IT management ecosystem, and why it is a popular approach because of its cross-platform compatibility in a heterogeneous data center.

Converged Infrastructure, Part 3

2010-08-12T17:30:00.000-07:00

Converged Infrastructure: What it Is, and What it Isn't

In my two earlier posts, I first took a stab at an overview of converged infrastructure and how it will change IT management, and in the second installment, I looked a bit closer at converged infrastructure's cost advantages. But one thing that I sense I neglected was to define what's meant by converged infrastructure (BTW, Cisco terms it Unified Computing). Even more important, I also feel the need to highlight what converged infrastructure is not. Plus, there are vendor instances where The Emperor Has No Clothing -- e.g. where some marketers have claimed that they suddenly have converged infrastructure where the fact remains that they are vending the same old products.

Why splitting hairs in defining terms? Because true converged infrastructure / unified computing has architectural, operational, and capital cost advantages over traditional IT approaches. (AKA Don't buy the used car just because the paint is nice)

Defining terms - in the public domain
Obviously, it can't hurt to see how the vendors self-describe the offerings... here goes:

Cisco's Definition (via webopedia)

"...simplifies traditional architectures and dramatically reduce the number of devices that must be purchased, cabled, configured, powered, cooled, and secured in the data center. The Cisco Unified Computing System is a next-generation data center platform that unites compute, network, storage access, and virtualization into a cohesive system..."

Egenera's Definition

"A technology where CPU allocation, data I/O, storage I/O, network configurations, and storage connections are all logically defined and configured in software. This approach allows IT operators to rapidly re-purpose CPUs without having to physically reconfigure each of the I/O components and associated network by hand—and without needing a hypervisor."

HP's Definition

"HP Converged Infrastructure is built on a next-generation IT architecture – based on standards – that combines virtualized compute, storage and networks with facilities into a single shared-services environment optimized for any workload."

Defining terms - by using attributes
Empirically, converged infrastructure needs to have two main attributes (to live up to its name): It should reduce the quantity and complexity of physical IT infrastructure, and it should reduce the quantity and complexity of IT operations management tools. So let's be specific:

Ability to reduce quantity and complexity of physical infrastructure:

virtualize I/O, reducing physical I/O components (e.g. eliminate NICs and HBAs)
leverage converged networking, reducing physical cabling and eliminating re-cabling
reduce overall quantity of servers, (e.g. ability to use free pools of servers to re-purpose for scaling, failure, disaster recovery, etc.)

Ability to reduce quantity and complexity of operations/management tools:

be agnostic with respect to the software payload (e.g. O/S independent)
fewer point-products, less paging between tool windows (BTW, this is possible because so much of the infrastructure become virtual and therefore more easily logically manipulated)
reduce/eliminate the silos of visualizing & managing physical vs virtual servers, physical networks vs virtual networks
simplified higher-level services, such as providing fail-over, scaling-out, replication, disaster recovery, etc.

To sum-up so far, if you're shopping for this stuff, you need to
a) Look for the ability to virtualize infrastructure as well as software
b) Look for fewer point products and less windowing
c) Look for more services (e.g. HA, DR) baked-into the product.

Beware.... when the Emperor Has No Clothes...
In closing, I'll also share my pet peeve: When vendors whitewash their products to fit the latest trend. I'll not name-names, but beware of the following stuff labeled "converged infrastructure":

If the vendor says "Heterogeneous Automation" - that's different. For example, it could easily be scripted run-book automation. This doesn't reduce physical complexity in the least.
If the vendor says "Product Bundle, single SKU" - Same as above. "Shrink wrapped" does not equal "converged"
If the vendor says "Pre-Integrated" - This may simplify installation, but does not guarantee physical simplicity nor operational simplicity

Thanks for reading the series so far. I'm pondering a fourth-and-final installment on where this whole virtualization and converged infrastructure thing is taking us - a look at possible future directions.

Postcards from the IT Financial Management Association

2010-06-25T20:19:00.000-07:00

This week marks the third time I have been invited to speak at the ITFMA World of IT Financial Management conference. This is a really amazing/unique conference, created nearly single-handedly by Terry Quinlan, their Executive Director. Quick overview:

The IT Financial Management Association (ITFMA) was established in 1988 and founded the IT Financial Management profession at that time. ITFMA is the only association dedicated to this profession and provides a comprehensive education program on the principles and practices used to financially manage Information Technology (IT) organizations. ITFMA is the national leader in the education of IT financial management professionals and the only recognized provider of certification in the various financial disciplines of IT financial management.

The attendees are largely non-technical, but are comprised of financial managers, controllers, project managers and purchasing managers all in the IT field mainly with F1000 companies.

And what sets this conference apart for me is the fact that 90% of the topics of conversation are non-technical. It's not about the speeds-and-feeds, but rather about the project management, cost accounting, charge-back, managerial and regulatory issues facing IT. It gave me pause that, while technologists focus on keeping the electrons moving, there are also folk who keep the paper and the money moving.

On particularly illustrative conversation I had in mind -- with an IT financial manager from the State of Oregon, who oversees the state's shared/hosted IT infrastructure. They were promised by a large national consulting company that through consolidation of equipment and data centers, the state would save tons of $$ and reduce the managerial headcount as well. As it was described to me, the technical consolidation was largely a success, but the consultant failed to accurately account for the business and managerial staffs associated with the IT. And over time, while the square feet of data center shrank, the overall IT staffing continued to grow. Lest we commit the sin of assuming that all of IT is technologists.

Overall, the ITFMA is a "must-attend" -- especially now that IT is going through such large changes as data center consolidation, virtualization, automation and cloud computing. All of these have non-linear impacts on IT finances, and all can cause disruptive effects on topics like capital forecasting, project management, expense vs investment projections, etc. Not to mention the newer issues caused by cloud computing such as data ownership, security, operations control, etc.

The event is a relative bargain to attend, and Terry always finds classic, historic venues for the conferences.

Converged Infrastructure Part 2.

2010-06-07T16:43:00.000-07:00

Part 2. Converged Infrastructure’s Cost Advantages

In my first installment about converged Infrastructure, I gave an outline of what it is, and how it will change the way in which IT infrastructure is managed.

In this installment, I’ll go a bit deeper and explain the source of capital and operational improvements converged Infrastructure offers – and why it’s such a compelling opportunity to pursue.

But first, the most important distinction to make between converged infrastructure and “the old way of doing business” is that management – as well as the technology – is also converged. Consider how many point-products you currently use for infrastructure management (i.e. other than managing your software stack).

This diagram at right has resonated with customers and analysts alike. It highlights, albeit in a stylized fashion, just how many point-products an average-sized IT department is using. This results in clear impact in

Operational complexity – coordinating tool use, procedures, interdependencies and fault-tracking
Operational cost – the raw expense it costs to acquire and then annually maintain them
Capital cost – if you count all of the separate hardware components they’re trying to manage

That last bullet, the thing about hardware components, is also something to drill down into. Because every physical infrastructure component in the “old” way of doing things has a cost. And I mean I/O components like NICs and HBAs, not to mention switches, load balancers and cables.

What might be possible if you could virtualize all of the physical infrastructure components, and then have a single tool to manipulate them logically?

Well, then you’d be able to throw-out roughly 80% of the physical components (and associated costs) and reduce the operational complexity roughly the same amount.

In the same way that the software domain has been virtualized by the hypervisor, the infrastructure world can be virtualized with I/O virtualization and converged networking. And, once the I/O and network are now virtualized, they can be composed/recomposed on demand. This eliminates a large number of components needed for infrastructure provisioning, scaling, and even failover/clustering (more on this later). And, if you can now logically re-define server and infrastructure profiles, you can also create simplified Disaster recovery tools too.

In all, we can go from roughly a dozen point-products down to just 2-3 (see diagram above). Now: What’s the impact on costs?

On the capital cost side, since I/O is consolidated, it literally means fewer NICs and elimination of most HBAs since they can be virtualized too. Consolidating I/O also implies converged transport, meaning fewer cables (typically only 1 per server, 2 if teamed/redundant). And a converged transport also allows for fewer switches needed on the network. Also remember that with few moving (physical) parts, you also have to purchase few software tools and licenses. See diagram below.

On the operational cost side, there are the benefits of simpler management, less on-the-floor maintenance, and even less power consumption. With fewer physical components and a more virtual infrastructure, entire server configurations can be created more simply, often with only a single management tool. That means creating and assigning NICs, HBAs, ports, addresses and world-wide names. It means creating segregated VLAN networks, creating and assigning data and storage switches. And it means automatically creating and assigning boot LUNs. The server configuration is just what you’re used to – except it’s defined in software. And all from a single unified management console. The result: Buying, integrating and maintaining less software.

Referencing the diagram at right, here's what this looks like on a physical level is fewer components: Costly NIC and HBA cards are virtualized, with their physical transport now consolidated over Ethernet ports, and switches/cables now replaced by a logically-configured switch.

Ever wonder why converged infrastructure is developing such a following? It’s because physical simplicity breeds operational efficiency. And that means much less sustained cost and effort. And an easier time at your job.

Next installment: What Converged Infrastructure is not.

Converged Infrastructure. Part 1

2010-05-06T16:08:00.000-07:00

Since joining Egenera, I've been championing what's now being termed Converged Infrastructure (aka unified computing). It's an exciting and important part of IT management, demonstrated by the fact that all major vendors are offering some form of the technology. But it sometimes takes a while for folks (my analyst friends included) to get their heads around understanding it. So I'm going to take a stab at a multi-part Primer on the topic.

Part 1: What is Converged Infrastructure, and how it will change data center management

Converged Infrastructure and Unified Computing are both terms referring to technology where the complete server profile, including I/O (NICs, HBAs, KVM), networking (VLANs, IP load balancing, etc.), and storage connectivity (LUN mapping, switch control) are all abstracted and defined/configured in software. The result is a pooling of physical servers, network resources and storage resources that can be assigned on-demand.

This approach lets IT operators rapidly repurpose servers – or entire environments – without having to physically reconfigure I/O components by hand—and without the requirement of hypervisors. It massively reduces the quantity and expense of the physical I/O and networking components as well as the time required to configure them. A converged infrastructure approach offers an elegant, simple-to-manage approach to data center infrastructure administration.

From an architectural perspective, this approach may also be referred to as a compute fabric or Processing Area Network. Because the physical CPU state (i.e. naming and configuration of I/O, networking and storage naming) is completely abstracted away, the CPUs become stateless and therefore can be reassigned extremely easily creating a “fabric” of components, analogous to how SANs assign logical storage LUNs. And, through I/O virtualization, both data and storage transports can also be converged, further simplifying the physical network infrastructure down to a single wire.

The result is a “wire-once” set of pooled bare-metal CPUs and network resources that can be assigned on demand, defining their logical configurations and network connections instantly.

BTW, there is another nice resource -- a white paper commissioned by HP (!) executed by Michelle Bailey at IDC. In it she defines what is a converged system:

"The term converged system refers to a new set of enterprise products that package server, storage, and networking architectures together as a single unit and utilize built-in service-oriented management tools for the purpose of driving efficiencies in time to deployment and simplifying ongoing operations. Within a converged system, each of the compute, storage, and network devices are aware of each other and are tuned for higher performance than if constructed in a purely modular architecture. While a converged system may be constructed of modular components that can be swapped in and out as scaling requires, ultimately the entire system is integrated at either the hardware layer or the software layer.

Converged Infrastructure and Software Virtualization

A Converged Infrastructure is different from—but analogous to—hypervisor-based server virtualization. Think of hypervisors as operating “above” the CPU, abstracting software (applications and O/S) from the CPU; think of a Converged Infrastructure as operating “below” the CPU, abstracting network and storage connections. However, note that converged Infrastructure doesn't operate via a software layer the way that a hypervisor does. And converged Infrastructure is possible whether or not server virtualization is present.

Converged Infrastructure and server virtualization can complement each other producing significant cost and operational benefits. For example, consider a physical host failure where the entire machine, network and storage configuration needs to be replicated on a new physical server. Using Converged Infrastructure, IT Ops can quickly replace the physical server using a spare “bare metal” server. A new host can be created on the fly, all the way down to the same NIC, HBA and networking configurations of the original server.

A Converged Infrastructure can re-create a physical server (or virtual host) as well as its networking and storage configuration on any “cold” bare-metal server. And in addition, it can re-create an entire environment of servers using bare-metal infrastructure at a different location as well. Thus it is particularly well-suited to provide both high-availability (HA) as well as Disaster Recovery (DR) in mixed physical/virtual environments – eliminating the need for complex clustering solutions. And in doing so, a single Converged Infrastructure system can replace numerous point-products for physical/virtual server management, network management, I/O management, configuration management, HA and DR.

Converged Infrastructure - Simplifying Management for “The other half” of the Data Center

In the manner that server virtualization has grown to become the dominant data center management approach for software, converged infrastructure is poised to become the dominant management approach for “the other 50%” of the data center – its infrastructure.

However adoption will take place gradually, for a few reasons:

IT can only absorb so much at once. Most often, converged infrastructure is consumed after IT has come up the maturity curve after having cut their teeth on OS virtualization. Once that initiative is under way, IT then begins looking for other sources of cost take-out.... and the data center infrastructure is the logical next step.
Converged infrastructure is still relatively new. While the market considers OS virtualization to be relatively mature, converging infrastructure is less-well understood.

But there is one universal approach that can overcome these hesitations -- money. So, in my next installment, I'll do a deeper dive into the really fantastic economics and cost take-out opportunities of converging infrastructure...

IT Industry Analysts - Falling Into the Bond Rating Agency Trap?

2010-03-16T08:00:00.000-07:00

One of the leading causes of our recent economic melt-down was that "independent" credit rating agencies had a conflict-of-interest with the firms they were supposed to be watching.The very firms tasked with objectively gauging risk were also being paid by the firms they were evaluating...And in the end, the big losers weren't either of them... it was the public.

Well, beware that some of the same could be happening in the IT space.

I'll change the names to protect the innocent -- but let's say that I recently attended a day-long IT analyst event, one where all of the senior analysts trot-out their recent research. And to be honest, most of it was of very high quality.

But in one session which focused on an up-and-coming trend in IT, the analyst only cited the major IT vendors (think: HP, CSCO, IBM, Dell etc.) as the leading innovators and players in the space. It was complete Bunk. Of the four "leading" vendors mentioned, only one of them had any significant innovation in the space. Two others so coated their offerings with "marketecture" that real innovation was tough to discern. And the final crime was that 2-3 smaller vendors I know who actually pioneered the space weren't mentioned at all. And they're the ones providing *real* products with real value today.

Yes, the analyst had a responsibility to his customers (IT end-users) to watch the big players in the industry. And to be sure, the big vendors dominate most market spaces. But the analyst also has a responsibility to truly master his market space and to report-back on the true leaders, innovators, and visionaries. Instead, I believe he unwittingly fell prey to the big vendors that pay much of his firm's bills in order to stay in the analyst's limelight. The failing here is industry-wide, and the IT consumers of the analyst's information are the real losers. Innovation isn't recognized, and therefore value isn't really transferred. And nearly all large industry analysts are guilty of this at some level.

In contrast, another friend of mine is a technology industry analyst with a major international financial institution. When he interviews me on my industry, company and product, he's clear that his reports are not commissioned by vendors, nor even by his bank's clients. There cannot be so much as a hint of conflict-of-interest in his work. Think about it.

---

Major IT Industry analysts have been my friends for years. I've worked for IT vendors small and large, and IT analysts have been (and mostly still are) great sounding boards for new ideas, helped identify market opportunities, and have added lots of marketing value if/when they approve of your product. And IT analysts add value on the IT consumer side too - by identifying trends, pointing-out leading vendors, and recommending best-practices.

But sometimes these folks fundamentally fail at what they're "paid" to do. My advice: Always get a second opinion.

Hosting & Cloud Computing market index: Update

2010-02-01T08:04:00.000-08:00

This month's updates to my original index indicate that the hosting market - and particularly those companies that are in the cloud hosting market - are doing quite well at holding their own against the falling NASDAQ index. Although the NASDAQ component was down ~ $7, my broad hosting component was down less than $1, while my cloud computing component was actually up ~ $7.

One other point of note: Apparently Merriman Curhan Ford also believed that coverage of this space was now warranted:

"We believe Cloud Computing represents a fundamental shift with regards to how IT organizations manage and source data center computing resources. Companies such as Terremark, Rackspace, SAVVIS and NaviSite are at the forefront of this development," said Alex Kurtz, senior vice president and technology equity research analyst of Merriman Curhan Ford. "Our core differentiator in covering this space is leveraging our expertise within our existing coverage of IT systems vendors, who are competing for the same IT budget dollar and impacted by the same macro trends as a Terremark or a Rackspace."

If you think Converged Infrastructure & Fabrics are niche, guess again

2010-01-26T11:36:00.000-08:00

A few weeks ago, I Tweeted about an analyst conversation where it was looking like the market for Fabric Computing / Unified Computing would be growing rapidly in the foreseeable future.

Another analyst friend of mine quickly commented back – sarcastically – that the market was sure to be in the billions of dollars.

I was feeling a little unsure about this market until a few weeks later when I was shown a technology report from Thomas Weisel Partners. Although the market definition for converged infrastructure (also known as Unified Computing) was still forming, TWP felt that sales of Converged Infrastructure solutions could rise as high as $15 billion by the end of 2014. Billion with a “b”? Right-on…

Then there is a report by Gartner Research on fabric-based computing… which estimated that by the end of 2012, roughly 30% of the world’s top 2000 companies would have some form of fabric-based computing architecture. (Under the heading of “fabric” falls Unified Computing as well as Converged Infrastructure).

So, why is the market (for fabric computing, converged infrastructure, unified computing) still considered so new in the market, yet forecast to be so booming in 2-4 years?

First of all, what we’re talking about here are systems like Cisco UCS, Egenera PAN Manager, HP VirtualConnect, IBM Open Fabric Manager, and a few others. At the heart of each system is technology (sometimes HW, sometimes SW, sometimes mixed) that virtualizes I/O and leverages converged networking.

And why are vendors all chasing this approach? For a number of reasons --

It’s incredibly complementary to virtualization: in the same way that the hypervisor changed how SW is abstracted, provisioned, managed and migrated, Converged Infrastructure changes how IO/networking/connectivity is assembled and managed. This gives vendors a valuable set of new offerings, and can tie management of infrastructure to management of VMs – yielding end-to-end abstraction of the entire data center. Roughly as much $ is spent managing infrastructure as it is managing software… to the TAM is huge here.
It changes how availability is delivered: By manipulating IO addressing, networking and connectivity, Converged Infrastructure Management can re-provision failed hardware – either in the form of physical servers, or indeed, entire environments. Thus, Converged Infrastructure has the potential to displace a big chunk of traditional clustering software… (nearly a $ billion, if you follow IDC’s estimates)
It changes how networks are physically wired and managed: Converged Infrastructure uses fewer IO components (either a single LOM or a single CNA), converged network protocols, fewer cables, and generally fewer switches. This yields a lower CapEx investment, and a commensurate lower OpEx to manage. The opportunity to sell alternative approaches to each of these technologies is immense.
Converged Infrastructure is highly complementary to shared storage: the pervasiveness of SAN storage is a major enabler of a more virtual/flexible data center. As physical/virtual servers move, migrate and scale, storage simply follows. An increasing ratio of servers – especially blades – are being shipped with HBAs, indicating that SAN use is on the upswing.

As to evidence that this market is shaping-up, we need only look to the magnitude of investment that Cisco, Egenera, HP, IBM – and even Emulex and Qlogic – are pouring into this market. Methinks we’ll see the hockey-stick shortly.

Hosting & Cloud Computing market index: Update

2010-01-04T07:54:00.000-08:00

Last month I proposed that another way to measure adoption of cloud computing (or, at least, expectations of adoption) was to look at the stock market performance of a bundle of publicly-traded service provider companies.

Since then, I've expanded the list, and carried the range back 24 months.

The total list (the "broad hosting index") consists of: Digital Realty Trust, DuPont Fabros, Equinix, Internap, Iomart, Macquarie Telecom, Navisite, Rackspace, Savvis, Switch & Data, Telecity, and Terremark. I also baselined my "virtual" fund against the NASDAQ index. I created another virtual fund (a subset list of the above) consisting of Equinix, Navisite, Rackspace, Savvis and Terremark - representing service providers with substantial businesses in the Cloud hosting space as well.

Here are some interesting observations of the value change of US$100 invested equally across each index:

Since Jan 2008:

Nasdaq: + ~14%
Broad Hosting index: + ~90%
Cloud Subset: + ~50%

But, I also looked at the change since the market bottomed-out in March of 2009. Since then, the picture is a tad different:

Nasdaq: + ~ 55%
Broad Hosting index: + ~135%
Cloud Subset: +~ 115%

These numbers tell me that the performance (or at least, expectations of performance) in the hosting space far exceeds the broader NASDAQ technology sector. Interestingly, the "cloud index" under-performs the broader index. No explanation here other than the fact that we're dealing here with a statistically low number of companies, and a few "high performers" in the broader index seem to be lifting it above the cloud index.

I'll plan on updating this periodically. Comments, additions, etc. welcome!

Hosting & Cloud Computing: Numbers Don't Lie

2009-12-16T10:59:00.000-08:00

There's lots of chatter in the market today regarding the value of using outside data centers, hosting services and cloud computing. But listening to pundits/analysts trying to objectively predict true value left me hollow.

While I'm not an investment professional, I do know that the stock market doesn't lie.... so instead, I thought I'd look at a bundle of stocks from publicly-traded data center companies in the data center space, and compare against a market benchmark.

I chose companies on publicly-traded markets in both the US as well as in Europe. My criteria were somewhat subjective, but basically the companies had to have a primary business operating data centers. I also excluded Telcos because it is difficult to separate their carrier revenues relative to hosting revenues. So, my initial "virtual fund" consists of 12 companies: Digital Realty Trust; DuPont Fabros; Equinix; Internap; Iomart; Macquarie Telecom; Navisite; Rackspace; Savvis; Switch & Data; Telecity; Terremark.

I also took a 5-company subset of these public companies that had significant offerings in the cloud computing space (Equinix; Navisite; Rackspace; Savvis; Terremark). I labeled this "virtual fund" a cloud-only index.

The chart at right is my best attempt to (a) tabulate historic end-of-month closing price of each stock; (b) calculate month-to-month percentage gains for each; (c) create "virtual funds" where $100 would be invested equally across each vehicle (initially $8.33 in each of the 12 hosting stocks, and initially $20 in each of the 5 cloud-related stocks). The benchmark I used is the Nasdaq index, also assuming an initial $100 investment.

Not surprisingly (for me, anyway) both "indexes" are outperforming the Nasdaq -- perhaps proving the thesis that datacenter operation and application outsourcing is indeed a growth market (or at least a speculative growth market?) as compared to the general technology market. What would be equally useful (but not an analysis I've done) is to chart gross revenues for the Index companies. This would be a telling barometer of actual business.

I'll continue to update this index at the end of each month. Comments, additions and suggestions welcome!

Emergence of Fabric as an IT Management Enabler

2009-12-08T08:47:00.000-08:00

Last week I attended Gartner's annual Data Center Conference in Las Vegas. Four days packed with presentations and networking (of the social kind). Lots of talk about cloud computing, IT operations, virtualization and more.

Surprisingly a number of sessions directly referenced compute Fabrics -- including "The Future of Server Platforms" (Andy Butler), "Blade Servers and Fabrics - Evolution or Revolution" (Jeff Hewitt), and "Integrated Infrastructure Strengths and Challenges" (Paquet, Dawson, Haight, Zaffros). All very substantive analyses of what fabrics _are_... but very little discussion of why they're _important_. In fact, Compute fabrics might just be the next big thing after OS virtualization.

Think of it this way: Fabric Computing is the componentization and abstraction of infrastructure (such as CPU, Memory, Network and Storage). These components can then be logically re-configured as-needed. This is very much analogous to how OS virtualization componentizes and abstracts OS and application software stacks.

However, the focus by most fabric-related vendors thus far is simply on the most fundamental level of fabric computing, which is simply virtualizing I/O and using a converged network. This is the same initial level of sophistication when the industry believed that OS visualization was only about the hypervisor. Rather, we need to take a longer view of fabric computing and think about higher-level value we create by manipulating the infrastructure similar to how we manipulate VMs. A number of heady thinkers supporting the concept of Infrastructure 2.0 are already beginning to crack some of these revolutionary issues.

Enter: Fabric as an Enabler

If we think of "fabric computing" as abstraction and orchestration of IT components, then there is a logical progression of what gets abstracted, and then, what services can be constructed via logically manipulating the pieces:

1. Virtualizing I/O and converging the transport

This is just the first step, not the destination. Virtualizing I/O means no more stateful NICs and HBAs on the server; rather, the I/O presents itself to the OS as any number of configurable devices/ports, and I/O + data flow over a single physical wire. Transport can be Ethernet, FCoE, Infiniband, or others. In this manner, the network connectivity state of the physical server can be simplified and changed nearly instantaneously.

2. Virtual networking

The next step is to define in software the converged network, its switching, and even network devices such as load balancers. The result is a "wire-once" physical network topology, but with an infinitely reconfigurable logical topology. This permits physically flatter networks. Provisioning of the network, VLANs, IP load balancing, etc. can all be simplified and accomplished via software as well.

3. Unified (or Converged) Computing

Now things get interesting: Now that we can manipulate the server's I/O state and its network connections, we can couple that with creating software-based profiles of complete server configurations -- literally defining the server, its I/O, networking, storage connections, and even what software boots on it. (Software being either a virtual host, or a traditional native OS). Having defined the entire server profile in software, we can even define the entire environment's profile.

Defining servers and environments in software allows us to provide (1) High Availability: With a hardware failure, we can simply re-provision a server configuration to another server in seconds -- whether or not that server was running a VM host, or a native OS. (2) Disaster Recovery: we can re-constitute an environment of server profiles, including all of their networking, ports, addresses, etc., even if that environment hosts VMs and native OS's.

4. Unified Management

To achieve the ultimate in an agile IT environment, there's one remaining step: To orchestrate the management of infrastructure with the management of workloads. I think of this as an ideal Infrastructure-as-a-Service -- physical infrastructure that adapts to the needs of workloads, scaling up/out as conditions warrant, and providing workload-agnostic HA and DR. From an IT agility perspective, we would now be able to abstract nearly all components of a modern data center, and logically combine them on-the-fly as business demands require.

Getting back to the Gartner conference, I now realize one very big missing link -- while Gartner has been promoting their Real-Time Infrastructure (RTI) model now for some time, they have yet to link it to the coming revolution that will be enabled by fabric computing. Maybe we'll see some hint of this next year.

Infrastructure Virtualization: The Next Logical Step

2009-11-19T12:49:00.000-08:00

2010 will be an interesting year for virtualization - but not from the perspective you're probably thinking. It will be the year of the virtual infrastructure, not of the virtual machine.
Yes, the O/S virtualization market is maturing as it transforms how servers and applications are managed. The major vendors all offer hypervisors and management to accomplish server consolidation, live migration, HA, lifecycle management, lab management, and more. And they're even offering higher-level tools for DR and cloud computing... Read more on VMBlog.com

Infrastructure 2.0 – A Virtual Analogy

2009-10-27T23:09:00.000-07:00

Is OS virtualization an end in itself? Is it both necessary and sufficient for all things Cloud and IaaS? Is it the panacea IT Operations has been looking for? From where I see it, abstracting the OS is certainly a great start, but it’s actually only 50% of the goal.

To a degree, OS virtualization is the “shiny metal object” de jure in that it’s captivating everyone’s attention. It is of course very valuable, and is causing an important inflection point in datacenter operations and economics. But there is a less-visible, less sexy side to datacenter operations and economics that lies “below” the CPU in the stack...

Read more on the Infrastructure 2.0 Blog

Differing Target Uses for IT Automation Types

2009-10-06T08:30:00.000-07:00

One of the most oft-repeated themes at this year's VMworld was that of "automation." Everybody claimed they had it, but on closer investigation it had any number of poorly-defined meanings.

A specific angle I want to address here is that of infrastructure automation; that is, the dynamic manipulation of physical resources (virtualized or not) such as I/O, networking, load balancing, and storage connections - Sometimes referred to as "Infrastructure 2.0". Why is this important? Although automation of software (such as provisioning & manipulation of VMs/applications) usually captures attention, remember that there is a whole set of physical datacenter infrastructure layers that IT Ops has to deal with as well. When a new server (physical or virtual) is created, much of this infrastructure also has to be provisioned to support it.

There are 2 fundamental approaches to automation I'll compare/contrast: Let's loosely call them "In-Place" Infrastructure Automation, and Virtualized Infrastructure Automation.

Confession: I am a champion of IT automation. The industry has evolved into a morass of technologies and resulting complexity; the way applications (and datacenters) are constructed today is not the way a greenfield thinker would do it. Datacenters are stove-piped, hand-crafted, tightly-controlled and reasonably delicate. Automating how IT operates is the only way out -- hence the excitement over cloud computing, utility infrastructure, and the "everything-as-a-Service" movement. These technology initiatives are clear indications that IT operations desires a way to "escape" having to manage its mess.

At a high-level, automation has major top-level advantages: Lower steady-state OpEx, greater capital efficiency, and greater energy efficiency. And, automation also presents challenges typical of paradigm changes: distrust, organizational upheaval, financial and business changes. The art/science of introducing automation into an existing organization is to reap the benefits, and mitigate the challenges.

As infrastructure automation moves forward, it appears to be bifurcating along two different philosophies. Each is valid, but appropriate for differing types of uses:

"In-place" infrastructure automation: (distinct from run-book automation) Seeks to automate existing physical assets, deriving its value from masking the operational and physical complexity via orchestrating in-place resources. That is, it takes the physical topology (servers, I/O, ports, addressing, cabling, switches, VMs etc.) and orchestrate things to optimize a variable such as an SLA, energy consumption, etc.
Virtualized Infrastructure automation: Seeks to first virtualize the infrastructure (the assets as above) and then automate their creation, configuration and retirement. That is, I/O is virtualized, networking is frequently converged (i.e. a Fabric), and network switches, load balancers, etc. are virtualized as well.

Each of these two approaches has properties with pros and cons with which I'm familiar -- having worked for companies in each space. I'll try to elucidate a few of the "high points" for each:

"In-Place" Infrastructure Automation:
Examples: Cassatt (now part of CA), Scalent

Automates existing assets: Usually, there is no need to acquire new network or server hardware (although not all hardware will be compatible with the automation software). Thus "in-place" assets are generally re-purposed more efficiently than they would be in a manually-controlled scenario. Clearly this is one of the largest value propositions for this approach - automate what you already own.
Masking underlying complexity: A double-edged sword, I suppose, is that while "in-place" automation simplifies operation and streamlines efficiency, the datacenter's underlying complexity is still there - e.g. the same redundant (and sometimes sub-optimal) assets to maintain, same cabling, same multi-layer switching, same physical limitations, etc.
Alters security hierarchy: Since assets such as switches will now be controlled by machine (i.e. the automation SW automatically manipulates addresses and ports) this architecture will necessarily modify the security hierarchy, single-point-of-failure risks, etc. All assets fall under the command of the automation software controller.
Broad, but not complete, flexibility: Because this approach manipulates existing physical assets, certain physical limitations must remain in the datacenter. For example, physical server NICs and HBAs are what they are, and can't be altered. Or, for example, certain network topologies might not be able to be perfectly replicated if physical topologies don't closely match...or, if physical load balancers aren't available, servers/ports won't have access to them. Nonetheless, if properly architected, some of these limitations can be mitigated.
Use with OS virtualization: This approach usually takes control of the VMM as well, e.g. takes control of the VM management software, or directly controls the VMs itself. So, for example, you'd allow the automation manager to manipulate VMs, rather than vSphere.
Installation: Usually more complex to set up/maintain because all assets, versions, and physical topography necessarily need to be discovered and cataloged. But once running, the system will essentially maintain its own CMDB.

Virtualized Infrastructure Automation:
Examples: Cisco UCS, Egenera, Xsigo

Reduction/elimination of IT components: The good news here is that through virtualizing infrastructure, redundant components can be completely eliminated. For example, only a single I/O card with a single cable is needed per server, because they can be virtualized/presented to the CPU as any number of virtual connections and networks. And, a single virtualized switching node can present itself as any number of switches and load balancers for both storage and network data.
Complete flexibility in configuration: By abstracting infrastructure assets, they can be built/retired/repurposed on-demand. e.g. networking, load balancing, etc. can be created at-will with essentially arbitrary topologies.
Consistent/complementary to OS Virtualization models: If you think about it, virtualized infrastructure control is pretty complementary to OS virtualization. While OS virtualization logically defines servers (which can be consolidated, moved, duplicated, etc.), infrastructure virtualization similarly defines the "plumbing" and allows I/O and network consolidation, as well as movement/duplication of physical server properties to other locations.
New networking model: One thing to keep in mind is that with a completely virtualized/converged network, the way the network (and its security) is operationally managed changes. Organizations may have to re-think how (and who) creates and repurposes network assets. (Somewhat similar to coping with "VM Sprawl" in the software virtualization domain)
Use with OS virtualization: This approach is usually 'agnostic' to the software payload of the physical server, and is therefore neutral/indifferent to the VMM in place. Frequently the two can be coordinated, however.
Installation: Usually relatively simple. Few components per server, few cables, especially in a 'green field' deployment. Installation of software/BIOS on physical servers is probably not what you're used to, though.

Ideal use of these two approaches differs too. Obviously, "In-Place" Infrastructure Automation is probably best-suited for an existing set of complex datacenter assets - especially in a Dev/Test environment. As you'd expect , a number of existing lab automation products out there target this market. On the other hand Virtual Infrastructure Automation can certainly be deployed on existing assets, but its real value is for new installations where minimal hardware/cabling/networking can be designed-in from the ground up. Most of these products are designed for production data centers, as well as cloud/utility infrastructures.

My overall sense of the market is that adoption of "in-place" automation will be driven primarily by progressive IT staffs that want a taste of automation and service-level management. Virtualized Infrastructure Automation adoption, on the other hand, will tend to ride the technology wave driven both by networking vendors and OS virtualization vendors.

Stay tuned for additional product analyses in this space...

A real-world cloud user shares his findings

2009-09-29T15:47:00.000-07:00

I subscribe to a number of mailing lists from my alma mater. A few weeks ago, an alum "John" posted a request for recommendations for a cloud computing vendor for his small investment firm. What follows is his email to the group following responses he received.

This is an incredibly illustrative peek inside of the "real world" of cloud computing, and what prospective SMB users are looking for and concerned about. As well as what's "Good Enough". I've not edited anything....

I had many requests to share our findings so I figured I would share with the group. I appreciate all of the input I received. It has been really helpful.

~ John

Having looked into cloud computing solutions for our small investment firm over the past few months, we have learned a lot about the growing movement towards remote data storage and accessibility. Our goal has been to find a cost-effective solution for our IT needs that would make it convenient for employees of our company to access our shared network (documents and emails) all over the globe without much hassle, difficulty, or expense. While the cloud computing landscape is still relatively new, what is already available is exciting. Both Google, Microsoft, and other companies have products available such as Google Apps and Microsoft Office Live, but neither has fully come to the point of being able to handle our business needs. We are currently in the process of setting up a Google Apps trial period, through a consultant, to try out business e-mail and calendar via Google’s Gmail and Google Calendar. We will do this test while retaining our current Microsoft Exchange server.

There have been many issues to consider as we have been speaking with various consultants and researching all of the available alternatives. First, since we are an SEC-registered investment adviser with lots of confidential and sensitive information on our hands, issues regarding the security of our electronic files – both in terms of disaster recovery as well the integrity of the company with whom we are entrusting to house our data – are paramount. This also ties in with the issue of record retention, which is equally important to us. In terms of data storage and backup – our current system is not ideal. We need to retain copies of all e-mails and files for at least seven years, if not more, and this information needs to be secure and easily accessible. There seem to be some progress in this area (Google Postini and Amazon S3, for example), but as of yet, there is not yet one system that can do all of these things in the way we’d require.

Second, since we currently are not pleased with our current remote network access - we would like an easy and inexpensive way to access email and our network drive from any computer with Internet access. We have discovered that while web-based, unlimited e-mail and calendar storage are currently available from multiple providers, a solution for mass file storage that would essentially replicate our shared network drive and allow large files for multiple software applications to be stored/backed up in the cloud does not yet exist at an attractive price. In particular, a system where we could modify docs in the cloud without having to download and upload/re-save the file each time it needs to be edited.

One interesting product we discovered during our search is called Dropbox. You download Dropbox to one computer, save any type of file you would like to a “Drop Box drive”, and it syncs up automatically with the Web. Then, when you are at home or traveling, you can access those docs through a web browser... or you can download Drop Box onto another computer anywhere and you can edit the docs directly in Drop Box. The only glitch is that Dropbox does not yet have file storage capacity for a company with over 200GB of data to store and seems to be geared more for individual users. Word on the street is that Google will be coming out with a new product soon that has similar features to Dropbox, but on a much larger scale that would be useful for businesses.

In terms of cost and ease, Google Apps seems to be the best solution for us right now (it comes out around $50/user/year), at least for the e-mail and archiving component. Microsoft’s upcoming 2010 Web Apps platform seems appealing as well, particularly because we might be able to edit complex Excel documents directly in the cloud from anywhere.

Bottom line, what we have learned is that this rapidly-developing option for IT is not yet 100% ready to cover all the bases our business needs, but it will probably get there sometime in the next year or two. For the time being, we are going to see how the e-mail works and go from there.

Alternative Recommendation for DCeP "Service Productivity"

2009-09-22T14:38:00.000-07:00

Back in February of this year, The Green Grid published a paper listing proposed Proxy measures for data center productivity, specifically Data Center Energy Productivity (DCeP).

This paper followed a much earlier output from the group in 2007 - which helped define the now much-used PUE and DCiE metrics which I wrote about back then. Those metrics were (and are) nice if what you care about are "basic" efficiencies of a data center -- simply how much power is getting to your servers relative to all of the other power being consumed by infrastructure systems (e.g. lighting, power distribution, cooling, etc.). But the shortcomings are they don't quantify the "useful output" of a datacenter vs. power input. So, for example, you could have a fantastic PUE... but with a datacenter full of idle servers.

Again, enter The Green Grid to take analysis to the next level. The excellent paper published in February details 8 "proxy" approaches (i.e. not necessarily metrics) that could be used by data center operators to begin to baseline efficiencies based on "useful output". The Green Grid also set up a survey where they have been soliciting feedback from users regarding the appropriateness, usefulness, etc. of these proxies.

Why 8 approaches? Because not everyone agrees on what "useful work output" of a datacenter really is. Should it be Bits-per-kWh (proxy #4)? Weighted CPU utilization (proxies #5 & #6)? Compute units delivered per second (proxy #7)? Each has its pros and cons. Fortunately, the Green Grid recognized that nothing's perfect. Says the paper: "...The goal is to find a proxy that will substitute for a difficult measurement, but that still gives a good-enough indication of useful work completed."

In addition, the Data Center Knowledge blog pointed out:

The new goal is to develop a simple indicator, or proxy, rather than a full metric. The Green Grid compares the proxy to EPA mileage ratings for new cars, which provide useful data on energy efficiency, with the caveat that “your mileage may vary.” The proposals “do not explicitly address all data center devices and thus fall short of a complete overall measure of data center productivity,” the group says.

To this end, the issue was also recently dealt with extremely eloquently in Steve Chambers' ViewYonder perspective on datacenter efficiency - and has the right idea: Why not base efficiency on the service provided (as opposed to CPUs themselves, or some abstract mathematical element). This approach is very similar to what I proposed a year ago February, Measuring "useful work" of a Datacenter"

In short, the proposal is to compare the data center Service's SLAs with the power the overall datacenter consumes.

Why use the "SLA" (Service Level Agreement)? Two reasons. (1) The SLA is already part of the vernacular that datacenter operators already use. It's easily understood, and frequently well-documented. (2) The SLA encapsulates many "behind-the-scenes" factors that contribute to energy consumption. Take this example: Not all 1,000 seat email services are created equal. One may be within a Tier-I data center with a relatively low response rate requirement and allowing users only 500MB of storage per mailbox. Another enterprise with the same email application may be operating in a Tier-III datacenter environment with a rigorously-controlled response rate, a full disaster-recovery requirement, and 2GB of storage per mailbox. These two SLA examples are quite different and will therefore consume different power. But wouldn't you now rather compare apples-to-apples to see if your particular instantiation of these 1,000 mailboxes was more efficient to another enterprise with the same SLA?

How would such a proxy/measurement be accomplished? The approach is somewhat analogous to the Green Grid's proxy #1 ("Self-assessment reporting"), coupled with peer-reporting/comparison of data as is done with the DOE's DC-Pro tool.

Thus, data centers would
1) quantify the number of Services and SLAs for each,
2) measure overall power consumed,
3) upload these numbers to a public (but anonymized) database.
After a while, there would be statistically-significant comparisons to be made -- say a "best practice" energy efficiency range for a given Tier-III email application with 2GB storage and disaster-recovery option.

I'm open to other suggestions of how to pragmatically apply application SLAs vs Watts to gauge overall datacenter energy efficiency - again, my earlier proposal of this is here. But it seems that the SLA encapsulates all of the "output" related service metrics, while being agnostic to the actual implementation. Seems elegant, if you ask me.

An Ideal Datacenter-in-a-Box, Part II

2009-09-14T08:45:00.000-07:00

Last week I posted a Blog outlining Dell & Egenera's latest Datacenter-in-a-Box offering. More than one person took note of how I compared its simplicity in contrast to other offerings in the same space, but failed to detail the specifics of Egenera's PAN Manager software and how it mapped to 13 common IT Service Management functions.

The 13 different functions are mapped onto the data center "stack" at right. They span management of both physical and virtual software, servers, I/O, networking, etc. -- as well as higher-level functions such as High-Availability and Disaster Recovery.

The Dell PAN offering unifies 12 of the 13 functions, and provides them from within a single console is called PAN Manger. (The 13th function if provided via the Dell Management Console.) This single-console infrastructure management software consists of the base PAN Builder software, as well as two optional modules, PAN Server Portability, and PAN Portability.

So, using the diagram from last week, the functionality maps as follows:

PAN Builder:

VM server management
Physical server management
Software (P & V) provisioning
I/O virtualization & management
IP load balancing
Network virtualization & management
Storage connection management
Infrastructure provisioning
Device (e.g. switch & load balancing) failover

PAN Server Portability:

Physical N+1 failover (HA)
Virtual host N+1 failover (HA)

PAN Portability:

Disaster recovery (DR) for entire mixed P & V environments

I hope this helps detail not only the Egenera product, but also illustrates what's possible when the industry combines server management with virtual I/O and virtual networking & switching. It's the perfect complement to O/S virtualization, and massively simplifies traditional IT Operations Management.

An Ideal Datacenter-in-a-Box

2009-09-10T07:00:00.000-07:00

Today's announcement marks a modest but meaningful step in Egenera's relationship with Dell, and in overall Simplification of IT .

Essentially the punchline is this: We've taken the most commonly-purchased hardware configuration and management tools used by mission-critical IT Ops, and integrated them into a single product with a single GUI that you can install and use in ~ 1 day. That's essentially the idea behind the "Datacenter-in-a-Box:"

Most common configuration: Blades + Networking + SAN Storage

Most useful tools to manage VMs + physical servers + network + I/O + SW provisioning + workload automation + high availability

That's what Egenera's done with Dell. It's a "unified computing" environment (to borrow a term) - but has integrated with it all of the most popular higher-level management functions too. That's to say it includes I/O virtualization, a converged network fabric (including virtual switches and load balancing - based on std. ethernet), and then includes tools for software provisioning, VM management, and high-availability to "universally" manage both physical and virtual workloads simultaneously. Pretty cool - and highly simple to use.

Don't believe all this stuff can be so simple? Here's evidence & illustrations why this move will help drive data center management toward greater simplification:

(1) Check out how easy it is to provision a complete compute environment with N+1 failover in 6 steps
(2) Compare the level of complexity reduction compared to some similar prod ucts
(3) The Dell PAN Datatcenter-in-a-Box (DCIB), together with the Dell Management Console, provides a massively simplified management landscape as compared with alternative solutions. To wit:

The set of "traditional" products you'd need to buy/integrate.
ALL of these functions are already integrated within the Dell PAN DCIB:

Then, the roughly-equivalent solution you'd compose with HP:

And finally, the roughly-equivalent solution you'd compose with Cisco and their partners:

I'd also be remiss without pointing out that this product SKU configuration is available directly from our friends at Dell - and was born directly from customer requests for such a building-block. Folks who've already purchased this technology based on the Dell PAN System include

Federal users who may replicate an entire mission-critical environment across dozens of aviation-related locations
Financial-services users who wanted a consolidated approach to ensuring high-availability across dozens of blades w/different workloads
Commercial customers wanting a flexible environment on which to run the company's SAP
A Federal hosted services provider wanting five-9's of availability plus being able to re-configure systems/capacity a la an "internal" cloud
Overseas users acting as an internal IT service provider seeking 'universal' HA and DR for all workloads

Plus thousands more worldwide locations where you can find the same PAN Manager software.

If you don't believe Dell hardware is ready for the Data Center, then think again.

Where the Server Industry Went Amiss

2009-09-07T09:15:00.000-07:00

I've been doing an analysis regarding how "complexity" has evolved in the datacenter. Fundamentally, just why is it so hard to configure & provision new (physical) servers? Why is clustering inherently so complex? Why do we have data networks, storage networks and management networks (all distinct, I might add). How come we have all of these layered management systems?

OS virtualization has massively simplified complexity at the software level by abstracting-away the machine-level CPU commands, and has even contributed to simplifying networking between virtual machines. But we're still left with physical complexity at the physical I/O, networking and control levels - the other physical piece-parts (KVM ports, NICs, HBAs, etc.).

Ultimately, all of this complexity gradually resulted from incremental server hardware evolution… the motherboards to be exact. Way back when the computer industry was just getting started, motherboards harbored a simple CPU and remedial I/O (e.g. an audio jack to a cassette tape for storage...). But as processors got more sophisticated and datacenter environments grew, CPUs were integrated with more complex I/O (e.g. Network Interface Cards) as well as with storage connectivity (e.g. Host Bus Adaptors). Plus, there was usually a local disk, of course.

This meant that the server retained static data, specifically things like I/O addressing and storage connectivity naming, not to mention data on the local disk -- resulting in the server having a static “state". Usually the local network had state too – ensuring that the IP and MAC address of the motherboard were attached to switches and LANs in a particular way. Add to this the fact that with critical applications, all of these components (plus naming/addressing) were frequently duplicated for redundancy.

This meant that if you had to replace (or clone) a physical server, say because of a failure, you had to re-configure all of these addresses, names, storage connections and networks – and sometimes in duplicate. This resulted in lots of things to administer to, and lots of room for error. And frankly, this is where fundamental “data center complexity” probably arose from.

In response to dealing with failures and complexity, vendors developed special-purpose clustering and failover software – necessarily closely-coupled to specific software and hardware – to provide the re-assignment of state to the new hardware and networking. This software often required hand-crafted integration and frequent testing to ensure that all of the addressing, I/O, and connectivity operations worked properly. And many of these special-purpose systems are what are in use today.

Similarly, there are equally complicated software packages for scale-out and grid computing, that perform similar operations – not for the purpose of failure correction, but for “cloning” hardware to scale-out systems for parallel computing, databases, etc. But these systems are equally complex and usually application-specific, again having to deal with replicating Stateful computing resources.

So the industry, in an effort to add “smarts” and sophistication to the server – to enable it to fail-over or to scale – has instead created complexity and inflexibility for itself. Had the industry instead defined I/O, networks and addressing logically, then the way we assign/allocate servers would have been forever simplified and streamlined.

Fortunately, some technologies are being applied to somewhat revert/simplify:

I/O virtualization appliances which logically consolidate all I/O into one reconfigurable physical card (e.g. Xsigo)
Infrastructure virtualization software which logically defines all I/O, networking and switching so that any CPU-I/O-Network config. can be defined to take the place of any other CPU-I/O-Network config. (e.g. Egenera, Cisco UCS and to some degree HP's VirtualConnect)
CPU pooling hardware/software which replace traditional I/O to make multiple physical servers act as large multi-core CPUs (e.g. 3leaf)

Unfortunately, the industry's own momentum sustains the level of complexity - most players continue to develop software products to handle/abstract the increasing complication. Nor is in the interest of the board designers & silicon manufacturers to _reduce_ the number of chips & cards associated with servers. So, we may not see a significant architectural change in stateful processing units - until the industry gradually acquiesces that there is an alternative to all of this madness.

Products for Cloud Ops vs. Traditional Ops

2009-08-24T20:57:00.001-07:00

Most in IT agree that cloud computing - while not a technology - does impact how technology is used within IT, and also implies a change in how IT operations will manage infrastructure. So it comes (to me) as no surprise that a number of traditional "point-product" IT Service Management (ITSM) products might be obviated by the new cloud computing operational paradigm - while others may morph in how they're used.

I touched on this topic a little over a year ago when looking at ITIL, ITSM and the Cloud as well as assessing how capacity and consolidation planning will likely change.

When I read Gartner's Hype Cycle for IT Operations Manag ement 2009, I was struck by how many technology categories may really need to change (or may simply become unnecessary) in a cloud model. (BTW, I should point out that cloud computing itself is at the peak of Gartner's overall Technology Hype Cycle for 2009.)

I see roughly five dimensions for how ITSM product use might shift within a cloud model:

Overall increase in use, due to IT Operational needs created by the automation and dynamics within a cloud infrastructure. With more dynamic/unpredictable resource requirements, some ITSM tools may become more valuable than ever. For example, take Billing/Chargeback. Clearly any provider of a public (or internal) cloud will need this to provide the pay-as-you-go economic model, particularly as individual resource needs shift over time. Same clearly goes for tools such as Dynamic Workload Brokering, etc.
Overall decrease/obviation of need, due to the the automation/virtualization within a cloud infrastructure. As automation begins to manage resources within the cloud, certain closely-monitored and managed services may simply be obviated. Take for example application-specific Capacity Planning; no longer will this matter to the degree it used to - now that we have "elastic" cloud capacity. Similarly, things like event correlation _might_ no longer be needed -- at least by the end-user -- because automation shields them from need to know about infrastructure-related issues.
Shift in use to the cloud operator - that is, the IT Service Provider will tend to use certain ITSM tools more. For example, Asset Management, Global Capacity Management and QoS tools necessarily mean nothing to the end-user now, but may still be critically-important to the SP.
Shift in use to the cloud end-user - that is, the cloud end-user may tend to use certain ITSM tools more - chiefly because they do not directly 'own' or manage infrastructure anymore - just executable images. i.e. end-users using IaaS clouds will need to maintain their Application and Service Portfolio tools to manage uploadable images etc. Conversely, End-Users may no longer care about Configuration Auditing tools - since that would not be managed by the cloud provider.
Transition from being app-specific to environment-specific - that is, a shift from tools being used to monitor/control/manage a limited-scope application stacks, to being used to do the same across a large shared infrastructure. As above, Capacity and Consolidation Planning tools are no longer of interest to the end-user on a single-application-scale. But to the cloud operator, knowing "global" capacity and utilization is critical.

In retrospect, I can probably concoct exceptions to almost every example above. So keep in mind the examples are illustrative only!

The diagram below is also mostly conceptual; I am not an ITSM professional. But while it may be a bit of a 'hack', I'm hoping it provides food-for-thought regarding how certain tools may evolve, and where certain tools may be useful in new/different ways. I've selected a number of ITSM tools from Gartner's IT Ops Management Hype Cycle report to populate it with.

What's your "data center complexity factor"?

2009-08-17T09:11:00.000-07:00

After speaking with IT users, analysts and vendors, I've tried to draw-up a "map" of some of the most common data center management tools directly related to operations, and how they layer across typical infrastructures. So far, I see about 13 tools in use. (I'm not counting higher-level administrative tools e.g. compliance mgmt, accounting, problem management etc. - but watch this space for a future Blog)

I'm curious to see (a) if I have it right, and (b) how does *your* infrastructure management 'stack' up? Can you do better than 13? Worse?

Quantifying Data Center Simplification

2009-07-14T08:27:00.000-07:00

Ever read that marketing fluff that says "blah blah simplifies your data center"? Ever wonder what that means and whether there is any quantifiable measure?

Infrastructure & management simplification is more than simply reducing ports, cable counts, and more than simply virtualizing/consolidating. (In fact, if done improperly, each of these approaches ultimately adds management complexity)

To me, true simplification isn't 'masking' complexity with more management and abstraction layers. Rather, it's about taking a systems-style approach to re-thinking the componentry and interaction of items across both software and infrastructure. For example, independently-managed components (and management products) can consist of

Server/CPU status, workload repurposing
Server console/KVMs
Physical app management
Virtual app management
Physical HA/DR
Virtual HA/DR
Storage connectivity
I/O management
Networking & switch management
software systems Management

It's not just about reducing cables & ports!

Three observations I've recently made have driven this concept home to me.

1. The arising of true Infrastructure management: systems like PAN Manager which essentially manage all of the above bullets together as a true "system" (see my earlier post on 6 simple steps to take to managing IT infrastructure) Nowhere else will you see as many as 6-7 complex IT management functions reduced to a single console.

2. An average case-study of a PAN Manager user. For example take a major Online Grocer dealing with a storefront website (environment was BEA WebLogic, Oracle9i RAC, CRM, business intelligence, etc.) for delivery admin and payment processing. Complexity consisted of traditional systems management, and then the addition of clustering and the 1:1 duplication of server, network and SW tools.

With a systems-style management approach, ultimately, servers, ports, cables, NICs, HBAs, disks, recovery systems -- and most of all, admin time and OpEx -- fell dramatically with a PAN-managed, systems approach to simplification. That took componentry from ~1,500 "moving parts" down to under 200. To me, "elegant engineering" equates to simplification.

3. Major equipment vendors are offering similar infrastructure management products to those from Egenera. But the "systems" issue still persists, even if some of them have solved for the networking, I/O & switching parts. So, after a pretty detailed analysis I did, it's still obvious that multiple "point products" are still needed to operate these products. Those products still represent a non-systems approach, to me at least.

What would you rather manage? A bunch of point products that mask complexity, or a true system that re-thinks how data center infrastructure is run? I'm thinking the PAN Manager-run system :)

Why (and How) Low-Cost Servers Will Dominate

2009-07-07T09:36:00.001-07:00

Or, why high-end servers will be obviated by software...

I begin this blog with a true story. 2 weeks ago I was training a new Account Executive about the virtues of server automation, I/O virtualization, converged networking, etc. To his credit (or mine?), within the first hour he blurts out "then if a customer uses this stuff, they should be able get five-9's of availability from run-of-the-mill hardware, right?"

And that's the point: The age of high-end, super-redundant, high-reliability servers is slowly coming to an end. They're being replaced by volume servers and intelligent networks that "self heal". (Head nod to IBM for coining the term, but never following through)

I pointed-out to my trainee that folks like Amazon and other mega-providers don't fill their data centers with high-end HP, Sun or IBM gear anymore. Rather, companies like Google use scads of super-inexpensive servers. And if/when the hardware craps-out, it is software that automatically swaps-in a new resource.

It's like the transformation back in the late 1700's with Eli Whitney and mass production, where mechanical systems - including the now-famous Colt revolver - were made to simple, standard, interchangeable specifications. Similarly today, rather than hand-crafting every system stack (software, VMs, networking - everything) we're moving to a world where simple, standard HW and configurations can do the 99% of the job. And it's the software management that simply works-around failed components. This trend in IT was noticed back in 2006 (probably earlier) as the "Cheap Revolution" pointed out in a now-famous Forbes Article.

So what's the punch-line here? I believe that the vendors who'll "win" will be those who are effective at producing low-cost, volume servers with standard networking... But most of all, the winners will be effective at wrapping their wares in a system that is designed for automatic interchangeability.

To wit: in a recent IDC study, while all server sales segments were forecast to fall in 2010, the Volume segment was the only one expected to experience a gain.

I believe that the days for buying super-high-end, high-reliability servers are numbered (for all but some of the most critical telco-grade apps). The Dells of the world have an opportunity; and the Suns, HPs and IBMs will need to re-think the future of their "high-end".

Were I a vendor with a strong volume server play, I would continue to push on hardware pricing, and begin to emphasize a hardware/network self-management strategy.

One other (slightly random) analogy. Shai Agassi's Better Place company *isn't* a car company. It's really a software and networking company. With the right network and infrastructure, the vehicles are efficient and always have a 'filling' station to keep running. Similarly, IT is transforming from it "being about the hardware" to being about how the hardware is networked and managed. Think about it.

HPQ & CSCO: Analysis of New Blade Environments

2009-06-29T21:47:00.000-07:00

I've been spending some significant time analyzing new entries into the blade computing market, and poking around in the corners where the trade rags and analysts have failed to investigate. And, as the line goes, "some of the answers may surprise you."

The two big recent entrants/announcements were Cisco's Unified Computing System (made this past March) and then HP's BladeSystem Matrix (made in June). Both are implicitly or explicitly taking aim at each other as they chase the enterprise data center market. They're also both teaming with virtualization providers, as well as hoping for success in cloud computing. Each has a differing technology approach to blade repurposing, and each differs in the type (and source) of management control software. But how revolutionary and simplifying are they?

HP's BladeSystem Matrix architecture is based on VirtualConnect infrastructure, and bundled with a suite of mostly existing HP software (Insight Dynamics - VSE, Orchestration, Recovery, Virtual Connect Enterprise Manager) which itself consists of about 21 individual products. Cautioned Paul Venezia in his Computerworld review:

“The setup and initial configuration of the Matrix product is not for the faint of heart. You must know your way around all the products quite well and be able to provide an adequate framework for the Matrix layer to function.”

From a network performance perspective, Matrix includes 2x10Gb ‘fabric’ connections, 16x8Gb SAN uplinks, and 16x10Gb Ethernet uplinks. The only major things missing from their "Starter Kit" suite they offer are the addition of VMware - not cheap if you choose to purchase it - as well as the addition of a blade (or two) to serve as controllers of the system.

From Cisco, the UCS System is based on a series of server enclosures interconnected via a converged network fabric (which does a somewhat analogous job of repurposing blades as does HP's VirtualConnect). The UCS Manager software bundled with the system provides core functionality (see diagram, right). Note, that at the bottom of their "stack", Cisco turns to partners such as BMC for "higher level" value such as high-availability and VMware for virtualization management. As sophisticated as it is, in contrast to HP, this software is essentially "1.0" and full integration w/third-party software is probably a bit more nascent than with HP.

As you would expect, the system has pretty fast networking; Cisco’s system includes 2x10Gb fabric interconnects, 8x4Gb SAN uplink ports, and 8x10Gb Ethernet uplink ports. (But as the system scales to 100's of blades, you can't get true 10Gb fabric point-to-point.)

But really, how simple?

What I continue to find surprising is how both vendors boast about simplicity. True, both have made huge strides in the hardware world to allow for blade repurposing, I/O, address, and storage naming portability, etc. However, in the software domain, each still relies on multiple individual products to accomplish tasks such as SW provisioning, HA/availability, VM management, load balancing, etc. So there's still that nasty need to integrate multiple products and to work across multiple GUIs.

A little comparison chart (at right) shows what an IT shop might have to do to accomplish a list of typical functions. Clearly there are still many 3rd-party products to buy, and many GUIs and controls to learn.

Still, these systems are - believe it or not - a major step forward in IT management. As technology progresses, I would assume both vendors will attempt to more closely integrate (and/or acquire?) technologies and products to form more seamless management products for their gear.

RTI Fabrics... not just a networking play

2009-06-11T13:55:00.000-07:00

Pete Manca, Egenera's CTO, posted an excellent Blog Explaining RTI Architectures, (a term coined by Gartner some time ago) and does a nice job of taking a pretty objective approach to 3 types:

"A converged fabric architecture takes a single type of fabric (e.g. Ethernet) and converges various protocols on it in a shared fashion. For example, Cisco’s UCS converges IP and Fiber Channel (FC) packets on the same Ethernet fabric. Egenera’s fabric does the same thing on both Ethernet fabrics (with our Dell PAN System solution) and on an ATM fabric (on our BladeFrame solution)...

"Dynamic Fabrics are not converged, but rather separate fabrics that can be have their configuration modified dynamically. This is the approach that HP uses. Rather than utilize a converged fabric, HP has separate fabrics for FC and Ethernet. These fabrics can be dynamically re-configured to account for server fail-over and migration. HP’s VirtualConnect and Flex10 products are separate switches for Fiber Channel and Ethernet traffic, respectively."

"The 3rd type of fabric is a Managed Fabric. In this architecture there is no convergence at all. Rather, the vendor programs the Ethernet and Fiber Channel switches to allow servers to migrate. This is a bit like the Dynamic Fabric above, however, these typically are not captive switches and there is no convergence whatsoever."

I'll take some liberty here, and emphasize a pretty important point:

Converged /managed fabrics aren't attractive just because they simplify networking. It's because they are a perfectly complementary technology to managing server repurposing as well. That's for *both* physical servers and virtual hosts.

It's no wonder why IBM (with their Open Fabric Manager), HP (with their Matrix bundle), Cisco (with UCS) and Dell/Egenera (with the Dell PAN System) are all pushing in this area.

Why? Because once you have control over networking, I/O and storage connectivity, you've greatly simplified the problem of repurposing any given CPU. That means scaling-out is easier, failing-over is easier, and even recovering entire environmentns is easier. You don't have to worry about re-creating IPs, MACs, WWNs etc., because it's taken care of.

So, if you can combine Fabric control with SLA management and then with server (physical and virtual) provisioning, you've got an elegant, flexible compute environment.

CA's Acquisition of Cassatt - Hindsight & Foresight

2009-06-02T15:02:00.000-07:00

Today I read the press release and Gordon Haff's analysis that Computer Associates has acquired Cassatt -- a former employer of mine.

CA probably appreciates that they have a real gem. But like all things Tech, most cool products are not "build it and they will come". However, I can say that Bill Coleman (Cassatt's CEO) and Rob Gingell (Cassatt's CTO and former Sun Fellow) really have a break-the-glass vision. Now lets see if the new lease-on-life for the vision (and product) will take shape.

Vision vs. speedbumps
Cassatt's vision - led by Rob - is still out in front of the current IT trends... but not by too far. As much as 3 years ago, the company was anticipating "virtualization sprawl", the need for automating VMs, the expectation that IT environments will have both physical and virtual machines, and the fact that "you shouldn't care what machines your software runs on, so long as you meet your SLA". That last bit, BTW, presaged all of our current 'hype' about cloud computing!

The instantiation of these observations was a product that put almost all of the datacenter on "autopilot" -- Servers, VMs, switches, load-balancers, even server power controllers and power strips. The controller was then managed/triggered by user-definable thresholds, which could build/re-build/scale/virtualilze servers on-the-fly, and do just about anything needed to ensure SLAs were beging met. And it worked, all-the-time making most efficient use of IT resources and power. As Rob would say "we don't tell you what just happened - like so many management products. We actually take action and tell you what we did." Does it sound like Amazon's recent CloudWatch, Auto-Scaling and Elastic Load Balancing announcement? Yep.

Finally, the coup the company had -- and what the industry still has to appreciate -- is that the product takes a "services-centric" view of the data center. Rather than focusing on *servers* the GUI focuses on *services*. This scales more easily, and gives the user a more intuitive sense of what they really care about -- service availability... not granular stuff like physical servers or how they're connected.

Unfortunately for Cassatt, there is an inherent tension between how ISVs develop products, and how IT customers buy them. ISVs are always looking for the next leap-frog.... while IT customers almost always play the conservative card by purchasing incremental/non-disruptive technology.

So the available market of real leap-frog CIOs is still small... but growing. I would expect the first-movers to adopt this won't be traditional enterprises -- but rather Service Providers, Hosting Providers and perhaps even IT Disaster Recovery operations looking to get into the IaaS and/or Cloud Computing space.

What it could mean to CA
So why would CA buy Cassatt? Unfortunately, it's not to acquire Cassatt's customers. It is much more likely to acquire technology and talent.

Given that CA seems to be a tier-2 player in the data center management space, Cassatt would help them legitimize their strategy, and pull-together a cloud-computing play that other competitors of CA's are already moving down the road on. Cassatt's product ought to also complement CA's "Lean IT" marketing initiative

The other good news is that CA has a number of Infrastructure Management products that ought to complement Cassatt technology. There is Spectrum (infrastructure monitoring), Workload Automation (more of a RBA soulution that might get partially displaced by Cassatt), Services Catalog, and Wily's APM suite. BTW, there's a pretty decent WP available on CA's website on Automating Virtualization.

Per Donald Ferguson, CA’s Chief Architect: “Cassatt invented an elegant and innovative architecture and algorithms for data center performance optimization. Incorporating Cassatt’s analysis and optimization capabilities into CA’s world-class business-driven automation solution will enable cloud-style computing to reliably drive efficiencies in both on-premises, private data centers and off-premises, utility data centers. We believe the result will be a uniquely comprehensive infrastructure management approach, spanning monitoring, analysis, planning, optimization and execution.”

I could see CA now beginning to target large enterprises as well as xSPs to begin to leverage Cassatt technology, as their engineering teams begin integrating bridges to other CA suite products. It will also take CA's sales and support organizations some time to digest all of this, and then bring it to market through their channels.

But Cassatt will bring to them a bunch of sharp technical and marketing minds. Stay tuned. CA's a new player now.

New AWS enable "Real" Elastic Clouds

2009-05-18T07:11:00.000-07:00

Yesterday Amazon announced a new set of services for their EC2 "elastic compute cloud" and these perhaps represent the real "holy grail" for cloud computing. While not new concepts, they illustrate how "real" cloud computing elasticity works, and challenge a few other virtualization & automation providers to do the same.

Amazon CloudWatch: A for-fee ($0.015 per AWS instance monitored) service that:
provides monitoring for AWS cloud resources... It provides customers with visibility into resource utilization, operational performance, and overall demand patterns—including metrics such as CPU utilization, disk reads and writes, and network traffic...

Amazon Auto Scaling: a free service that:
automatically scales your Amazon EC2 capacity up or down according to conditions you define. With Auto Scaling, you can ensure that the number of Amazon EC2 instances you’re using scales up seamlessly during demand spikes to maintain performance, and scales down automatically during demand lulls to minimize costs. Auto Scaling is particularly well suited for applications that experience hourly, daily, or weekly variability in usage. Auto Scaling is enabled by Amazon CloudWatch...
Amazon Elastic Load Balancing: A for-fee ($0.025/hour/balancer + $0.008/GB transferred) which
automatically distributes incoming application traffic across multiple Amazon EC2 instances. It enables you to achieve even greater fault tolerance in your applications, seamlessly providing the amount of load balancing capacity needed in response to incoming application traffic. Elastic Load Balancing detects unhealthy instances within a pool and automatically reroutes traffic to healthy instances until the unhealthy instances have been restored...

To-date, users of Amazon EC2 have had to do these sorts of things manually, if at all. Now Amazon is building these services into AWS (as well as into Amazon's pricing and business model).

Not entirely new concept:
Note that what Amazon is doing is not entirely new. For example, if you're considering building these sorts of capabilities into your own "internal cloud" infrastructure, there are a few products that provide similar solutions. e.g. in 2008, Cassatt (RIP?) announced its own capacity-on-demand technology, which created/retired entirely new instances of a service based on user-definable demand/performance criteria. (I should add that Auto Scaling & CloudWatch operate similarly -- you can define a number of performance and SLA parameters to trigger grow/shrink scaling commands).

Similarly, Egenera's PAN Manager approach dynamically load-balances networking traffic between newly-created instances of an App. And, products such as 3Tera also enable users to define components (such as load balancers) in software. All of this adds-up to truly "adaptive infrastructure" that responds to loads, failures, SLAs etc. automatically.

The challenge to others:
So, if Amazon can instantiate these services in the "public" cloud, then I would expect others -- notably providers such as VMware, Citrix, MSFT etc. -- to provide similar technologies for folks building their own infrastructure.

For example, in VMware's "vCloud" I would expect to see services (some day) that provide similar monitoring, auto-scaling, and load balancing features. If virtualization providers are to take "internal cloud comuting" seriously, these are automation-related services they'll be required to provide.

Kudos to Amazon:
And finally, Amazon has done two saavy things in one move - (a) they've once again shown the world what a true "cloud" computing infrastructure ought to do, and (b) provided another nice value (and revenue stream!) to complement their per-instace EC2 fees. Remember: The easier they make it to scale your EC2 instances, the more $ they make...

Looking forward to how the industry will respond....

Profiling questions nobody's asking re: cloud applications

2009-05-12T09:33:00.000-07:00

I find it odd that so much is being written about defining cloud terminology, cloud operation, and cloud construction... But so little attention is being paid to identifying & profiling which applications are best-suited to actually run in an external "cloud."

I'd like to see a comprehensive list (or decision-tree?) of what ideal application properties pre-dispose apps for being well-suited to run in a cloud. And, for that matter, what qualities might potentially disqualify apps from running in a cloud as well. (BTW, a great Blog by John Willis, Top 10 reasons for not using a cloud, was another initiator of my thought process)

Customers of mine are attracted to the value prop of a cloud (or a "utility" infrastructure)... but need guidance regarding what apps (or components) should be candidates for these environments. And recent conversations with prominent analysts haven't helped me... yet.

I'm also surprised that consulting/service companies aren't all over this issue... offering advice, analysis and infrastructure "profiling" for enterprises considering using clouds. Or am I missing something?

So, with no further ado, I've begun to jot down thoughts toward a comprehensive list of application properties/qualities where we could "rank" an application for its appropriateness to be "outsourced" to a cloud. I've chosen to annotate each factor with a "Y" if the app is appropriate for an external cloud, a "N" if not, and a "M" if maybe.

Dynamics/Cyclicality

Y Apps with highly dynamic (hourly/daily/etc.) or unpredictable in compute demand.
A cloud's elasticity ensures that only enough capacity is allotted at any given time, and release when not needed. This avoids having to buy excess capital for "peak" periods.
M Apps where compute demand is "flat" and/or constant.
Not clear to me if it makes sense to outsource an app if it's demand is "steady-state" - maybe just keep it in-house and purring along?
M Apps where demand is highly counter-cyclical with other applications
In other words, if an application runs out-of-phase with other apps in-house (say, backup apps that run in the middle of the night when other apps are quiescent) then it might make sense to keep in-house... they make better use of existing capital, assuming that capital can be re-purposed.

Size / Temporality

Y Apps that are very "big" in terms of compute need, but which "go away" after a period of time
Such as classic "batch jobs", whether they're daily trade reconciliations, data mining projects, scientific/computational, etc.
N Apps that are "small" and constant in nature
Apps such as Edge apps like print services, monitoring services etc.
Y Apps that are part of test environments
Apps - and even entire environments - which are being tested prior to roll-out in production. This approach eliminates costly/redundant staging environments
M Apps for dev and test uses
For example, where environment and regression testing is concerned, and environments are built and torn-down frequently. However, certain environments are inherently bound to hardware and/or tested for performance, and these may need to remain "in-house" on specific hardware (see below)

Inherent application functionality

N Apps that are inherently "internal"
Such as internal back-up software, "edge" applications like printer servers, etc.
N Apps that are inherently bound to hardware
Such as physical instances of software for specific (e.g. high-performance) hardware, or physical instances remaining so for scale-out reasons. Also, physical instances on ultra-high-reliability hardware (e.g. carrier-grade servers) .

Responsiveness/Performance

N Apps needing high-performance, and/or time-bound requirements
such as exchange trading algorithms, where response and deleay (even down to microseconds) is critical, and needs to be tightly monitored, controlled and optimized

Security / Auditability / Regulatory / Legal
NB: Also see an excellent Blog by James Urquhart on regulatory issues in this space.

M Apps where data must be maintained within (or outside of) specific county borders
Data within certain borders may be subject to search/access by the government (e.g. Patriot Act). Data may be required to be maintained within sovereign borders... or it may be preferred that the data explicitly be maintained outside of sovereign borders to avoid such searches.
M Apps requiring tight compliance/auditability trails
Ordinarily, I'd give this a "N", but some tools are coming out that help ensure compliance for apps that exist in the cloud. Apparently HIPAA regulations essentially prohibit use of clouds right now. Stay tuned here.
N Apps manipulating government data, e.g., where laws require direct data oversight
Many government databases are required to be maintained within government facilities behind government firewalls.
N Apps where software licensing prohibits cloud
e.g. some software licensing may be tied to specific CPUs; some licensing may not permit virtualization (as is found in most clouds); certain licensing may not permit use outside of specific physical domains.

Curious to hear whether these are some of the properties being taken into account... and what other pointers people have. And most of all, curious to hear whether (and if so, which) service providers & consultancies are currently using guidelines such as these.

In their own words: Valley CTOs' Blogs & Tweets

2009-05-06T16:24:00.000-07:00

I noticed that I subscribe to feeds from a number of CTO-like folks, so I thought I'd publish a few of my favorites:

Sun's CTO, Greg Papadopoulos - Blog

I love listening to Greg - definitely a visionary, definitely well-connected. He's the guy I referenced back in 2007 when he observed that The World Only Needs 5 Computers. Looking like this concept is really coming true.

Cisco's CTO, Padmasree Warrior - Blog - Twitter

Padmasree's blog started getting major traction when Cisco "leaked" their UCS system. As Cisco's visionary, she has unbelievable insight into where IT is going, with a great sense of "humanity" and realism thrown in.

Amazon's CTO, Werner Vogels - Blog - Twitter

What can I say? As the chief spokesperson for Amazon Web Services, he does a great job championing All Things Cloud. I had the opportunity to see him a few months ago at the Cloud Computing Expo in NY, and his vision is compelling.

Intel CTO, Justin Rattner - Blog

While only an occasional Blogger, he definitely reflects Intel's position on a number of issues and technologies.

BMC Software CTO, Tom Bishop - Blog

Tom has a great way of posing thought questions and industry issues from an Enterprise Management perspective. He doesn't seem to get lots of comments on his Blog, though. Wonder why. :|

Novell's CTO, Jeff Jaffe - Blog

A frequent blogger (and, I might add, so is Novell's CMO). I like this blog b/c I think it gives a real sense of where his (and Novell's) mind is at.

HP's Cloud Services CTO, Russ Daniels - Blog

Russ is HP's visionary in the cloud/SaaS space. Very cool guy. Unfortunately his Blog reads more like Twitter updates. But he's posted a few interesting videos to the web recently.

These are the guys I track... and I'm surprised that more tech companies either don't have official CTOs, or don't tend to condone Blogging/Tweeting.

Which others do you follow?

Infrastructure Orchestration in use within SPs & Hosting providers

2009-05-05T10:14:00.000-07:00

For the past months I've held that new technologies are OK... but the litmus test is whether they're actually used and valuable in the real world.

One of those new technologies in the Enterprise Data Center space is what I call Infrastructure Orchestration (others term it fabric computing or unified computing). HP, IBM and now even Cisco have solutions in the space, but I believe only Egenera has been doing it the longest, and has the broadest installed base of enterprises in the real-world using it and expanding footprint.

With the explosive growth of virtualization, this segment of technology is hotter than ever. Why? In the way virtualization abstracts & configures the software world (O/S, applications, etc.), Infrastructure Orchestration abstracts and defines/configures the infrastructure world (I/O, NIC cards, HBA cards, storage connectivity, LANs, switches, etc.). So, not only can you define a virtual server instantly, you can define a *physical* server (maybe a virtual host, or a physical machine) down to I/O, NICs, Storage and Network. By doing this, you can reconstruct an entire data center -- giving you a unified approach to HA and/or DR. Cool.

I've been pointing out applications for this technology in Healthcare as well as in the Financial sector, and I thought it would also be useful to illustrate value in the Service Provider / Hosting market.

For this segment, the Infrastructure Orchestration approach is essentially used to build Infrastructure-as-a-Service, or IaaS. In the past it's been called "utility computing" but in the era of cloud computing, this seems to be the term in use.

Savvis

In 2004, Savvis set a goal to become the industry’s first totally virtualized utility computing data center, integrating virtualized servers, storage, networks, and security into an end-to-end solution. Today, the service provider houses over 1,425 virtual servers running on 70 industrystandard Egenera servers, 370 terabytes of storage and 1,250 virtualized firewalls.

As a complement to its managed hosting and collocation business, the company has built huge, scalable service platforms that can be leveraged by multiple clients with full security. This utility approach enables them to charge customers for resources more closely tailored to their actual needs. Each year, more revenues and profits are generated from utility hosting contracts with business and government customers ranging from start-up entrepreneurs to the largest enterprises in the world, enabling Savvis to compete and win against traditional hosting providers and outsourcers.

Albridge Solutions:

Albridge Solutions migrated from UNIX servers to industry-standard servers running Linux and Egenera-based Infrastructure Orchestration. Initially, they considered building a virtualized environment by combining virtualization and management point-products. They discovered, however, that resulting complexity would be overwhelming. Servers from the industry’s largest vendors were also ruled out since their legacy architectures made virtualization and resource sharing impossible. Today, using industry-standard servers and Egenera's software, Albridge can run any application on any server at any time based on demand... regardless of whether those applications are virtual, or native.

Panasonic Electric Works Information Systems Co., Ltd.

Panasonic chose Egenera products to consolidate servers and reduce floor-space. Along with enabling server consolidation, the software is delivering superior high availability (HA) and disaster recovery (HA). Applications running in the data center include an order-processing service for the manufacturing industry, a content delivery system and Electronic Data Interchange (EDI). Based on results, Panasonic has designated Egenera software as its standard infrastructure virtualization management software for mission-critical processing.

Lessons from Glassblowing for High-Tech marketing

2009-05-04T09:55:00.000-07:00

This past weekend I spent 6 hours learning the very basics of glassblowing. It's been on my "bucket list" for quite some time, and when a good friend suggested we try it, I jumped on the opportunity. But what I didn't realize was that there were lessons I got in the studio that are metaphors for my "day job" too.

BTW, the lesson was given at San Francisco's Public Glass studios - a fabulous resource near one of San Francisco's largest artists communities. Cool galleries but even cooler artisans at work. Glassblowing has been around since ancient Egypt, I think. And in many ways it hasn't changed very much. The tools are still very simple, and the raw materials are still the same. And so begin my observations:

It's way harder than it looks: Nothing beats experience and experimentation, and no amount of watching beats actual doing. You notice this the second you take your first blob of glass from the furnace and simply try to keep it symmetrical and from falling to the floor. You have to develop an intuitive feel for temperature, malleability, and a muscle-memory for working with the material.

All the business books in the world only get you so far. You need to get your hands dirty. And frankly, nothing beats learning from a really good failure. Once you see a product cancelled (or for that matter, a company die) you finally gain a real appreciation for what to do, not just what not to do. Ya' can't get that from a book.

Heat is your friend (but be careful): You find that you only have a minute or two of work time before you need to re-heat a piece. But be wary - you're operating at temperatures above 2,000F (and as high as 2,500F sometimes) which means even standing in front of an oven - even 6 feet away - is something you can only tolerate for a few seconds. Going back-and-forth doesn't give you lots of time to cool-of and pat your forehead.

Hype, profile and momentum is what you strive for. But it can be fleeting. When you're "hot" you've got clout, but it dies-down quickly. Drumming-up conversations - or even controversy - in the social network realm is great. It keeps you in play. Just don't overdo it or you'll be toast. :)

From basic materials & tools can arise massively different implementations: Yes, there are a few different types of glass (some w/higher melting points, clarity, etc.) and a few different tools (basic steel pincers, scissors, wooden shaping cups, and yes - even wet newspaper to help shape). But that's it. Then the creativity begins. How you manipulate the glass viscosity, temperature gradients & selective cooling, layers of glass, color etc. is infinitely variable. The sky's the limit.

And even in tech, the basic marketing principles (the four P's, segmentation, etc.) haven't changed in a long time. But using them in clever/innovative ways is the trick. Making sure you stand-out in the crowd, above the noise-level, is still more of an art than a science. Play with the combinations, repeat them, think about re-combining in new ways. Be creative and brainstorm with others.

Keep things moving! Hot glass is essentially fluid, much like really think molasses. The second you stop spinning the glass, it'll start sagging. Plus, the really good artisans spin smoothly and transition back/forth smoothly too. Never stop.

And, never stop experimenting; never let-off on the accelerator with PR, AR, or marketing programs; keep the "buzz" going, keep the plates spinning. If you're complacent or don't have an agenda for next month or quarter, start now.

Cool slowly: Too much thermal change is bad. Big pieces experience internal thermal stresses, and will shatter if cooled too fast. Most pieces have to be cooled in a controlled manner over 24 hours.

And too much change is bad to any organization. Plan making your go-to-market changes slowly, over many quarters. I've seen organizations that want to radically change marketing themes and messages every quarter (or month!). Give the market at least 2-3 quarters to absorb new positioning/messaging. Unless you're consumer goods, customer buying cycles can be long... the changes will confuse them if too often.

Work as a team: Big pieces need at least two -- and sometimes as many as four -- people to help. Different pieces need to be prepped, warmed, blown, held, etc. It's choreographed in advance. Everyone knows their job. Running into someone holding a piece of glass at 2,000F can really spoil your day.

Ditto. In business, as in art, working as a team is critical -- Always good to have frequent status meetings, and over-communicate your actions/intentions. Just because a project looks like you can do it alone doesn't always mean you should. Socialize your efforts even as you're doing the project -- and even ask for input even if you may not need it -- getting early ownership from others means buy-in from them too.

Pointers for IT Business Continuity & Pandemics

2009-04-29T11:19:00.000-07:00

One topic close to my heart is the concept "working anywhere." Not just telecommuting, but literally working within a distributed workforce.

While the chatter in the market is all about connectivity, I'll point out that a laptop and a VPN aren't sufficient -- there are managerial and organizational issues to take notice of.

Now, some jobs don't need you to "be in the office" and in fact, sometimes being in the office is plain wrong. For example, if you're a salesperson, you're probably in an office 10% of your time, if that much -- so why do companies pay for that permanent office space? The other example of when going to the office is wrong is during a potential Pandemic... As we've seen this week in Mexico, the government has already taken steps to reduce congregating individuals by closing-down the schools. This could easily escalate into voluntary (or mandatory) steps whereby adults need to stay home from the office.

But how would this affect business operations? Most business continuity / continuity of operations (BC/COOP) focus on keeping the IT portion of the business running... but what if there aren't any people to run them?

Lots of content is now being written by industry analysts; In a recent Forrester Research Blog, Stephanie Balaouras writes "Swine Flu? What It Means For IT Professionals"; they cite a recent joint Forrester and Disaster Recovery Journal survey regarding BC strategies.

Gartner Research has also put out a press release "Swine Flu Is a Reason to Act, Not Panic," and Networkworld has a pretty insightful "10 Tips for swine flu planning."

But let me share one aspect that these high-level recommendations don't touch-on as they should: Managing the organization when it's distributed and/or remote.

Getting all of the VPN and laptop technology into the hands of workers is one thing; keeping them functioning as a cohesive group with management oversight and direction is another. If people end up working remotely for 2+ weeks, this becomes a critical issue.

I've spent some serious professional time examining this, particularly with one of the leaders in the field, Sun Microsystems... where a significant fraction of employees are remote/virtual workers. Sun has a mature approach to implementing this, called Sun OpenWork. Check out a very excellent whitepaper on "Managing in a virtual organization"

You should also check out TeleTrips, who specialize in consulting for distributed/virtual organizations, telecommuting programs, scenario planning, and online tools. Whether your enterprise wants to invoke a mobile work program, or just have a BC/COOP plan in place should a disaster occur, these folks help examine work and management practices, facility locations and technology infrastructure, and then put in place the appropriate programs & training.

It's all very different from a laptop and a modem. And should a natural disaster strike (esp. here in sunny CA) people might not be able to, or might be told not to, come to the office. Be prepared.

Who's using Infrastructure Orchestration in Healthcare?

2009-04-27T10:22:00.001-07:00

Last week I wrote a bit about how Infrastructure Orchestration is being applied in the Financial industry.

It's the approach that HP (w/Matrix Orchestration Environment plugins), IBM (Open Fabric Environment) and Cisco (Unified Computing System or UCS) have entered into recently, and where firms like Egenera (with PAN Manager software and the Dell PAN System) have been selling for some time. The beauty of infrastructure orchestration is that it abstracts-away all of the "plumbing" of the Server such as I/O, networking and storage connectivity -- which makes it an absolutely *ideal* complement to virtualization. So this technical approach is gaining greater installed-base every day.

Taking the conversation from "cool stuff here" to "it's in real use in the real world", I thought I'd give some examples of users of Egenera's PAN Manager software in the Healthcare industry:

Cambridge Health Alliance:

Cambridge Health Alliance chose PAN Manager to manage the infrastructure for their ambulatory product suite, in a strategic initiative to automate its ambulatory-care environment. Over five years, the Alliance expects to save $2 million, including $1 million in initial capital costs. Equally significant, the software is reducing system administration requirements—enabling Alliance IT professionals to focus on activities that add real value to the user community. I like this part: “If we’d purchased any other platform to support the... applications, we would have had to hire more system administrators. The simplicity and automation of the Egenera system take the place of two people.” Chief Information Officer, Cambridge Health Alliance

Emory Heathcare:

Emory Healthcare moved mission-critical applications from proprietary UNIX® and mainframe platforms to a virtualized infrastructure based on industry-standard servers and PAN Manager software. Benefits included improvements in system administration, TCO, utilization, and provisioning time. A system administrator can configure and allocate a virtual -- or physical -- server in minutes. In addition, multiple operating systems and OS images can be run on a single server, enabling IT to create test systems on servers otherwise used for failover. PAN Manager security facilities also help ensure that patient records are preserved in accordance with government mandates.

Metavante Healthcare:

With annual growth of 35 to 40 percent, Metavante Healthcare Payment Solutions needed to upgrade data center infrastructure just to keep pace. The company chose PAN Manager as an Infrastructure Orchestration approach for its unique virtualization capabilities. PAN Manager had quantifiable improvements in availability, performance, flexibility, management, and cost savings. Again, I live for this: “I was looking for a system designed from the ground up to overcome the limitations of standard servers… We didn’t see an architecture like Egenera’s from anyone else.” Vice President and CTO, Metavante Healthcare Payment Solutions

SCBIT (Schanghai Center for Bioinformation Technology):

SCBIT chose PAN Manager infrastructure orchestration for its ability to simplify consolidation, virtualization and management; to reduce application time-to-market for applications; and to lower data center costs. PAN Manager provided the flexible allocation and repurposing that SCBIT required: The agency can run any of its 10+ applications on any server at any time. PAN Manager also enables SCBIT to make every application highly available at virtually no cost and provides a unique N+1 approach to disaster recovery. Their evalutation also showed performance advantages of running Oracle 10g on servers with PAN Manager, versus traditional systems. Cool.

Overall, it's nice to see that this technology has major footing already. My belief is that as Virtualization becomes more pervasive, and as data center technologies become more complex, we'll see this infrastructure orchestration approach begin to displace "traditional" approaches for IT Management. It's simple, elegant, runs on standard x86 boxes, and provides broader reliability than typical clustering solutions.

Keep your ears open. for more.

Who's using Infrastructure Orchestration in Finance?

2009-04-20T10:28:00.000-07:00

With all the talk lately of Egenera's unified infrastructure orchestration I thought it would be useful to shed some light on who's *really* using this technology, and how.

Indeed, there are a number of similar technologies out there (Cisco with its UCS, and HP with its own offerings). But sharing actual users/uses, I believe, takes the conversation from "hey, cool stuff here" to "gee, it really is useful in the real world."

The beauty of infrastructure orchestration is that it abstracts-away all of the "plumbing" of the Server such as I/O, networking and storage connectivity -- which makes it an absolutely *ideal* complement to virtualization.

The results of this approach is that physical servers can be repurposed easily, regardless of whether they're running physical or virtual software. And fast repurposing means you can deliver instant High Availability (HA), entire environment disaster recovery (DR), and near-instant scaling (capacity-on-demand). In fact, even without VMs, some consolidation is possible by being able to use the same box for different uses at different times.

Within the past few years, a number of financial-services firms have adopted this approach using Egenera's PAN Manager software:

Commerzbank NA:

After the 9/11 terrorist attacks, Commerzbank NA launched an initiative to enhance its disaster-recovery strategy. The goal was to deploy a pool of virtualized servers that could run UNIX or Windows on demand, to complement the flexibility already achieved with virtualization on the storage and network sides.

Today their DR site not only ensures business continuity, it plays an active role in daily computing requirements—notably improving utilization. Plus, Commerzbank NA has consolidated 140 legacy servers into 48, slashed server-configuration time from two days to one hour, and reduced floor-space requirements by 60%. Niice..

Standard Chartered PLC:

IT architects at Standard Chartered PLC decided to centralize data center ops for core retail-banking application, available to customers at 1,200 locations worldwide. They selected Egenera's selected PAN Manager to do the job. As a result, Standard Chartered has cut total cost of ownership in half compared to their previous proprietary solution and can now bring a new country online in nine days rather than 45 days... as estimated for its legacy architecture.

Farm Bureau - Western Computer Services:

Western Computer Services, Inc. (WCS), a wholly owned subsidiary of Farm Bureau Financial Services, deployed Egenera's PAN Manager software the foundation for its new service-based architecture, serving multiple Farm Bureau Financial Services companies throughout the United States. The Farm Bureau chose Egenera to power the delivery and management of Web-based insurance services to thousands of personal and commercial insurance customers.

Investment Banks....
Egenera has a bunch of Investment Bank customers - in uses for HA, DR and repurposing, for applications from order management, order routing, and other client services. But these guys won't let me use their names :(

Postcards from SDForum's Developer's Conference

2009-04-17T17:00:00.000-07:00

SDForum hosted another wide-ranging, star-studded conference Friday. The topic was posted as "Shaping the new age of application development" but - from the start - had overtones of SaaS, cloud computing, and new business models.

The day opened with James Staten of Forrester Research - giving his usual riveting, insightful view on clouds, cloud adoption and directions. I think he also was able to add some sobriety to the hype, pointing out that Infrastructure-as-a-Service was the technology most likely approach to mature first. He expects to see "cloud hype" to die-out around 2010.

James was followed by a panel hosted by Chris Yeh, focusing on new software business models. Pretty lively discussion about how packaged software is moving to subscription, how SaaS is re-making how software is consumed, and how simple financial models will help shape how products/services are packaged and priced. Frankly, the internet is changing all types of business models... e.g. a question from the audience focused on traditional advertising business models, where "classifieds" were advertising in a newspaper where "news" was the content. But with the advent of Craigs List, "classifieds" *is* the content. Hmm.

Now a shift: a panel on Mobile development - Panelists from Nokia, Sun's Java division, iPhone developers, etc. One really interesting insight: think of the phone application market as "verticals" and "horizontals". While there are a few million phones that are "vertical", i.e. Blackberry, iPhone, etc., there are a few *billion* phones globally that are more basic, but where there is a more massive market to write to. The horizontal market for mobile dwarfs that market that we think of as the "advanced" iPhone market!

After lunch, we got a really cool and riveting presentation from Clara Shih from Salesforce.com on "Understanding the Facebook Era" Her premise: Facebook is CRM for individuals. Ergo, there is a natural connection between Facebook and SalesForce.com... (Faceconnector) Also, there is the need for "online identity" and Facebook is making a play for owning that 'credentials' space (Facebook Connect). But with online identity, there is the potential for massive online data mining and demographic research.

My own panel was moderated by Chris Preimsberger of eWeek - focusing on cloud infrastructure. Hamid Pirahesh of IBM led-off with a really elegant 4-quadrant perspective of Traditional vs. Cloud models, and internal vs. external models. Lots of good conversation about why "cloud" infrastructure differs from traditional infrastructure.

Finally - the VC panel... Accel, Draper Fisher Jurvetson, Hummer Winblad, and Benchmark. First question off-the-block: "Where are you seeing the growth?" Answers: (a) small-budget items that can be purchased in bite-sized pieces, (b) IT infrastructure equipment that saves money, (c) data analytics, cloud and cleantech. Mostly pretty bullish on the startup market, apparently lots of series-A and B happening now. And lots of really good advice for wannabe startups on how the VC process works.

Private Clouds are Real and In Use

2009-04-13T09:30:00.000-07:00

Many moons back in July '08 I wrote whether an "Internal Cloud" was an oxymoron, as well as a piece on how to build such internal architectures.

Turns out this is really happening, and I'm thrilled. In "Why 'Private Cloud' Computing Is Real -- And Worth Considering," InformationWeek reported that this isn't your father's Oldsmobile... it's real and it's different:

...The Department of Veterans Affairs has deployed a small internal cloud. It wanted an early-warning system that could analyze data from its 100-plus clinics and hospitals and spot outbreaks of infectious diseases, and it had to do so on a tight budget. The project, dubbed the Health Associated Infection and Influenza Surveillance System, was built on six standard blade servers with converged network and storage I/O. The CPUs can be managed individually or as a virtualized whole, with workloads shifted and capacity summoned as necessary.

The system runs Egenera's cloud management software, PAN Manager, which manages I/O, networking, and storage for the servers as a logical set. It can execute several applications, while always having enough horsepower to do its main job. The system's Dell blades and storage can be virtualized as a pooled resource in such a way that processing power can be devoted quickly to the VA's cloud, its highest-priority task. In many ways, the VA's new system anticipated Cisco's recently introduced "unified computing" platform, a virtualized, multiblade server chassis with converged I/O that Cisco touts as just the thing for cloud computing.

I've spoken with the VA's CIO; they're running both physical and virtual applications (i.e. physical Oracle and virtualized services) but want to be able to scale transparently, with extremely high levels of availability, etc. Plus, like many IT Operations, they have trouble anticipating end-user demand -- so they require instant "elasticity" in the system. Hence, the "cloud" model for IT operations fits the bill. While they don't call it that ("cloud" seems to be the sexy term right now) it's how they're operating.

My suspicion/hope is that as this simplified model for internal IT matures, more IT operations folks will see the light (through the clouds).

More later.

Egenera creates physical & virtual servers transparently

2009-04-07T07:33:00.000-07:00

Egenera just announced that it's embedding virtual machine provisioning within its PAN Manager software for free.

PAN Manager is the "secret sauce" Egenera has been using to manage its own hardware for years, and recently OEM'd to Dell as part of their Dell PAN System. The software has always had the ability to provision physical and/or virtual servers with mission-critical levels of availability, including networking, I/O and storage configurations.

Now Egenera has bundled the virtual server provisioning option in at no charge into the core management product. The technology is based on the latest Citrix (Xen) technology.

That means that Egenera's own hardware -- as well as Dell Blades -- not only operates with a "unified computing" architecture (as some would call it!), but that operators have the option to create physical or virtual servers on-the-fly, as conditions require. And, regardless of the type of server (or software workload) the servers are protected with HA and DR, even across remote locations.

For Dell, this gives them a play in the mission-critical computing market, with a way to either embed VMs within their bladed environments, or to support other virtualized environments like VMware or Microsoft -- all with similar five-9's of availability.

Using the system is super simple and elegant. You'll never need to re-cable a server again, and never have to worry about I/O, network configuration or DR configuration. IMHO it will be the infrastructure model the industry will migrate to.

What IT Can Learn From the Bean Counters

2009-04-06T09:15:00.000-07:00

I spent a day last week at the IT Financial Management Association's (ITFMA) meeting in Atlanta. Among other things, I gave an introductory presentation on the basics of IT energy consumption & efficiency metrics.

The ITFMA meeting is just what you'd expect: lots of Sr-level folks, mostly finance, assigned to supporting IT organizations, data centers, etc. Some were technical, some not. While there was a good turn-out (and there is each year) I was surprised that it wasn't larger. This seems to be *the* place where the business-side of IT is examined -- and is one of the best-kept secrets in the industry. I certainly hope the conference continues to grow.

The topics - and questions - were telling of the state of IT... from a business perspective. Follows are a few of my takeaways from some of the talks/presentations:

Data centers & energy efficiency: It was surprising to me how many companies admit that their data centers are 20+ years old -- tacitly acknowledging the inefficiencies they're coping with. Just as many were planning migrations or upgrades, even in this economy.
Chargebacks: A surprising number of firms are using chargebacks, or planning to. Overall the industry average across IT is somewhere in the 20% range. But the people here seemed more likely to have already adopted it. And, with the advent of shared IT (i.e. virtualization) the trend toward chargebacks, and chargeback tools, seems to be growing.
Software use & metering, + Cost allocation: As above, this continues to be a hot topic -- if you're doing chargebacks, you need to monitor how (and by whom) software is being used. So, lots of interesting topics on licensing, metering, etc., and lots of new vendors pushing solutions.
SW asset management: Another subset of above. But now, with virtual servers everywhere, outsourcing parts of applications, etc., the topic of asset management seemed to be in a number of placces
IT Process & Financial management: Dovetailing ITIL management processes with financial management. This isn't trivial for large organizations. Running IT involves capital improvements, problem management, change management, new service delivery, service retirement, etc. Finance has to work alongside all of these processes while the plane continues to fly.
SaaS and Cloud: As you'd expect, there were a number of talks regarding just what Software-as-a Service is, as well as what cloud computing promises. Not so much from a technical perspective (remember the audience) but from a business, compliance and security perspective. Ultimately -- from a business perspective, the Cloud model represents a new business outsourcing model which is very convenient for certain classes of applications.

FYI, the ITFMA's next upcoming meeting is in Charleston, SC on July 13-17, 2009 and will have four concurrently held conferences:

• IT Chargeback & Activity Based Cost Management Conference
• IT Expense & Asset Management Conference
• ITIL Financial Management Conference
• IT Telecommunication Financial Management Conference

eBay Is Really a Huge PaaS Provider

2009-04-01T10:06:00.001-07:00

While Amazon, Salesforce.com, Google & others seem to be getting the "cloud" attention lately, it seems that eBay is doing a brilliant platform move - very much akin to what Salesforce.com is already doing.

eBay has announced its Selling Manager Applications Beta, which is really a number of hosted services, each complete with its own API.

eBay Selling Manager Applications allows developers to help professional sellers manage their online businesses within their trusted My eBay experience. Professional sellers are like any small business, and quality selling management tools embedded into the Selling Manager experience will make them more efficient and profitable. Sellers will be able to find and easily subscribe to applications through an applications directory.

Some of the APIs include
- Shopping
- Merchandising
- Feedback
- Client Alerts
- Platform Notifications
- Large Merchandise
- Research

eBay's roadmap page and current .PDF. eBay is also implementing these APIs using the gadgets specification as defined within OpenSocial.

What appeals to me here is that, unlike "raw" hosted compute services (a la AWS), and unlike raw platform tools (a la Google App Engine), eBay has chosen to host core services for use (and re-use via mashups) by others. eBay is putting its core IP on the web for others to use.

Not convinced at the potential? Consider the following thought experiment: What if eBay provides a highly-reliable, generalized trading application that matches Puts and Calls? What if a public company were to allow the engine to match and record trades of its shares using that engine? Could it eventually usurp the NASDAQ engine?

Now think even a bit more differently -- rather than a service that matched simple pricing (for shares or for simple commodities), what about a service that matched trades for more complex variables? For example, different grades and granular chemical compositions of crude oil, complete with amounts and delivery dates? Jobs and job seekers? Singles looking for dates?

eBay clearly has the opportunity to be more than an online e-commerce site. It could be the "Exchange Platform" in the sky. Very cool possibilities, well beyond the simple way we're thinking of cloud services today.

Intel Nehalem + Dell + Repurposing

2009-03-31T05:08:00.000-07:00

With all of the Intel/Nehalem (Xeon 5500 series) processor announcements this week, one stands out.

As Dell announced it's new 11th-generation (11G) server product line, Egenera also announced PAN Manager support for Dell hardware, such as Dell m610 11G blades as well. This means that Dell blades are exceptionally well-suited for high-performance, highly-consolidated, virtualized workloads -- all with five-9's of reliability. It also means that Nehalem-based hardware, networking & infrastructure can now be managed using infrastructure orchestration (what others are calling unified computing) today.

That's a huge shift from only a year ago - Dell finally has an enterprise-grade, mission-critical level-of-reliability offering for virtual, physical, or mixed applications. Citrix, Microsoft, VMware, Linux, Unix.

Very cool, guys.

Videos: What is Infrastructure Orchestration

2009-03-24T17:18:00.000-07:00

My first endeavors into online video. No pictures of me, but at least you get the audio.

Here is a brief, high-level overview of what's meant by Infrastructure Orchestration, including some messy annotations of mine while I speak:

And here's another video of it in action: The Dell PAN system that you can buy today:

Join me at NY Cloud Computing Expo

2009-03-23T17:19:00.000-07:00

This week I'll be putting the finishing-touches on my Day #1 presentation about "Building a Compute Cloud Infrastructure" and where to begin (3:25pm next Monday, to be exact). It's really about my experiences of how enterprises and SPs alike have been approaching building a foundational "Infrastructure-as-a-Service" -- regardless of whether they're planning a virtual, physical, or mixed environment.

Syscon has put together quite an assembly of keynotes - lead-off by Werner Vogels of Amazon and Kristof Kloeckner of IBM.

My angle is simple: I find that whenever and wherever people talk about clouds they usually draw a little stack diagram... and at the bottom of the stack is usually a box that says "commodity infrastructure" on which all of the virtualiztion, PaaS and SaaS stuff sits. Well, what *IS* that foundation made from? How do you assure a scalable, five-9's infrastructure for your cloud (or for anything else for that matter)? So clearly I'll also be alluding to Infrastructure Orchestration (aka unified computing) as well.

And that's it. I plan to first discuss the blind assumptions many of us in IT have been following when architecting systems, and how to get around these 'old world' assumptions. I'll then transition into how enterprises been constructing these more adaptive, resilient IaaS foundations. Also, I'll touch on how they work and what they look like in real life. That's it.

Hope to see you Monday in NY at the Roosevelt Hotel.

Unified computing is so easy - 6 easy steps

2009-03-23T15:36:00.000-07:00

With all of the chatter now about "Unified Computing" - as well as all of the skeptics thinking it's blue-sky future, I wanted to outline how incredibly simple it really can be. Even I can do it :)

At Egenera, we've been in this business since 2001, terming this technology "Infrastructure Orchestration."

When I asked my SEs how to describe using PAN Manager with the Dell PAN system to abstract data center infrastructure in software, they gave me back a surprisingly simple set of instructions. Not technical acronyms or jargon. I personally watched (and participated in) getting a compute environment, complete with high-availability fail-over and DR, up-and-running in under 15 minutes. And the cool part is that it included both native OSs, as well as VMs.

Using our GUI, the Administrator then

Defines resources –Identify available individual building-block resources which include pools of blades, internal switches, external switches, disks/LUNs, and OS images.
Organizes resources – Define logical groupings & access privileges for different pools and/or allocations as-needed by the business. Each group and its resources are distinct and secure from the others.
Builds profiles and servers – Assign physical blades; assign network connectivity; assign disks (each LUN is presented as a SCSI device); assign an OS (which could a native OS, as well as a VM host OS like VMware ESX); finally, boot the server profile
Assigns HA policies – Specify specific failover blades or shared pools, before or after building/booting the server
Defines DR policies – Entire server environment configurations (or subsets) can be defined and instantiated either on-demand, on-schedule, or any other reason.
(Optional) Reassigns servers – As simple as point/click/reboot. More than one server profile can be assigned to each blade. Change can be triggered via schedule or other commands.

The beauty of defining the server and its infrastructure in software is the ability to re-create it on demand, from a true "bare metal", wire-once environment.

Any engineer would appreciate how elegant a solution -- and set of user instructions -- this is.

A data center trifecta? (Prediction)

2009-03-18T09:46:00.001-07:00

Once in a while, I sit back and think about what are the real transformational forces that will change how IT operates. And I've come to the conclusion that there are 3 fundamental technological movements that will do this... and the 3rd will surprise you.

1. Software virutalization.
No surprise here. Abstracting software from the underlying CPU yields mobility, consolidation, and degrees of scalability. It also simplifies automated management and portability of workloads, such as with virtual appliances / AMIs. Aside from a few managerial kinks being worked-out, this technology is already accepted de-facto, esp. because we're already seeing it become a commodity play, and sedimenting into other products.

2. Infrastructure orchestration / unified computing.
This is the one we're all beginning to hear about. As I recently outlined, this technology is a perfect complement to software virtualization -- it essentially gives "mobility" to infrastructure, rather than to software. It allows IT operations to define I/O, storage connectivity and networking entirely in software, resulting in stateless and re-configurable CPUs. Egenera was the pioneer in this area, but the market is now getting an adrenaline shot-in-the-arm from Cisco's UCS announcement.

Unified computing / Infrastructure orchestration is valuable because it enables a highly-reliable, scalable and re-configurable infrastructure -- a perfect platform for physical *and* virtual software. It permits IT to "wire-once" and then create CPU configurations (virtual NICs, HBAs, networks/VLANs, storage connections) using a unified/consolidated networking fabric. Plus, it is a simple, elegant, cleary-more-efficient approach. Think of this as provisioning hardware using software. This approach has numerous positiv properties - not the least of which are that you can clone hardware configurations when you need to (a) scale, (b) migrate, (c) fail-over, and (d) recover from entire system failures [disasters]. Again, regardless of physical or virutal payloads.

We'll see this technology begin to accelerate in the market. Promise.

3. Intelligent software provisioning
"Huh?" I hear you say? Yes! While I'm not sure what this market segment may eventually be called, it represents the third critical data center managemnet component. #1 gives software mobility; #2 yields infrastructure flexibility. And #3 is how the actual software (physical, virtual, appliances, AMIs, etc.) is constructed and "doled-out" as a workload on #1 and #2.

My eyes were opened after learning more about a company called FastScale. Picture an intelligent software provisioning system that knows minimum-amount-required of software libraries needed to run an OS or application. As it turns out this is usually something only around 10%-15% of the multi-gig bag-of-bits you try to boot every time you bring up a server. And that even includes the VM, where virtual systems are involved.

The result? 3 really nice properties:
a) Speed. Getting applications up-and-running an order-of-magnitude faster. Not having to move as many bits over the network to boot a given server is a real time and money saver.
b) More efficient consolidation. With smaller software footprints, more VMs, appliances, etc. can fit on a given memory footprint. That means that denser consolidation is frequently possible -- not to mention $ savings on those gigs of memory you have to buy when you consolidate.
c) Inherent configuration management. With a database of all libraries and bits, plus knowing where you put them, you can always monitor configurations and verify compliance, etc. Plus, you can track what patches went where (and frequently, you may find you don't even need the patch if it's not a critical component to the reduced libraries you're using!)
d) Ability to provision into any form of container: In other words, this system can provision onto a bare-metal CPU, into a VM, or for that matter, into an appliance like an AMI if you're using a compute cloud. Wow, very neat.

This intelligent provisioning approach is also highly-complementary to existing compliance and config management products like OpsWare (HP) or BladeLogic (BMC).

Summary
So what if you have all 3 of these technologies? You'd have a data center where

Workloads were portable, and relatively platform-independent
Infrastructure was instantly re-configurable and adapted to business conditions, failures, etc.
Software could be distributed and brought-up on the order of seconds, allowing near-instantaneous adaptation to scale, business demand or failures.

Pretty sweet, eh?

What is Infrastructure Orchestration / Unified Computing?

2009-03-16T07:45:00.000-07:00

Infrastructure Orchestration and Unified Computing are both terms referring to a technology whereby server CPU, I/O, storage connectivity, and network, are all able to be defined and configured in software. The advantage of using this approach allows IT operators to rapidly repurpose CPUs without the constraints of physically having to reconfigure each of the I/O components by hand – and without the requirement of a hypervisor. It massively reduces the quantity and expense of the physical I/O and networking components (because much of the I/O is consolidated) as well as the time required to configure them. In return, it offers an elegant, simple-to-manage approach to data center infrastructure administration.

From an architectural perspective, this approach is also referred to as a “compute fabric” or a “Processing Area Network” since the physical CPUs are made stateless because their physical addressing (of I/O, Network and Storage naming) is completely abstracted away. And, by abstracting the I/O, both data and storage connections can be converged, further simplifying the network infrastructure. What is left is a collection of raw, pooled CPUs that can be assigned on-demand, and whose logical configurations and network connections can be instantly defined.

Infrastructure Orchestration is very different from – but highly complementary to – hypervisor-based virtualization. Think of hypervisors as operating “above” the CPU, abstracting software (applications and O/S) from the CPU thereby giving the software portability. Think of Infrastructure Orchestration as operating “below” the CPU, abstracting network and storage connections, and thereby giving the CPU itself portability. Note that a major difference is that the Infrastructure Orchestration does not operate via a software “layer” the way that a hypervisor does.

The complementarity between Infrastructure Orchestration and virtualization is significant. Take an example such as a VM host failure, where the entire physical machine, network and storage configuration needs to be replicated on a new physical server. This can be accomplished with a spare “bare metal” server where a new host can be created on the fly, all the way down-do the same NIC configuration as the original server.

Now, expand this example to the scenario of an entire environment failure. Infrastructure orchestration can re-create the physical machine hosts as well as their networking, on an otherwise “cold” bare-metal and non-dedicated infrastructure at a different location.
Best of all, the properties of Infrastructure orchestration, such as the ability to provision a new server quickly, apply to both physical servers as well as to virtual servers. So this is an ideal technology to use when managing mixed physical and virtual environments, including “cloud computing” infrastructure.

Finally, Infrastructure Orchestration (unified computing) is a central technology to creating highly-reliable, dynamic data center. The technology is also core to a “Real Time Infrastructure”(as defined by Gartner Research) as well as to “Organic IT” (as defined by Forrester Research), both computing architectures that rapidly responds to changes in demand, to failures, and to unpredictable business demands.

First fruits from the Dell/Egenera deal

2009-03-12T08:53:00.000-07:00

It appears the Egenera/Dell deal is moving forward with velocity. A first joint customer was announced today, the US Department of Veterans Affairs (VA) Corporate Data Center Operations (CDCO).

This is the first of a pipeline of customers buying-into Infrastructure Orchestration (also referred to as Fabric Computing or Unified Computing) - first offered in 2001 by Egenera with their high-end BladeFrame + PAN Manager software, and now being mainstreamed as the Dell PAN System by combining PAN Manager with Dell hardware.

The VA's CDCO is really a hosting facility - much the way an xSP hosts applications for third-parties. In this case, they're hosting a mission-critical application for an influenza early-warning system. I've met with their CTO, who's a pretty forward-thinking guy. He recognizes the fact that his "customers" frequently change requirements, and computing demands frequently change too. So, for an environment comprising physical databases and virtualized instances, the Egenera/Dell system provided significant agility (ability to re-provision quickly) while maintaining a mission-critical level of availability.

The Dell/Egenera deal has been getting some profile lately - as it takes on similar technologies such as IBM's Open Frabric Manager, and HP's Insight Orchestration products. Stay tuned for some more juicy news.

Energy efficiency and dynamic infrastructures

2009-03-11T11:06:00.000-07:00

I just read an interesting blog by Rob Aldrich on the advantages of "unified fabrics" when pursuing energy efficiency.

This story is more than just about gaining energy efficiency by reducing the amount of network hardware. Rather, it's all about using (and re-purposing) compute hardware more effectively.

A "unified fabric" is part of what we term "infrastructure orchestration" and "unified computing" - it is the abstraction of compute, I/O, storage and network infrastructure into a dynamically-configurable "fabric". In that way, servers and their associated infrastructure, can be logically-created and/or re-configured. It's a fantastic complement to virtualization. One way to think about it is that VMs provide a logical way to create or re-configure new software server stacks. In turn, infrastructure orchestration (the "fabric") is the way to provide a logical way to create or re-configure I/O, network and storage. So when you're moving VMs around (on purpose, or in response to an unplanned event) you can create compatible infrastructure on-the-fly.

What's the efficiency story here? The ability to re-purpose entire compute, network and storage systems on-the-fly, in response to compute demand. By using these resources more efficiently, data centers ultimately need fewer physical assets -- and those assests consume less power.

Going a step further, these assets can be re-purposed in response to energy efficiency triggers. Workloads can be moved a few feet in response to "hot spots", or to entirely different geographic locations based on power cost/availability.

What if software virtualization is the tip of the iceberg?

2009-03-05T14:17:00.000-08:00

Yesterday I attended the IDC Directions conference in San Jose. Chock-full of great presentations, networking and predictions, topped-off with a closing keynote from Nick Carr on The Big Switch.

Over lunch, I was able to sit-in on a discussion with two IDC analysts discussing how virtualization is expanding the role of the network, and the steps IT operations were taking to simplify the data center. It was all about unifying and consolidating the data center network, they said.

"But aren't we unifying and consolidating servers too?" I thought. And then it occurred to me that perhaps there is a logical progression of simplification that IT is doing, without knowing it's doing it. Maybe it's the "consolidation maturity model" or just stages of IT's "simplification progression." Either way, it seems to dovetail with how the industry is progressing.

Seems to me these 4 stages happen roughly in order:

1. Hardware consolidation. We're in the thick of this today. When most IT professionals think "virtualization" they are mostly wanting to consolidate/reduce hardware. Software is abstracted away from the CPU, making management of applications more efficient. Players: VMware, Citrix, MSFT, Parallels, etc.

2. I/O consolidation: As I've pointed out in the past, I/O has unfortunately been tightly-bound to the CPU, creating complexity in configuration, addressing, wiring, etc. Instead, I/O can be abstracted in software, and then logically instantiated/assigned. The industry is now beginning to realize that I/O consolidation -- especially in the world of many VMs per host -- is helping to simplify IT management. Players: 3Leaf, Xsigo, etc.

3. Network consolidation: We're just beginning to scratch the surface of network consolidation, with a few competing technologies in the space. But essentially it's about converging data, storage, and even out-of-band management information along a single (high bandwidth, low-latency) wire. This concept is an ideal complement to I/O consolidation. All major networking vendors, plus peripheral vendors like Qlogic and Emulex

4. Compute consolidation: Perhaps the final, but highest-value, step... and it's very different from Hardware consolidation. This is about being able to create stateless CPUs, with the abililty to logically assign them on-demand, to different workloads. It is about creating a pool of CPUs (similar to a pool of network resources or a pool I/O, above) that can be used most efficiently, because they are ultimately re-assignable. Players: HP, IBM, Egenera

Now, this progression doesn't have to be followed strictly. Most of the industry is certainly doing #1, with their favorite VM vendor. Point-product vendors are pushing #2, while networking vendors are pushing #3. And the forward-thinking folks are already pursuing all four to create truly dynamic IT foundations.

But it is seeming to me that what we see as Virtualization (hardware consolidation) is just one of a few upcoming "waves" of change about to simplify how IT is operated and managed.

Egenera PAN now available on Dell blades

2009-03-03T12:24:00.000-08:00

Since about 2001, Egenera has been selling its high-end hardware/software combination that unified re-purposing of physical and virtual servers. It combined super-high-performance blades, with their PAN Manager software. PAN stood for Processing Area Network, akin to a SAN in its ability to logically re-allocate CPUs, and its ability to provide ultra-high reliability. Hundreds of customers at thousands of locations knew this to be the creme-de-la-creme of solutions.

In his bl og today, Egenera's CTO talks about PAN Manager now available on Dell Blades.

That means a whole-new level of affordability/performance. It also helps put Dell's blade strategy square in the data center space - and able to take on HP and IBM. Out-of-the-box, Dell blades can support mission-critical levels of availability, regardless of P or V payloads.

It also endorses Egenera's historic approach to Infrastructure Orchestration and the network fabric (what others are also calling unified computing).

IT's Blind Spots in the data center

2009-03-03T11:51:00.000-08:00

A few weeks ago I wrote about where IT Ops might be missing-the-boat regarding implicit assumptions we're making as an industry. I called these the "big blind spots". More from Egenera was published today in the Wall Street Journal's MarketWatch website.

I arrived at these observations when I speak with data center managers who admit that they don't have solutions to these issues. When I tell them about Egenera, their eyes widen, and they go "you can do that?"

The net-net of these "blind spots" is:

1. Virtualization does not equal agility.

Sure, VMs get you part of the way, but what about agile infrastructure? This issue seems to have been overlooked. We still have to run-out to the floor to change NICs, switches, etc.

2. Virtualization does not equal simplification

In many ways, virtualization simplifies sofware. But we also fail to notice the complexity it creates for managing I/O, storage connectivity and other physical-world management issues.

3. Not all High Availability (HA) and Disaster Recovery (DR) is solved by VM technology.

Implicitly assumed in the market is that VMs are the panacea. But the reality is that not everything gets virtualized... therefore there are "siloed" needs for HA & DR

4. Provisioning is assumed only for software.

Everyone seems fixated on Provisioning = Software. But hardware, I/O, storage and network have to be provisioned too.

5. Not everything is, or will, be virtual.

c'mon folks. Depending on who you speak with, only 20%-25% of new servers are virtualized today. And many large data center operators admit that some servers will *never* be virtualized. So, what are we going to do about unifying P+V management?