Wednesday, April 30, 2008
We opened with 4 perspectives on public-sector data center work:
Andrew Fanara (of the EPA Energy Star program) gave an overview of their progress toward an Energy Star spec for computer servers for Tier-I/Tier-II data center use. The first version of the spec has already gone out for review/comment; the 2nd draft is dues mid-June; the 2nd stakeholder meeting is set for July 7 in the Seattle area; the final version of the spec is targeted by the end of CY2008. Andrew expects the spec to focus on 5 areas:
1) Net AC/DC power efficiency
2) Standardized reporting/polling
3) Power and temperature reporting
4) Idle power levels
5) Power management and virtualization “hooks”
On Buildings & data centers:
Michael Zats (of the EPA buildings division) then went on to describe the Energy Star Labeled Buildings program, and plans to extend the ratings system to data centers. There are currently 1,700 partners leveraging the energy star for buildings program, representing 11 billion square feet of commercial/industrial space. Over 70,000 buildings are currently measured/tracked, and 4,000 buildings have the Energy Star lable (representing the top 20% of the population). Michael is not spinning up a similar program for data centers, and asked the audience for volunteers – EPA needs at least 125 data centers to populate their comparison data base (with 12 months of data) to begin the process… and if you’re interested, materials can be found on the Energy Star Site ; additional inquiries can be made at firstname.lastname@example.org
On Data Center Technology:
Paul Scheihing of the US DOE reviewed their Save Energy Now program and their DC-Pro data center assessment too plans. Their goal is to drive a 25% reduction in energy intensity for data centers. They are/will be providing Tools, training, information, standards and assessment tools to the public. But what’s really great about this initiative (far ahead of any private-sector initiative, as far as I can tell) is that it’s got these multiple resources. The tool not only helps you rank the overall efficiency of your data center against others (confidentially, of course) but will provide tons of input/ideas as to where to look for savings and efficiencies. (I actually grabbed Mike Zats outside the hall, and he spoke highly of the DOE’s tool, almost to the point where it was too sophisticated). Paul also made mention of DOE’s sponsoring some technology demonstration projects, but I have yet to see how/when/where these will be presented.
On other public sector work
Ray Pfeifer of the Silicon Valley Leadership Group’s Energy Efficient Data Center Project then gave an overview of their initiatives, with a goal in CA of a 15% reduction in industrial sector consumption. (Cassatt, I might add, is an active member of this project).
There was then an insightful panel covering external work being done by ASHRAE, SPEC/SPECpower, and the Green Grid – and then, somewhat of a treat: Tosh Shimizo, GM of IT and new business with Tokyo Electric Power Co. gave an overview of his company’s data center facilities and level of power grid reliability. They provide 64GW of power to Tokyo, and have only 3 mins of downtime/year… far better than what is found in the US, the UK, or France.
The second treat was Will Forest of McKinsey & Co. presenting a fantastic data center efficiency report to the group. It’s chock-full of great charts and analyses In short, it focused on improving Power Capacity planning, power efficiency, and asset management. Organizationally, he also advocated the “one throat to choke” approach to making the CIO accountable for the TCO of the entire data center (assets + facility). McKinsey’s recommendation was CADE (Combined Average Data center efficiency) essentially Facility efficiency x Asset efficiency. Deeper-down, facility efficiency is composed of facility energy efficiency * facility utilization, and asset efficiency is composed of IT energy efficiency * IT utilization. Simple…. But I suspect there will be disagreement on units…
Finally, for me, Christina Page, Yahoo!’s director of climate and energy strategy gave a lunchtime presentation on curbing carbon emissions, and what Yahoo! is doing to become carbon-neutral. The big takeaway: not all kilowatts are created equally – be careful *where* you use them (is the source coal? Nuclear? Wind? US? India?) and *when* you use them (peak? Off-peak?). These factors are as important as the net kW that you consume.
Thanks for reading. Now it’s back to the airport…
… and don’t forget the theme: measure, measure, measure.
Tuesday, April 29, 2008
I do have to note the slow start today bsy ome vendor-sponsored Keynotes. It opened with Mark Thiele, VMware's Director of Business Operations, giving IMHO a distinctly plain-vanilla overview of virtualization's benefits. Yawn. But things picked up with Henry Wong of Intel (and a frequenter of meetings such as this, the Green Grid, and others) giving a somewhat more interesting retrospective -- he had found a presentation given to this group in 2006 called "picking the golden nuggets", 5 suggestions he made to the group a few years ago:
- Use power-saving features
- Turn-off comatose servers
- Eliminate bloatware
- Improve site infrastructure energy efficiency
The third, and most interesting am keynote was from Dave Anderson, IBM's Green Architect. He advocated a completely holistic approach to managing power and energy efficiency in data centers, and predicted that Cloud Computing is ultimately where IT is evolving to. Resources will be dynamically allocated -- with data centers compared to orchestras, where "ensembles" of similar equipment being purposed and re-purposed for like tasks. This is the only way we'll reap breakthrough IT efficiency. Kinda sounds familiar...
Next was a panel on "The business case for IT energy efficiency" moderated by Jon Koomey (of Stanford U as well as Lawrence Berkeley Labs) and Debra Grove, and including Andrew Fanara of the EPA/Energy Star program, and representatives from Allstate Insurance, Yahoo! and HSBC. Talk was wide-ranging, and included the (highly likely) legislation that could help cap/trade carbon emissions, and closing the social division between IT ops and Facilities. And I believe that all agreed that "green" ultimately has to result in $$ savings, not just goodwill.
Following lunch, I gave a talk on "Is it bad to turn off servers - and other myths hurting energy-efficient operation of your data center". Long title, short message: "Conventional wisdom" in data centers is confining our thinking, and limiting breakthrough efficiencies. If we look to combine efficient equipment with efficient operation of that equipment, then we can move beyond incremental efficiency increases.
I then attended an insightful presentation on Energy Star computer power supplies by Chris Calwell of Ecos, and a SPECpower overview by Klaus Lange of SPEC. These guys are the ones doing the heavy-lifting for our benefit. SPECpower had been initiated back in 2006, and is an elegant approach to rating compute load vs. power consumption. It's so easy to run that you can even do it at home -- even though vendors are expected to provide customers with a series of data for their own products. Finally we have a generally-accepted way to benchmark server power consumption under load! Similarly, studies are being done on power supplies for Tier I servers by Ecos (on behalf of Energy Star) for a forthcoming Energy Star ratings. It became clear that many power supplies are likely over-sized -- and, at low loads, are (1) terribly inefficient, and (2) have a seriously-degraded power factor. But there are products out there with superior (90-95%) performance at nearly all power levels.
We closed today with a reception as well as a series of Uptime Institute Green IT awards for a number of innovative projects - which I believe will be posted to the Uptime Institute Awards site shortly.
Oh - and a somewhat retold refrain heard from a number of participants today: "what are we doing when we get back to the office? measure, measure, measure..."
Monday, April 28, 2008
Ken Brill and Pitt Turner both gave pretty compelling keynotes -- first, by challenging the participants to providing a 50% energy reduction for data centers in the next 36 months. The possibility of this was driven home by the fact that Amazon Web Services can already deliver a CPU-Instance/Hour for $0.10. Could the rest of us? But more on all of this later.
Ken pointed out that at the current rate, there will be 15 million servers running by 2010, with that number doubling every 5 years or so. And what was really riveting was that the largest 1/3 of membership's data centers is growing at 27% CAGR; and even the average data center growth for membership was 12% CAGR. Clearly there is a massive construction (and energy use) boom in data centers in the U.S. This spells crisis.
Next, he focused on the excalating power/OpEx costs of operating a $2.5k server -- with much data taken from Jon Koomey's excellent studies. In a Tier II data center, that server's OpEx costs for power topped $5k; in a Tier III data center, OpEx costs were $7.7k, and in a Tier IV environment, power OpEx costs were a whopping $8.4k... all with power costs far outstripping the capital costs.
And so, Ken's request to the group: the only way to obtain a breakthrough 50% improvement on data center power consumption was to look at IT Operations, as driven from top-line management... rather than to pursue it from incremental facilities-based improvements.
Ironically, two of the first three sessions of the morning - one on DC power, the other on power & cooling - focused on infrastructure improvements. And this, in my opinion, set the somewhat sobering theme for the day: while the participants would *like* to get to breakthrough energy strategies, the vendors are mostly stuck in pursing only incremental improvements. (Example: why would a leading power/cooling vendor focus on talking about hot/cold aisle containment, when they could have included forward-thinking adaptive management technologies that place cooling where it's needed while encouraging compute workload management?)
Next I attended a reasonably interesting roundtable panel titled "What can IT executives to Right Now to Increase Energy Efficiency?" co-chaired by Will Forest and Matt Stansberry. It became clear pretty early that the issue-at-hand wasn't one directly of technology. Rather, it was social: IT is incentivized on Uptime and on service delivery... not on power/energy cost. So, the concept of chargebacks, cost-transparency, and the "shining a flashlight" on cost are some of the places to begin to change behavior.
There were a number of other interesting breakout sessions including mine, chaired by Jon Koomey titled "Enabling Power Features to save 30+ percent". Jon kept this pretty focused, and we had a lively session with representives from HP, Microsoft, IBM, Visa and me from Cassatt. In the end, I think the room agreed that power-management and server hibernation features could indeed be leveraged to save significant power, but had to be implemented wisely, with potential tradeoffs between efficiency and high-availability. But, in a number of cases, this approach was highly warranted.
Other fun diversions today:
- Chatting-it up with the very insightful bloggers Dave O'Hara of the Green Data Center Blog and Debra Grove of Grove Consulting (also with an excellent overview of the day in her blog)
- Taking a tour of Rackable Systems' ICE Cube Data Center container
- Sharing a drink (or two or three) with Kevin Heslin of Mission Critical Magazine
Tuesday, April 22, 2008
The Survey was a really wide-ranging look at the IT industry - who's taking what initiatives, where are the industry pain-points, what sources of information are trusted, etc., and we found a few surprising data points as well. Such as
- Where most enterprises go for "green" IT information may not be where you think
- The degree of energy waste in different departments is more acute than you might believe
- The willingness of some professionals to take new/different approaches to power management is more aggressive than we believed
I've been spending time researching a number of other very excellent data center surveys being conducted, and you should look at them together to provide a rounded-out picture of the state-of-the-industry. Here goes:
- My initial blog from January about the Cassatt survey
- My follow-up blog citing additional 3rd-party surveys (from AMD and SearchDataCenter) that had been conducted in the market
- My third blog - from March - citing more excellent survey work, this time by the Uptime Institute as well as from Gartner Research
- An excellent piece by Stacey Higginbotham (at Earth2Tech) that references the BPM Forum study outlining how energy consumption is still out-of-control
- A piece I found on Greenercomputing.com referencing a report by Lean and Green Leadership. A free synopsis of the report is available for download.
Monday, April 21, 2008
I recently floated the idea of implementing a mix of virtualization products in your data center as a way to better customize your virtual environments. The query was part of a larger discussion I wanted to get going about how VMware will compete when Hyper-V is generally released. I threw out the notion that data center managers might use, for example, Hyper-V for end-user file servers; VMware ESX, for apps that require dynamic load balancing, sophisticated disaster recovery and migration; and Xen for commodity Linux boxes.
The idea behind that supposition was to match your enterprise investments to appropriate workloads because, let’s face it, running everything on ESX is going to be expensive compared to other options. Big deal if you don’t get ESX-level features because you may get enterprise level features on silver-medal products.
A Supposition? No - more like a reality. If you look around, there is already VMware pervasive in the market, Parallels Virtuoso staking a claim in the MSP/hosting market, Xen making inroads in the Linux market, and other folks like Virtual Iron, Oracle and others about to pounce. Add-in the fact that IBM, HP and Sun each have their flavors of containers/partitioning. Oh... and don't forget that sometime soon you'll see virtual appliances (like rPath) that have embedded VMs from various vendors. And then, of course, Hyper-V.
Inside of 12-18 months, it will be a fact that data centers will have multiple virtualization technologies.
Strategically, organizations will need to prepare themselves for this - rather than create "virtualization silos." As you would expect the Invisible Hand to act, there are already a number of vendors looking closely at this space -- multi-virtualization management. Cassatt clearly comes to mind - but there will be others.
The next data center management "high ground" will certainly be those platforms that can leverage/control the multiple VM technologies out there, and dynamically control workloads, failovers, scaling, security, etc., including managing those machines that are not appropriate to virtualize at all. And my prediction is that the incumbent management platforms (i.e. OpenView & Tivoli) will not be in a position to do so. Look to the agile startups for that.
Sunday, April 20, 2008
Why validating? This isn't just a remix where you mash-up two songs - this is the artist offering-up original unmixed tracks... It's exactly the open-sourcing of music I predicted on my March 4 blog.
Boy, would I love to see other artists begin to do this -- It could be just what the recording industry needs to spice-up the market. Turn fans into an extension of the artists themselves!
Monday, April 14, 2008
And here's the kicker: He knows that during nights/weekends, user demand drops off precipitously.
Conventional wisdom has it that *all* of these servers should be on and ready 100% of the time. Hmmm. That's 400,000W (think 400 blow driers) and probably another 400,000W or so to keep them cool.
Fortunately, our friend realizes that even during peak hours, not all of these servers need to be on... and off-peak, many fewer. What he *does* need is to ensure availability -- that at any time he has headroom for a momentary peak, and that he can bring on more at a rate that keeps up with demand.
This is not a solution that virtualization, or more efficient power supplies can impact; but by breaking with conventional wisdom and power controlling the servers based on usage/demand, we hope that his organization will save $$ and even regain precious power/cooling headroom.
Sunday, April 13, 2008
His brilliant tenet is this: Millions of $ have been spent developing foundational technology like the Wii Remote -- which can then be applied to dozens of *different* applications. Factor in the ability of YouTube to share ideas, and you've got a fantastic acceleration of technology and solutions.
Friday, April 11, 2008
First, everyone agrees that there are quick-hit actions to be taken around cooling (covering blanking panels, clearing air ductways, etc. etc.). These are super-simple, super-quick, and have a modest payback. To be sure, there are more complext/long-term cooling initiatives (e.g. air-side economizers, higher-efficiency chillers & CRAC units, etc.) but I'll skip these for the moment.
Second, there is the ballyhooed consolidation initiative. Clearly this is a high-impact approach to $ savings, but it's also going to be a year + before the full savings is achieved. (FYI, it took Sun Microsystems 12 months to complete their consolidation, and Gartner Research found that half of survey respondents expected that a consolidation project would take between 7-18 months). Also, assume that once you make the investment, it will take a year to reduce your server count. And, because most hardware is on a 3-4 year depreciation schedule, it could be as long as ~ 4 years or more before those machine expenses come off the books, and the project has complete payback.
Third, and the quickest-hit approach, is simple server power management. Implementing intelligent server shutdown (based on schedule, load, etc.) takes only a few months at most, and doesn't involve any SW configuration changes. Payback can be < 1 year. Granted, you don't remove any machines, but the power savings are essentially immediate -- and if you still want to go ahead and do a consolidation project later, you can continue to power-manage the consolidated machines.
I expect that we'll be seeing more from DOE, EPA/EnergyStar, Green Grid & Uptime Institute on these tradeoffs -- between impact/complexity, and between efficient equipment and efficient operation of that equipment.
Wednesday, April 9, 2008
"...solution, built on TCS' best practices and proven methodologies, is enabled by Cassatt's software, which combines goal-driven automation with virtualization control to help customers create a real-time IT infrastructure that can achieve desired application service levels. TCS and Cassatt are now on their way to creating a highly visible Next Generation Infrastructure lab that will host Utility Computing enabled by Cassatt as a key solution station.Despite the fact that John once believed that Cassatt's products were "just plain silly", it would appear that he's come around...
And, BTW, he writes an awesome data center blog, e.g. recently wrapping up Data Center World, Green Data Center Topics, and a number of newsworthy blog feeds.