Monday, April 23, 2012

Will The "Stack Wars" Impact the Hybrid Cloud?

It's been a busy few weeks in cloud O/S land, first with the Amazon/Eucalyptus announcement and then with Citrix/CloudStack announcement, then last week's OpenStack show in San Francisco.

The market may be starting to shake-out in terms of dominant cloud APIs, major efforts to manage/govern cloud infrastructures, and who will (or will not) influence/direct these efforts. But the end-game is far from clear.

From the cloud API perspective, there are a few likely contenders/winners here:
  • The Amazon Web Services (AWS) approach: arguably dominating the market at the moment, explicitly supported by Eucalyptus, as well as by Nebula, and CloudStack
  • The OpenStack approach, which likely will adopt some of the AWS approach, but possibly in favor of another
  • The VMware vCloud approach, arguably dominant in the enterprise space, and being partially adopted by large service providers.
  • Other dominant SP/vendor approaches such as Google and MSFT Azure
From an infrastructure management code base perspective, we also have
Note that it's important to distinguish between the cloud APIs and the infrastructure management code bases.  As pointed out in the Rightscale Blog, there will likely be multiple implementations of given APIs... and that's good because it allows for innovation around factors such as performance and underlying technologies.

The third "leverage point" of cloud management and standardization is of the virtual machine images such as the Amazon Machine Image (AMI) or the VMware VMDK.

    What this all means to the future of Hybrid Cloud Computing

    I'm looking at all of this from the perspective of the future cloud ecosystem - one where cloud services (and infrastructures) can easily interact with each other. 

    The question that got me thinking is whether hybrid cloud service and provider ecosystems will ultimately be loosely-coupled or tightly-coupled?  In other words, will there be significant consistency between "like-designed" and "like-implemented" services... or not?

    The foundation of my question consists of two scenarios I believe will be played-out:
    1. A few close-knit ecosystems, each using a single cloud API and Machine Image, creating a small number "tightly-coupled" clusters of providers.  (No one would deny that AWS/AMI will be one of the winners) In this world, it would be relatively easy to achieve portability between cloud service providers - which doesn't just mean code, but would include portability of policies, monitoring, etc. as well.

    2. An larger, fragmented ecosystem with a more "loosely-coupled" set of players.  Here we might see a number of heterogeneous APIs, Machine Images and infrastructure technologies, perhaps based on special-case uses, e.g. for community or vertical-industry cloud providers.    
    In the first Tightly-Coupled scenario, which smells more like a "standards" play, use of only a few winning APIs and Image types would make it easier to deploy consistent tools for monitoring, metering, compliance, security, availability, etc. across different providers' infrastructure, simplifying efforts for enterprise IT.  Providers might compete on efficiency of implementation, for example.

    In the second Loosely-Coupled scenario, there might be quite a number of varied implementations, Image types and APIs. This approach might be advantageous for individual special-purpose cloud uses. But it would also require that integration across providers would rest on the shoulders of the user, likely Enterprise IT.  Lots of "glue logic" would be needed to integrate the various SP services into the enterprise's own technologies and processes, and potentially even use of conversions of code from one image type to another.
     
    Are Allegiances Forming?

    I'd say so, particularly of the "Tightly-Coupled" variety. Already, Amazon has licensed its API to Eucalyptus (see James  Staten's blog "Has Amazon Solved Its Private Cloud Dilemma?" or Lydia Leong's "The Amazon/Eucalyptus Partnership"). In my opinion, we'll see even more of these API alignment agreements, perhaps after this deal has some early user successes.

    Similarly, VMware is fronting its vCloud approach and machine images to a host of aligned Service Providers.  And even Microsoft is encouraging the use of Azure APIs and technologies to create private clouds.

    As the public and service provider cloud market matures, natural "Camps" will form around them.  Ultimately, the first question that we might find ourselves asking is "Are you a Microsoft, Amazon or VMware Cloud?". 

    Kind of a modern version of the "are-you-Mac-or-PC?"
    Posted by at 3 comments:
    Labels: ,

    Tuesday, March 6, 2012

    Meeting IRL: Why The Valley's So Cool

    IRL is a reference you see on twitter -  it means "In Real Life."

    Sometimes we underestimate the value of meeting in real life. It's more convenient to email than to pick up the phone. And more easy to call than to arrange to have lunch. And for those of us addicted to Twitter, we feel as if we have our own community of followers and frequent commentators at our fingertips.

    The value of meeting in real life struck a chord a few weeks ago while in Santa Clara at the Cloud Connect conference.  Many of the speakers, start-up stars, consultants, bloggers and tweeps were all known to each other.  Its a very clubby bunch... but mostly online in the virtual world.

    Once at the conference, there was this explosion of meetings and meetups in the hallway, over lunch, over drinks, even over Falafels (you know who you are).


    And then there was a re-launch party given by CloudScaling Inc.: Open bar, DJ, packed room, endless conversation groups - a who's-who of the Clouderati, hangers-on, and maybe even cloud groupies.  But symbolically, this party was important.  It not only signaled that "the valley is back", but also illustrated to me the importance that all of these people, from all over the country, needed to meet in real life. 

    In fact, I bet that the majority of the "who's who" in cloud - top consultants, bloggers, influencers - where all there, congregating.  No, not the Masters of Industry. But the people doing the work, the innovation, the evangelizing. Yes, it was clubby. And surprisingly small, if you think about it as an entirely nascent-but-growing industry.

    These people were performing and important social and economic function. They were exchanging ideas, opinions, information. Where is the industry going?  Which are the hot companies this month? Who got hired where?  Where is so-and-so working now?

    What makes the Valley so cool - and so hard to replicate

    Every so often I hear that a local government is looking to replicate Silicon Valley.  High-speed optical networking. Tax breaks. Cheap commercial real estate. Attracting VCs.   What they Really need to do is host more parties.

    I say that tongue-in-cheek, but in reality it is the social networking component to the valley that makes all of the difference - the talent fluidity, the idea fluidity.  And that's really hard to legislate.

    Back in 1990's, AnnaLee Saxenian authored a prescient book, "Regional Advantage: Culture and Competition in Silicon Valley and Route 128."  The thesis is why two regions, both with many great schools, and many great corporations, evolved so differently.  In the prologue she wrote
    "The valley is very fast-moving and start-ups have to move fast. The whole culture of the Valley is one of change. We laugh about how often people change jobs. The joke is that you can change jobs and not change parking lots. There's a culture associated with that which says that moving is okay, that rapid change is the norm, that it's not considered negative on your resume... So you have this culture of rapid decisions, rapid movement, rapid changes, which is exactly the environment that you find yourself in as a start-up.
    Indeed, the cultures are different. The companies that formed around 128 (and I worked at one for 4 years in the '80s) were large and mostly completely vertically-integrated.  They were their own technology ecosystem, designing hardware, coding software, and assembling things themselves. And once you were hired, being a "lifer" was what most people did.   Meanwhile in the Valley, each company was componentized. Your partners were across the street. Your new hires worked at your partners... or maybe even your competitors. (True, California cannot enforce noncompete clauses, whereas Massachusetts does)

    All this adds up to an incredibly driven, creative, localized and fluid talent pool. Doing what people do best - schmoozing, swapping ideas, creating the future.
    Posted by at 2 comments:
    Labels: ,

    Tuesday, February 21, 2012

    IT-as-a-Service: Resources and Pointers

    IT-as-a-Service. Possibly another buzzword that may reach the peak of hype and then fade away in a millisecond. But even if so, this new concept will persist by some other name.

    Unlike IaaS, PaaS, SaaS, etc. ITaaS isn't a technology model. It's a new mind-set and approach to delivering enterprise IT services - where IT operates and competes for "business" as-would a service provider.And, like a commercial SP entity, it actually tries to encourage service consumption, rather than discourage it.

    This is in contrast to the stereotypical IT department that runs a bottleneck help-desk, issues locked-down equipment, charges the enterprise with a flat operating "tax", and is organized along purely technological lines.

    This "new" IT thinks more like a business - not that it needs to deliver a profit per se, but that it's more customer (line-of-business) focused, understands its costs, and "competes" against alternatives that users now have outside the enterprise (a.ka. Shadow IT). The new IT thinks Self-Service provisioning, Choice Computing (BYoD), monitors variable costs and unit consumption, and is organized to deliver services, not technologies. 

    ITaaS Model - Components and Pointers

    When IT begins thinking about becoming an "internal" service provider, there are 3 conceptual models that need to shift - (a) how services are generated and consumed, (b) a shift in how technology is leveraged, and (c) a change in operations and organization.

    Where can you find the most authoritative information about ITaaS?  I'd like to believe that EMC is leading the way with, among others, our own IT Department.  But resources abound on the topic...  I've begun to collect useful pointers from many points of view. And, from time-to-time, I will update this list with additional pointers, insights and success stories.

    ITaaS Overview
    Don't take my word for it. Others are beginning to write about ITaaS, its benefits, its positive impact on business agility, and about where to start your plan:
    IT Leadership
    If the enterprise is to undertake the ITaaS transformation, then senior IT leadership - as well as line-of-business leadership - has to be 100% behind it.  But the traditional CIO-as-Technologist model necessarily has to give way to CIO-as-businessman.  Running IT like a business means complementing technology with knowledge of how the enterprise's core business runs, IT financial management, IT organizational transformation, and even IT services-supply-chain management.

    IT Financial Transparency
    A key characteristic of transformation to ITaaS is the ability to understand the costs of delivering individual services, and thus be able to allocate and price them appropriately. Once variable IT costs are understood, measured, and shared with the business, IT has a higher stake in ensuring that services are delivered and priced efficiently and transparently.  And, by creating and assigning per-unit costs to services, IT can more efficiently match supply with consumption. Ultimately, improved IT financial transparency ensures more accurate decisions made both by IT and lines of business.
     Reinventing the IT Consumption Model
    In a transformed IT environment, the goal is to drive an increase in service consumption by the business. The shift also includes IT viewing its business model as a profit center rather than a cost center, with more of a commercial mindset.  To accomplish this, IT must change its operating model to simplify access to services, and to facilitate delivery of services – regardless of whether they are internally or externally generated. And, if IT is to become this ‘broker’ of services, it must develop a customer-centric supply-chain approach to delivery of services that the business demands, no matter their origin.

    Transforming Organization, Roles, Skills
    As the IT organization transforms itself to become more like a service provider to the enterprise, internal IT skills, roles, and even the entire organizational structure will necessarily change. Traditional technology specialization areas will make way for more general, services-centric roles. Skills will shift from specialists who craft technology stacks, to generalists who manage holistic systems that produce user-centric services.  Service product managers (both in-bound and out-bound) will also be in demand to maintain value alignment with line of business users.
    Success Stories
    IT-as-a-Service is still new, and most companies who have embarked on the transformation are still mid-stream. But a few are making their journeys public. Here are a few I've found.
    Analyst Perspectives
    Industry analysts are beginning to adopt the "Run IT like a business" perspective, and even the ITaaS label. Here are some pointers (some reports require Analyst website logins/subscriptions)


      Posted by at 2 comments:
      Labels: , ,

      Monday, February 13, 2012

      My IT is Ready. My People Are Not.

      By now you may know that I buy into the enterprise approach where cloud computing and its operational model are enablers for something bigger: The transformation to IT-as-a-Service. That's when internal IT acts, operates and competes like an internal SP.

      But I've come to realize that such transformation is more of a skills and operational model shift than a technology shift (although technology is clearly an enabler). As was often mentioned at a recent CIO event I attended, "My technology is ready. My people are not" 

      This topic of talent, training and organization appears to be consistently glossed-over. But recently at EMC and with customers, I'm seeing the beginnings of a sea change in the industry.


      Early Indicators of Change

      My first sense of the need for change arose from a 2010 conversation with an analyst about using converged infrastructure. They made the observation that "to use a converged infrastructure to its fullest, you need a converged organization."  This got me thinking: when the boundaries between compute, storage and network begin to blur, so must the boundaries between traditional IT organizations. The advent of cloud computing (more specifically, IaaS and PaaS) highlights a similar issue.

      Slowly, the 'unit of management' in the data center is shifting away from the isolated IT component, and is moving toward the application service.  IT management is having to treat the infrastructure more holistically, more horizontally. With more automated provisioning, virtualization and elastic capacity, orchestration across all components is becoming a requirement. Similarly, the skills of IT Ops personnel need to be more integrated - individuals simply need more cross-specialization knowledge and training to ensure that services are delivered quickly and efficiently.

      My next sense of the breadth of change came when I joined EMC and saw what Sanjay Mirchandani (EMC's CIO) saw in transformation to IT-as-a-Service.  To him, cloud computing and infrastructure automation were merely enablers for IT organizations to become more of strategic enablers to the business, more of internal service bureaus, driving consumption of their services, and operating overall like a business itself. Toward this end, EMC-IT has been documenting its change now for a number of years. But, because this shift involved so much more than just technology, I began to realize that the skills and roles within "traditional" IT would become woefully inadequate in a number of years.

      In the transformed world of IT (and IT-as-a-Service) the organization needs to develop horizontal groups to manage broad platform architecture and management, as well as roles that 'broker' those services to the lines of business (service managers) and maintain alignment to the needs of users. There may even be roles that loosely align with 'sales and marketing' that help drive awareness and consumption of IT services - concepts that will surely seem foreign to any CIO.

      What's Next... and How To Prepare Your People

      When recently reading an article in Forbes I observed that others were noticing these needs (and opportunities) for IT skills of the future as well:
      "As a result of the shift to cloud, there is growing demand for professionals and managers that are more focused on business development than they are in application development. There will be greater opportunities for enterprise architects, and some offshoots will include cloud architects, cloud capacity planners, cloud service managers and business solutions consultants. Jobs being created may not always bear the term “cloud” in their titles, but cloud will form the core of their job descriptions.
      What caught me here was not all new roles will be technology-focused. Service managers and business solutions consultants - for example - will be client-focused. Think of that: Roles that exist solely to understand the line-of-business operating units, develop services (not technologies) to meet their needs, and help drive demand for those services. (See related post on A Marketing Lesson for IT )

      So if you believe the tenet that skills and roles will play a more critical role in the transformation of IT, then where does one start?  Here at EMC, I was pleasantly surprised to see that our own HR department is right on top of the problem.  Our IT HR director in fact was taking the lead in describing what change lay ahead.  In a recent communication of hers she noted
      "...we are in the midst of a process to build on our current skills to foster the ITaaS model, taking deliberate steps to fill any gaps by providing plenty of training and support. Last quarter, we assessed the skill level of each employee against a set of key competencies needed for their particular functions under ITaaS to determine what added training/support they need. In some cases, the skills we’ll be drawing on are very different than the typical IT focus. New areas of required expertise include things like product management and development, communications, marketing, negotiation and influence skills, and financial acumen.
      "Our prime focus in Human Resources is employee development and preparing employees for what our future IT organization will look like. This is part of our life-long learning and development program....  We are going to help [our people] gain new skills and provide them with the tools they need. A key part of the journey will involve learning to think like a service provider rather than the internal agency we are used to being. It will require a shift in our cultural mindset from discouraging business IT demand to actually selling and marketing IT services.
      Wow. IT shifting its mindset to selling and marketing...  That our HR group is this intimately involved in the transformation of the business I found exciting and enlightening. And as I looked further, I realized that we (and others, I'm sure) are beginning to take steps to prepare for this transition. EMC for example, recently Announced Cloud Architect and Data Center Architect certification tracks including expert-level tracks (and Brochure PDF )

      In many ways, existing hosting and public cloud service providers already know much of this; their business depends on good customer service, meeting requirements, and driving consumption (demand). Enterprise IT now simply has to learn from them - and adopt this new mindset.

      Where to start? Begin thinking about how the following job skills and orgs might play a role in advancing IT's strategic importance in the organization
      • New Consumption models - There is a need to simplify and drive how services are consumed. This is a 2-part process. Technically, there is the migration toward self-service catalogs. But there is also the marketing aspect that drives awareness and demand for these services - IT needs to think "commercially" - attract demand, and make access to it as frictionless as possible.
      • New Operational models: Architecturally and operationally, more of IT will become "horizontally" oriented, along the lines of business services and along the lines of customers. New orchestration-style tools and processes will be needed - implying the need for skills to manage them properly.
      • New Marketing roles: Yes, marketing.  To drive consumption, IT has to expose more of its services to more potential business users... and do so competitively. That is, IT needs not only to think about what the business needs, but what their next alternative (outside of the IT organization) is, and ensure that IT is the vendor of choice.  More about marketing IT here.
      • New Finance roles:  Implementing IT cost transparency doesn’t just require financial data, though that is what most people think about when they hear the term. It also depends on information about hosting, applications, infrastructure, licensing and maintenance, and other things that let us associate costs to services. Clearly these roles are net-new to IT
      • New Business relationships: Partnering with, and aligning to the business is about more than just providing basic IT services and helpdesk. It means working alongside line-of-business planning to understand what drives their top-line business, what is at the core of their competitive advantage, and what business agility can add to these. Dedicated roles - which are steeped in both business and technology skills - need to be acquired and assigned to individual business units.
      Other Resources:


        Posted by at 1 comment:
        Labels: ,

        Sunday, February 5, 2012

        The Growth of Data Growth: My Digital Contrail

        Although I'm not a Big Data aficionado, I'd recently been struck by a few statistics from the IDC Digital Universe study:  It is estimated that in 2011, 1.8 Zettabytes of information was created, 75% of which comes from individuals. And by 2015, that number may grow to 7.9 Zettabytes.

        So, just where does all of this data come from?  And the more I thought about it, the more I realized that each one of us (in developed countries, at least) kicks-off a near constant massive stream of data that gets stored somewhere, even if only transiently.

        In our seemingly innocuous day, we surely generate more data than we consume, some of which is captured by others, and only some which we might be privileged enough to retain.

        Thus I sat down to think about a typical "digital contrail" that I might generate. While I really can't quantify exactly how much stored data is created from each transaction, simply ball-parking the numbers would seem to support the Digital Universe claims.  And, if any of you reading this can help me quantify some of this, I'd be happy to append this post. Thanks in advance.

        Sources of my "Digital Contrail"
        • I make a cell phone call:  Phone location tracking data (i.e. from towers) created and tracked by the carrier; phone log files; data created and stored by multiple mobile apps and their own hosting infrastructures 
        • I browse the web: Site tracking; clickstream storage; site analytics; Email storage, including replication on devices as well as replication in geographical-mirrored data centers.
        • Driving my car: Location-based tracking by RFID tags at toll booths; unique instrumentation data such from as OnStar systems
        • Go to the bank: data streams initiated and stored from a simple ATM withdrawals; security analysis of banking transaction patterns; audit and verification trails for individual transactions; mirrored/backed-up data within the bank's data center
        • Go to the store: data streams initiated and stored from a simple credit card transaction; product inventory changes; buying patterns stored and allocated to individual affinity discount programs
        • Browse an online store: All of the above, plus clickstream storage and analysis
        • Plan some travel: Airline reservations & pricing systems such as SABRE ticketing; airline tracking databases; TSA flyer database updates & analysis
        • At my home: electricity usage via smart metering data collection
        • Using entertainment: Uploaded Photography and Video; sales pattern data and DRM data 
        • Go to the doctor’s office: Medical imaging , EMR data, reports, other records
        • Somewhere in the background: With everything I do, there are surely security systems,  kicking-off background data processes and analytics DB’s
        • Also somewhere the background: Every service is sourced from a data center, where all data (including device data) is surely replicated and backed-up, including log files.
        I finally thought through a simple habit I had, and how much storage space it spawned: I would receive an email with a PDF attachment - and carefully file the email in a folder while copying the PDF also into a separate folder related to the project. So I'd have 2 copies of the file on my PC, not to mention another copy on the Exchange server as well as one on the PC backup server file - 4 in all (assuming no deduplication system was in place). And if the email had been sent to others besides me... you get the point.  I was suddenly sensitized to data growth on a personal scale.

        So, now I've convinced myself that "the data's out there". I've created scads of data in the past 24 hour stint... and fortunately (or unfortunately) it's all recorded in different repositories. But now I begin to wonder - what *if* some of these structured and unstructured data streams were re-constructed, mashed-up and analyzed?  That bit makes me both nervous (from a privacy and security perspective) and excited (from a Big Data and personalization perspective).  More later when I stop to think about that one.



          Posted by at No comments:
          Labels:

          Thursday, January 19, 2012

          It Will Be a Data-Centric (Cloudy) World

          (Or, Where Tomorrow's Clouds Will Form)

          Move Mohammed to the mountain, or the mountain to Mohammed? 

          In the context of data, applications and cloud computing, this question takes on a new perspective - and the role of Mohammed and the Mountain may soon reverse.


          In the traditional application-centric (and static infrastructure) world, the Application is the immovable "mountain". Like a magnet, the app is permanently-located, attracting to it local data stores and peripheral support apps.  Administrators dote around it like worker bees around the queen.

          But for some uses and applications this may all change - altering with it the how-and-where compute and community clouds form.

          Observation #1: Apps are becoming mobile

          With increased use of a virtualization layer, migration tools, shared storage, fat network pipes, and virtual I/O and switching, we are all now realizing that where the executable application code resides is becoming far less important. Everything is just data - and can be moved/migrated. In mature virtual environments, VMs typically move between servers because of maintenance windows, because of capacity adjustments, etc.  But when VMs move move between physical data centers (separated by many miles or more) there is often a data movement as well. But there's no denying that the application is becoming more mobile.

          We also have the emergence of large data arrays and analytics appliances that embed internal servers that speed queries and analysis. VM's typically run on top of these servers being migrated in-and-out of the arrays as workloads and queries change. Hang on to this visual... we'll come back to it later.

          Observation #2: Big Data is becoming bigger

          When we start talking about hundreds of Terabytes - or even Petabytes - of structured/unstructured "big data", moving that data becomes increasingly physically difficult. Where it's generated is generally where it stays. Think about financial stock exchanges; retail data warehouses; medical imaging; geologic or climatological data. These stores are now becoming big and immovable.

          So, enterprises are now locating these data stores within critical data centers - within which they are co-locating the applications that require frequent access to that data. Sometimes that proximity is sufficient, and sometimes the analytics may even move within the array. But any way you look at it, those data stores are becoming the center of attention, around-which the applications now congregate.

          An interesting shift. But wait, there's more...
          ·     
          Outcome: Where Tomorrow's Clouds Might Form

          So let's expand this model from enterprise data centers to public clouds. Or even to "community clouds".

          Take the example of financial exchange data - NYSE’s Capital Markets Community Platform about which I blogged last year.  Here we have a special-purpose, "community cloud" - optimized for financial institutions, wherein they can locate trading and analytics applications. (Imagine a 3-person hedge-fund startup needing infrastructure). Operationally it's fit-for-purpose, with a high performance, low-latency compute backbone, with a common security/compliance envelope. But it's got another trait: At its core is a historic data warehouse of every tick for every trade. Now that's big data.

          If you think of it, the NYSE data store has become the "Mountain" around which applications (supplied by the cloud tenants) now congregate.They run their algorithms and analytics against the local data store. The data within the community cloud has become the anchor, the magnet. The apps are moved to be near the data.... not the other way around.  Remember that data array that had embedded VMs? Well, think of this model as that array on steroids.

          So, what might this mean if you want to build a differentiated cloud computing resource - say, targeting a specific industry vertical?  It says to me that the world will shift to a data-centric model. Focus on amassing and maintaining massive high-value data, all (presumably) requiring a similar security/compliance model. And then build a business by allowing tenants access by co-locating their applications in the same cloud as the data resides.

          I could see this transforming both the cloud service provider ecosystem, as well as entire industry groups. Consider new Cloud Service Provider models:  What if NOAA formed the Weather and Atmospherics Community Platform? If healthcare companies created federated Medical Records Community Platforms? If the USGS formed the World Geologic Community Platform? If other brokerages created equivalent capital markets platforms? 

          Cloud computing is shifting lots of conceptual IT models these days. But while you're considering what Cloud makes possible for applications, spend some time wondering what data makes possible for the Cloud.

          Other References

            Posted by at 2 comments:
            Labels: , ,

            Monday, January 2, 2012

            My Cloudy Nerdy Playlist for 2012

            Political movements have their anthems. Starbucks has its background music brand. But I thought it might be fun to collate a playlist that specifically appeals to the techie/cloudy group.

            So here goes. Some of these are pretty funny... others leave a bit to be desired.  And if you know of more (or have favorites of your own) send them my way and I'll add them...

            Songs about Clouds

            Other (mostly funny) songs about IT
            Techy Poetry
              Even MORE Links...
              Posted by at No comments:
              Labels: ,
              Subscribe to: Posts (Atom)