Sunday, August 26, 2012

The Rise of the Cloud Service Bus

In my recent GigaOm guest blog, I posited that the advent of hosted cloud services (particularly PaaS and SaaS) will slowly morph the role of the CIO into that of an IT Supply Chain Manager.

Technologically, I believe this move to a "Buy-and-Integrate" mentality (vs. a "Build-Everything" mentality) will open the door to a new class of products to assist with services integration. And, if you agree that the importance of leveraging external services will be elevated for the CIO, then I believe a significant enabling technology will be a rebirth of the need for a robust "service integration bus".

Why?  As I mention in my blog, enterprises integrating external services require:
 - Identity and access management for each provider
 - Data compliance, legal and regulatory audit access across each provider
 - Security compliance systems
 - Provisioning, including capacity forecasting
 - Performance (e.g. SLA) monitoring
 - Cost and budget tracking (i.e. for billing, showback and/or chargeback)
 - Disaster Recovery / Redundant service sources where needed
Some would call the above integration functions "Glue Logic." Indeed in the past, many of these functions were hand-integrated across the few external services that were leveraged, and custom-engineered into each internally engineered stack. But time is changing the model. With more turnkey services  sourced from cloud (IaaS/PaaS/SaaS) providers, the need for a more efficient integration function will escalate. Integration will need to be standardized and replicable, scalable and responsive to the business' needs.
You may recall one component integration approach has been the Enterprise Service Bus, primarily associated with SOA leveraging SOAP protocols. This Integration Bus was originally to orchestrate access and workflow between component services within the enterprise. (By the way, Microsoft offers a great overview of ESBs - albeit BizTalk centric - here on MSDN).

I believe that the "2.0" Integration Bus will be one which brokers higher-level services generated from external, public cloud providers - not just internal component services. And it will use more generalized interactions than SOAP, since the providers and their environments will be less standardized.

To this end, there are some great current/upcoming thoughts suggested by Mike Ponta of the notion of a "Cloud ESB", and can't wait to hear more.  A quick survey of the market also yielded what looks to be potentially promising integration technology coming out of Mulesoft called their CloudHub.

Conceptually, the "mashed-up" service was the 1.0 version of this integration concept. But as enterprise IT begins to regularly tap and integrate multiple external services, the 2.x integration busses will need a more structured, standardized and rapid approach to integration.

I can't wait to see what else the market will generate. Stay tuned.

Other resources/Blogs:


William Louth said...

Naturally I have a different view of what this bus will service and carry.

Though I do agree with service supply chain management as a means to manage the complexity of IT in the cloud across multiple providers (direct and indirect) but I disagree that it is a hub-spoke kinda approach as you seem to hint at with brokers.

CloudWays said...
This comment has been removed by the author.
Tony McCune said...

Very interesting idea. I was thinking about the exact same thing and came across your blog post after searching to see if anyone was talking about a Cloud Service Bus. Do you know of anyone doing this?

Mulesoft is building a lot of cloud infrastructure for service bus but it's living on top of the IaaS/PaaS providers echosystems.


Ken Oestreich said...

Tony -

It's time I did more research on this. Question is, should it be a full As-a-Service style product, or should it be a more traditional product that runs on-site. I could see 2 use cases...

(1) where the service runs on-site for enterprise IT, and orchestrates/coordinates interactions between external cloud services, and
(2) where the service runs in a multi-tenant infrastructure within a cloud service provider. That provider would broker/integrate external services, aggregate them, and deliver a fully-integrated set of IT services to clients.

Let me know if you find anything out there!

Ryan said...

I happened onto this looking for a "traditional" service bus that is cloud software aware, but you guys might want to check out Dell Boomi -

It is a kind of service bus that handles both on-premise to cloud and cloud to cloud integration.