Dion Hinchcliffe's Blog - Musings and Ruminations on Building Great Systems

As I highlighted recently on ZDNet, 48% of CIOs will be looking to actually start using their SOAs to connect to external partners this year. Unfortunately, we've been building landscapes of Web services for quite a few years now and for many, the tipping point for SOA adoption seems as elusive as ever. While trying to understand why this is, one common explanation I offer is that the A in SOA is often missing. When you ask server-side developers in a given organization what they are developing, they usually say Web services. When you talk to architects in the same organization, they usually say they are building SOAs.

This highlights a discernible and common disconnect between the tactical reality of delivering applications on deadline and the strategic goal of delivering carefully crafted services that are generic enough to be reusable, tested with all the technology stacks in the organization, and meet all other criteria of having well architected enterprise services.

This is where the World Wide Web continues to teach us effective techniques for service consumption and adoption. Amazon has tens of thousands of consumers of its various and sundry Web services that range from e-commerce to the compelling S3 storage platform. And they're making money doing it as well. The rise of mashups too has shown how easily that simple, composable services can be made into workable browser-based composite applications. All of these has given us the conception of Web-Oriented Architecture (WOA), which I've been writing about here on this blog for a while now. This is using the basic Web formats and protocols such as HTTP, XML, REST, and JSON as the "Unix Pipe of the Web" -- to quote a colorful phrase of Ray Ozzie's -- as the fundamental glue between systems. This allows widgets, Ajax applications, and mashups to be wired together so quickly it can almost be done in real-time with the latest tools.

Finally, we have Web 2.0 (most recent formal definition here), a way of leveraging the fundamental strengths of the Web to turn applications into platforms, exploit the potency of network effects, and otherwise take advantage of networks as true software platforms in their own right. However, the rules of networked platforms are very different than the ones we are familiar with on the computing-oriented platforms we know traditionally such as operating systems like Windows, Linux, and the application stacks that sit upon them. As it turns out, being a fundamentally communication-oriented platform, our networks impose a whole new set of rules for success that we are just now finally beginning to understand well. Surprisingly, among these, is the recent realization embodied by Reed's Law, which states that the instrinsic value of a network is much, much higher if the network is used in a social manner. Thus, in some important way, social networks tend to more fully leverage the value of networked applications and services.

Steeped in formal standards, byzantine product stacks, and software engineering principles, these are strange ideas for SOA architects to accept, much less embrace. Then there is the matter of usefully applying these ideas to create an effective service-oriented architecture that can be easily consumed by internal and external customers, and indeed, is preferred to use instead of reinventing the wheel. For the truth is, if the services most of us are building now were so much better than letting development projects just build it themselves, they would be beating a path to the nearest internal SOA representative to save themselves the cost and time. And while that is happening in some cases, SOA adoption studies and anecdotal evidence tells us it's just not happening enough.

So, in the spirit of lessons learned and to incorporate our most recent understanding of what works and what doesn't, I thought I'd put together a suitably provocative list of what SOA architects should be seriously thinking about in 2007:

Eleven Emerging Ideas for SOA Architects in 2007

1. Making services consumable in the browser. Increasingly, the common Web browser is the place where meaningful service integration is taking place. Because of this, building services that aren't easily consumable in the browser can be a death knell for the service because that puts its consumers in the business of building and maintaining adapters or using a Javascript SOAP stack -- if you can find one -- before the service can be used and measurable work accomplished. Ultimately, non-browser friendliness greatly reduces possible consumption scenarios for SOAs as we'll see in some of the points below. This doesn't' mean throw away your WS-* services. But it does mean you should automatically offer a REST or JSON version as well.
2. Considering syndication over "service-izing." The browser is an important consumption point but so too are the growing syndication ecosystems of which the blogosphere is the largest example. More and more tools are willing to consume RSS and ATOM, often in preference to SOAP, including the forthcoming version of Vista where syndication-friendliness is a core value. Carefully consider offering your services in RSS form or even ATOM, which has a two-way REST model. This will further increase consumption scenarios and therefore adoption. Content syndication is growing into a very potent force inside and outside the enterprise and plugging an SOA -- strategically or tactically -- into one of these ecosystems has terrific upside potential. Not every SOA service can or should be converted to a syndication model, but if you aren't considering this option with each service you create, you should be; there are tens of millions of RSS feeds available today, starting from zero in the beginning of 2003. How many SOAP services presently exist worldwide? Only a tiny, tiny fraction of this and there are good reasons for it.
3. Deeply embracing URI addressability. Of all the things in this list, this might be the most important one. The hyperlink is the fundamental unit of thought on the Web and it should be in your service designs and (hopefully granular) schemas as well. Giving each discrete piece of information, every service, and all content a globally addressable URI instantly gives a service, and the data it carries back and forth across its interface, access to countless new consumption and reuse scenarios. The most important of these is the leveraging of network effects via -- often social -- link propagation along with the ability to make all URI addressed information potentially crawlable, thereby making it transparent via search. The possibility of letting people find your service via an intranet or Web search engine because of the great content it has might seem a little odd at first but then again, that's what makes things work so well on the Web. You can learn about URIs on Wikipedia, and they can be a SOA's best friend.
4. Using Ajax as the face of your SOA. This point emphasizes yet more service-consumption in the browser. Why? Because the browser model, with our newest high-speed corporate networks, fast desktops, and latest browsers, has finally becoming a very capable way of distributing software and associated updates. No admin rights are required to run an Ajax application in the browser, no plug-ins must be distributed, and users find the experiences they offer compelling. And the Ajax technique requires Web services into order function because an Ajax app is nothing more than a Web page brought to life by Javascript. And if the page upon which the Ajax application resides ever reloads, the application is then forced to quit immediately. Because of this, Ajax software has a major dependence on quality Web services while running, Ajax and SOAs can go hand in hand and potentially complement each other very well.
   
   Finally, I'll point out, as I did in item #1, one key barrier to this unified vision of browser front-end and services back-end is that many SOA services today are just not Ajax consumable. Worse, virtually none are easily consumable by emerging Flash platforms for RIAs such as Flex or OpenLaszlo without a lot of work (or cost), because these platforms have very limited XML processing capabilities such as poor namespace support. This highlights a growing need to sort out the tolerance continuums that are probably too shallow in many enterprises. Note: It's still not easy to develop Ajax apps yet, and so it's worth reading my Seven Things Every Software Project Needs to Know About Ajax for more info about the challenges of applying this fast growing Web services-powered browser application model.
5. Monetizing Your SOA. On the SOA projects I've been on, many of those who own the systems being opened up as services don't like the results in the short term: more customer service, additional bandwidth and hardware to support unpredictable external use, more testing, and so on. Figuring out ways to meter usage, institute chargebacks, and even charging outright fees to external trading partners and customers allows the necessary negative feedback to discourage irresponsible or profligate use of services. This works well on the Web and the most successful APIs online are metered in some way.
6. Enable users as service consumers. This also cuts across some of the items above but is best exemplified by the software mashup phenomenon, which describes a method of quickly combining two or more sources of content into a new high-value application. For just the same reason that we have a PC on everyone's desk at work is the reason why almost everyone should be consuming the services in your SOA. Only most users don't have access to the Microsoft Office equivalent of a tool that allows them to mashup or wire together the services you have been producing over the years. The good news is that products specifically geared at enabling the consumption of SOAs by end-users are emerging including IBM's QEDWiki, JackBe's Presto, and many others. In a few years, it's likely that end-users will be one of the largest direct consumers of your services, particularly via syndication. Enable it and encourage it; it's just another way to make your SOA invaluable to the business and generally popular as well.
7. Virtualization, fast scaling, and on-demand architectures. All of the things driving down the economics of software hosting will allow your SOA to scale up to the Web. Many enterprises view their SOAs and enterprise systems as big, but not compared to the scale of the Web, particularly if provisioning is unmediated (thousands of informal users of your APIs.) Fast adoption is one of the worst nightmares for an SOA that is not well capacity planned and scaled. Just like operations has become a core competency of SaaS and Web 2.0 sites, so too is it in the highly spiky usage model of on-demand services where a successful network effect can cripple your availability and response times.
8. Offering an SOA as visual services via widgets. The rise of widgets on the Web, making it easy for anyone to put a piece of functionality on their Web page, was a big item in 2006. Widgets also have access to back-end infrastructure (i.e. an SOA) and are snippets of Javascript or Flash badges that allow little bits of data-driven functionality such as stock tickers, corporate news, and other information to reside in any Web page and be fed by back-end services. This is another effective way to put a "face" on an SOA and get it used in many different completely unexpected ways. And because just about anyone can drop a widget onto a Web page, this further unleashes all users -- instead of a few IT staffers -- to help deploy the functionality and data held within an SOA to the far reaches of an enterprise. Additional note: If you're not sure this will be important, check out the hundreds of services available as widgets already on WidgetBox.
9. Considering JSON as a service option. XML is NOT very fast as I've written about here before, particularly if there is lots of numeric information in the network payload. JSON, the Javascript Object Notation, has risen through the ranks quickly in the last year as a highly compact way to send information on the wire to a Web application. Even the co-inventor of XML, the venerable Tim Bray, has acknowledged the many valid use cases of JSON in networked applications. JSON may not be for you as an architect but your Rich Internet Application developers may very well feel the desire to place pins in your effigy as they try to figure out how to get your sophisticated XML payloads to parse quickly enough -- or in many cases -- to parse at all using the emerging Flash platforms. JSON is fast, compact, and is supremely easy to consume in the browser via Javascript's eval() function.
10. Encouraging and discovering emergent solutions. Like many are discovering out on the Web, being directly connected to your customers is a completely different proposition than shipping software on a CD. Many SOA practitioners are well aware of this of course, but even the most battle-hardened SOA practitioner would have to go aways to be aware of how extreme it's become online. I've come to describe this tight process of co-evolution via realtime feedback, harnessing user contributions, and becoming a platform that others actually build upon something known as Product Development 2.0. It's not hard to see what happens when users are tightly coupled into the systems that they use and begin using this connection to shape their needs. Even if your corporate SOA doesn't work this way today, it can be made to fairly easily with online metrics and monitoring, though like many SOA issues, governance and control soon become significant issues. Just remember, sites like Flickr deploy changes to production every 30 minutes while monitoring usage and making more changes in almost continuous real-time feedback loops. Other sites are literally letting their users shape the services available from the application itself such as Google with its Web Gadgets offering.
    
    Going further, the concept of Enterprise 2.0 is the front line where much of this particular change will begin taking place in the enterprise, with freeform, social, emergent tools like blogs and wikis that are so general purpose they can used in an almost limitless number of ways. Make no mistake; emergent application platforms are not an edge case trend and are already taking place in your organization with things like the guerrilla deployment of wikis that I'm increasingly seeing in the field. Understanding how an SOA fits into all this (as IBM has, which has labeled their new end-user mashup tool QEDWiki, "the Face of SOA") will be essential for fully leveraging a service-oriented architecture in this environment.
11. Leveraging the Global SOA. More and more I'm coming across impressive applications that marry the datasets contained within enterprises with the incredibly rich landscape of information out on the Web. And they are primarily impressive because of the data brought in from the Web. I've espoused the concept of the Global SOA, most notably in a cover story for the SOA Web Services Journal, that describes the Web as the richest set of services currently available to anyone, inside or outside the enterprise. It simply no longer makes sense to have an SOA that does not have access to the Global SOA on the Web where hundreds of high-value APIs are available and millions of lesser ones in the form of RSS and ATOM. Infoworld's David Linthicum had some good comments about the convergence of Web 2.0 and the Global SOA, and here is my own exposition with a good diagram that shows the overlap. The challenges around the governance issues of figuring out how to bring in external services safely and provisioning them for use as part of your enterprise SOA. Those who do this successfully can potentially garner an even great uptake on SOA usage as the number of high-value services available internally ramps up quickly.