Dion Hinchcliffe's Blog - Musings and Ruminations on Building Great Systems

The common patterns for assembling large systems out of smaller systems have been well documented in recent years, with one of the very best examples being Gregor Hohpe's Enterprise Integration Patterns. But as Web sites grow in size and scale -- my favorite quote here being Dare Obasanjo's "My Website is Bigger Than Your Enterprise" -- we're increasingly seeing a push both in the direction of "Web-ness" as well as a preference for radical simplicity in messaging patterns (REST instead of SOAP/WS-*). That doesn't necessarily mean that the grids, server farms, and new 384 processor core "superservers" are neccessarily getting any easier to take advantage of or manage. But we are finally learning how to build systems that are broadly consumable by the limitless variety of the Web's populace, as well as scale on-demand and perform better across the board.

Service-oriented architecture has become the end-game for designing and integrating systems-of-systems within the enterprise. This model as most people know it usually involves SOAP with one or more of the many, many WS-* protocols added to it. However, as we look at bigger and more complex integration scenarios on the Web, this model doesn't seem to be quite as popular or effective. The advent of Rich Internet Applications (RIAs) such as Ajax, Flex, OpenLaszlo, Avalon, and so on, doesn't favor it too much either, since REST tends to be the most consumable Web service type in RIAs by a good margin. And the browser, the preferred application delivery model these days, doesn't speak SOAP at all without serious help, never mind WS-*. I've often quoted the statistic that 80% of all software development should be based on SOA by 2008. If true, this means we clearly have important trends in opposition. What is a software architect to do?

The issue seems to be one that has plagued us in the software business for decades; namely a seemingly incessant urge to predefine all of our solutions before we're even sure what the problem is. The principles of agile software development tell us that we won't understand the problem until we're done developing the software, and that to attempt to design too much up front will result in software (and by extension, standards) that have the wrong features. What this means for the heavyweight standards business is left for the reader, but I'll quote here Groove's George Moromisato when he says that "the chance of any standard being adopted is inversely propotion to its complexity. " Consequently SOAP, and some of the other Web service protocols developed early on in the history the Web, seems to suffer from this very problem.

Enter the growing trend of Web-Oriented Architecture (WOA), an approach to Web services architecture that is, if anything, underprescribed, loose, and therefore very situationally adaptable. WOA has a tolerant, resilient, but steel-reinforced core based on the fundamental protocols of the Web (HTTP, XML, and URI.) And the big advantage of all this is the following: almost certainly the highest levels of interoperability, scalability, and consumability. The latter refers to the fact that REST-based architecture can be used in many more places and many in more ways than traditional SOAs can.

The issue with REST and WOA however continues to be a lack of good description for the lay programmer. Advocates of traditional SOA can point to many well-defined standards documents, descriptions, and reference models. REST and WOA don't have many of those things, each being more of a "style" instead of a standard. Yet, REST and WOA don't go farther than their underlying standards allow and have a malleability and range of motion that over-specified approaches can't possibly match. Unfortunately, Roy Fieldings classic description of REST in Chapter 5 of his dissertion remains impenetrable to the vast majority of developers, though I'm personally working on a couple of projects that might help resolve that.

A quest for REST frameworks to explore WOA

While Nick Gall excellently brought us the vision and concept of WOA, I'm currently focusing on articulating the concept of WOA client patterns, or the best ways to build well-behaved clients in Web-Oriented Architectures. I talk frequently to folks building large systems out on the Web and I've been noticing recurring problems and solutions when they build well-defined, resilient clients that work effectively in the chaotic, unpredictable environment of the Web. From this, I've tried carefully to collect some of the core patterns together and place them into a handful of high level buckets which I've coined WOA/Client and published recently in The Architecture Journal.

Since then, I've slowly begun describing the interior of each pattern. As I do this, I've been attempting to build capable RESTful services that I can create simple, useful reference models for others to see these patterns in action. Surprisingly, I'm still finding that while tooling for REST continues to improve, there still isn't as much out there as I'd like to see. And though fortunately, REST generally doesn't require much to implement, I'm especially looking for that Ruby on Rails-style of simplicity for connecting the service directly to a data-driven source of information. And as you will see below, that's fortunately where we're starting to see some particularly good news.

Some REST frameworks and approaches

While I host this site on .NET primarily because SOAP based services are so easy to implement in .NET, it's clear that the open source and Java communities have considerably richer support for REST today. The most advanced framework for REST that I've been able to find is Jerome Louvel's deep and powerful Java-based Restlet framework, available under the CDDL license. It provides full support for REST-based resources, representations, connectors, filters, and routers and also offers excellent examples of REST architectures in the well-written and detailed tutorial.

At the other end of the spectrum is REST-art, a simple and admittedly early-release of a basic, straightforward REST-enabling framework. Currently listed at the top of Sourceforge's project search for REST, REST-art was created by Thibault Ducray, who says "REST-art is a Java library (jar) based on the REST principles (services uses URIs, XML data, http verbs GET, POST, PUT, DELETE). It gives the ability to develop Web Services without using huge (and sometime commercial) SOAP frameworks."

Another sophisticated RESTful framework is the SourceForge project Cognitive Web, which has seemingly advanced capabilities that go well beyond the basics of REST and WOA. The home page of the Cognitive Web says that the project is a, "framework for building, deploying and managing well-described REST-ful Web services, including REST-ful Web Services realizations for RSS, XML Topic Maps, Structured Arguments, and Workflow."

But one of the most interesting REST frameworks I've seen is sqlREST, an easy-to-use framework that allows a set of relational database tables to be exposed as REST-compliant Web services. While not a production-ready release, it's the concept of sqlREST that is the most interesting; an automatic mapping of database information in relational format to an interactive view of the same data in XML/HTTP format, REST-style. As data-driven Web applications become more and more common, having an easy way to turn your Web application into an open platform will be a useful (and essential) technique for growth. sqlREST makes this possible with little development effort.

Finally, Matt Biddulph did a terrific job last November writing up how Ruby on Rails can be used to quickly and effectively build REST services with Rails using the built-in REXML facility. Ruby on Rails is famously known for reducing abstraction impedance across the board, and Matt's tutorial reminds us again how useful Rails is for making routine, yet normally complicated, programming tasks easy, at least once you learn the Rails way.

Speaking Schedule on Web 2.0, SOA, and WOA

I cover all this material, and a lot more, in my talks on the next generation of Web, IT, and business. While a few of my presentations are very technical and will cover the intricacies of SOA/Web 2.0 co-evolution or how to build scalable Ajax applications using WOA, I also talk about how the latest trends and service models on the Web are changing the way we work, live, and otherwise interact with each other. If you're going to any of these events, please be sure to stop me and introduce yourself.

• "21st Century Professional and Civil Society" Conference - Dallas, Texas - Saturday, September 16th, 2006 - I'll be speaking on the role of Web 2.0 trends and technologies and how they will affect professional and civic life in the 21st century. As well as covering the basic of Web 2.0 and moderating a panel, I'll be covering the innovative application of Web 2.0 platforms such as blogs and wikis to document important events like the genocide in Darfur to the great story of the use of emergent collaboration on the Web to locate one hundred thousand Hurricane Katrina survivors in a short few days.
• Interop New York 2006 - New York City, New York - Tuesday, September 19th, 2006 - I'll be giving one of my usual Web 2.0 in the enterprise presentations, which are invariably pretty popular. The session description says: "From a buzzword last year, to the thematic rallying cry of 700+ startups this year, and now apparently heading for mainstream adoption, Web 2.0 is the topic du jour for the consumers and businesses both. This session attempts to demystify the trend with practical description and examples along with a succinct explanation of Web 2.0. The second half explores how Web 2.0 is being used by companies in various ways to improve internal productivity and innovation, not to mention how to involve customers and reinvent the concept of customer service and marketing."
• The New New Internet - Tyson's Corner, Virginia - Tuesday, September 20th, 2006 - I'll be moderating some panels for this Web 2.0 conference which I'm helping organize in the Washington, DC area. We have Michael Arrington's TechCrunch, Harvard Business School's Andrew McAfee (of Enterprise 2.0 fame), Michael Platt from Microsoft's Architecture Strategy Group and a bunch of other great speakers. I do hope to see you there, it'll be the first major Web 2.0 in the enterprise conference on the East Coast and at the tony Ritz-Carton right in middle of the Silicon Beltway.
• CIPS (Canadian Information Processing Society) Winnipeg ‘06/’07 Kick-Off Dinner - Winnipeg, Canada - Wednesday, September 21st, 2006 - I'll be the evening's speaker and the subject will be Social Computing, a topic about how the widespread use of Web 2.0 technologies tends to create long term changes in the societies that apply them, particularly a shift from push to pull-based models for management and self-organization. Fascinating stuff. This is a topic I've covered quite a bit in recent months including most notably on ZDNet, where it was Slashdotted, and at the SOA Web Services Journal.
• AjaxWorld University Bootcamp - Santa Clara, California - Monday, October 2nd, 2006 - I'll be leading up an all-day, high-intensity bootcamp on how to build "addictive" Ajax applications using the latest frameworks and design patterns. We'll cover Dojo, Prototype, Script.aculo.us, as well as learn how to design irresistible user interfaces with concepts inspired by Kathy Sierra's Addictive User Experiences material. It will be the fastest way to learn Ajax best practices you can find anywhere. I hope to see you there.
• AjaxWorld Conference and Expo - Santa Clara, California - Tue.-Wed, October 3rd-4th, 2006 - Not only will the top 70 or so folks in Ajax and RIAs be there but a mini-conference on Ruby on Rails and Flex will be on site too. The list of well-known folks speaking is amazing and combined with the bootcamp above, should be the biggest Ajax event of the year. Disclaimer: I'm the conference chair of AjaxWorld this year. I'll also be speaking at a session titled Leveraging the Web 2.0 Movement. Description: "The emerging Web 2.0 movement is emphasizing the same things as SOA -- which is rapidly emerging as the best practice for building services, reusing IT assets, and providing open access to data and functionality. Web 2.0 and SOA are in fact beginning to converge. Web 2.0 describes the Internet as a "Global SOA" while at the same time laying out practices for building lighter weight SOAs based on REST and RSS. Furthermore, Web 2.0 and online services are expected to become the dominant application model with behemoths like Microsoft and Google fighting for dominance. More interestingly, this phenomenon is actually happening now and helping businesses deliver more value to their customers every day. This fact-filled session will discuss the latest trends citing industry sources, case studies, and other well-known examples."
• Office 2.0 Conference - San Francisco, California - Tuesday, October 12th, 2006 - Ismael Ghalimi has organized a terrific conference with a leading cast of speakers on the topic of the next generation of Web-based business software. Ismael calls this concept Office 2.0 and it's rife with important SaaS, Ajax, and Web-based implications for the software business. I'll be moderating a panel on Tuesday morning titled Making Office 2.0 Enterprise Ready.

That's it for now, expect some more WOA/Client patterns soon, particularly a rigorous exploration of contract checking with a sample REST service, instead of the SOAP one we did last time. I'll also be updating my publishing schedule soon; we have some exciting material on SOA/Web 2.0 converence as well as REST coming in a couple of months as well.