Agile Methods

A Search for REST Frameworks for Exploring WOA Patterns -- And Current Speaking Schedule

Dion Hinchcliffe — Sun, 10 Sep 2006 12:45:00 GMT

The common patterns for assembling large systems out of smaller systems have been well documented in recent years, with one of the very best examples being Gregor Hohpe's Enterprise Integration Patterns. But as Web sites grow in size and scale -- my favorite quote here being Dare Obasanjo's "My Website is Bigger Than Your Enterprise" -- we're increasingly seeing a push both in the direction of "Web-ness" as well as a preference for radical simplicity in messaging patterns (REST instead of SOAP/WS-*). That doesn't necessarily mean that the grids, server farms, and new 384 processor core "superservers" are neccessarily getting any easier to take advantage of or manage. But we are finally learning how to build systems that are broadly consumable by the limitless variety of the Web's populace, as well as scale on-demand and perform better across the board.

Service-oriented architecture has become the end-game for designing and integrating systems-of-systems within the enterprise. This model as most people know it usually involves SOAP with one or more of the many, many WS-* protocols added to it. However, as we look at bigger and more complex integration scenarios on the Web, this model doesn't seem to be quite as popular or effective. The advent of Rich Internet Applications (RIAs) such as Ajax, Flex, OpenLaszlo, Avalon, and so on, doesn't favor it too much either, since REST tends to be the most consumable Web service type in RIAs by a good margin. And the browser, the preferred application delivery model these days, doesn't speak SOAP at all without serious help, never mind WS-*. I've often quoted the statistic that 80% of all software development should be based on SOA by 2008. If true, this means we clearly have important trends in opposition. What is a software architect to do?

The issue seems to be one that has plagued us in the software business for decades; namely a seemingly incessant urge to predefine all of our solutions before we're even sure what the problem is. The principles of agile software development tell us that we won't understand the problem until we're done developing the software, and that to attempt to design too much up front will result in software (and by extension, standards) that have the wrong features. What this means for the heavyweight standards business is left for the reader, but I'll quote here Groove's George Moromisato when he says that "the chance of any standard being adopted is inversely propotion to its complexity. " Consequently SOAP, and some of the other Web service protocols developed early on in the history the Web, seems to suffer from this very problem.

Enter the growing trend of Web-Oriented Architecture (WOA), an approach to Web services architecture that is, if anything, underprescribed, loose, and therefore very situationally adaptable. WOA has a tolerant, resilient, but steel-reinforced core based on the fundamental protocols of the Web (HTTP, XML, and URI.) And the big advantage of all this is the following: almost certainly the highest levels of interoperability, scalability, and consumability. The latter refers to the fact that REST-based architecture can be used in many more places and many in more ways than traditional SOAs can.

The issue with REST and WOA however continues to be a lack of good description for the lay programmer. Advocates of traditional SOA can point to many well-defined standards documents, descriptions, and reference models. REST and WOA don't have many of those things, each being more of a "style" instead of a standard. Yet, REST and WOA don't go farther than their underlying standards allow and have a malleability and range of motion that over-specified approaches can't possibly match. Unfortunately, Roy Fieldings classic description of REST in Chapter 5 of his dissertion remains impenetrable to the vast majority of developers, though I'm personally working on a couple of projects that might help resolve that.

A quest for REST frameworks to explore WOA

While Nick Gall excellently brought us the vision and concept of WOA, I'm currently focusing on articulating the concept of WOA client patterns, or the best ways to build well-behaved clients in Web-Oriented Architectures. I talk frequently to folks building large systems out on the Web and I've been noticing recurring problems and solutions when they build well-defined, resilient clients that work effectively in the chaotic, unpredictable environment of the Web. From this, I've tried carefully to collect some of the core patterns together and place them into a handful of high level buckets which I've coined WOA/Client and published recently in The Architecture Journal.

Since then, I've slowly begun describing the interior of each pattern. As I do this, I've been attempting to build capable RESTful services that I can create simple, useful reference models for others to see these patterns in action. Surprisingly, I'm still finding that while tooling for REST continues to improve, there still isn't as much out there as I'd like to see. And though fortunately, REST generally doesn't require much to implement, I'm especially looking for that Ruby on Rails-style of simplicity for connecting the service directly to a data-driven source of information. And as you will see below, that's fortunately where we're starting to see some particularly good news.

Some REST frameworks and approaches

While there are dozens of well-known methods for connecting distributed systems together, protocols based on HTTP will be the ones that stand the test of time. And since HTTP is the fundamental protocol of the Web, those protocols most closely aligned with its essential nature will likely be the most successful. So WOA describes architectures based on HTTP, and WOA/Client is the best practices on the client side for participating in a WOA (disclaimer: this latter part is the result of my personal research and is nothing 'official.') In that vein, I've been building REST services to host on this site to demonstrate the power and efficacy of simple, contract-based Web clients that pull data from multiple sources, weave them together into useful forms, and then distribute updates and changes back to their source services. This will be a very common model of service consumption in the near future, particulary in the browser, and the world of mashups is the most clear example of this in the 'wild.'

While I host this site on .NET primarily because SOAP based services are so easy to implement in .NET, it's clear that the open source and Java communities have considerably richer support for REST today. The most advanced framework for REST that I've been able to find is Jerome Louvel's deep and powerful Java-based Restlet framework, available under the CDDL license. It provides full support for REST-based resources, representations, connectors, filters, and routers and also offers excellent examples of REST architectures in the well-written and detailed tutorial.

At the other end of the spectrum is REST-art, a simple and admittedly early-release of a basic, straightforward REST-enabling framework. Currently listed at the top of Sourceforge's project search for REST, REST-art was created by Thibault Ducray, who says "REST-art is a Java library (jar) based on the REST principles (services uses URIs, XML data, http verbs GET, POST, PUT, DELETE). It gives the ability to develop Web Services without using huge (and sometime commercial) SOAP frameworks."

Another sophisticated RESTful framework is the SourceForge project Cognitive Web, which has seemingly advanced capabilities that go well beyond the basics of REST and WOA. The home page of the Cognitive Web says that the project is a, "framework for building, deploying and managing well-described REST-ful Web services, including REST-ful Web Services realizations for RSS, XML Topic Maps, Structured Arguments, and Workflow."

But one of the most interesting REST frameworks I've seen is sqlREST, an easy-to-use framework that allows a set of relational database tables to be exposed as REST-compliant Web services. While not a production-ready release, it's the concept of sqlREST that is the most interesting; an automatic mapping of database information in relational format to an interactive view of the same data in XML/HTTP format, REST-style. As data-driven Web applications become more and more common, having an easy way to turn your Web application into an open platform will be a useful (and essential) technique for growth. sqlREST makes this possible with little development effort.

Finally, Matt Biddulph did a terrific job last November writing up how Ruby on Rails can be used to quickly and effectively build REST services with Rails using the built-in REXML facility. Ruby on Rails is famously known for reducing abstraction impedance across the board, and Matt's tutorial reminds us again how useful Rails is for making routine, yet normally complicated, programming tasks easy, at least once you learn the Rails way.

Speaking Schedule on Web 2.0, SOA, and WOA

I cover all this material, and a lot more, in my talks on the next generation of Web, IT, and business. While a few of my presentations are very technical and will cover the intricacies of SOA/Web 2.0 co-evolution or how to build scalable Ajax applications using WOA, I also talk about how the latest trends and service models on the Web are changing the way we work, live, and otherwise interact with each other. If you're going to any of these events, please be sure to stop me and introduce yourself.

"21st Century Professional and Civil Society" Conference - Dallas, Texas - Saturday, September 16th, 2006 - I'll be speaking on the role of Web 2.0 trends and technologies and how they will affect professional and civic life in the 21st century. As well as covering the basic of Web 2.0 and moderating a panel, I'll be covering the innovative application of Web 2.0 platforms such as blogs and wikis to document important events like the genocide in Darfur to the great story of the use of emergent collaboration on the Web to locate one hundred thousand Hurricane Katrina survivors in a short few days.
Interop New York 2006 - New York City, New York - Tuesday, September 19th, 2006 - I'll be giving one of my usual Web 2.0 in the enterprise presentations, which are invariably pretty popular. The session description says: "From a buzzword last year, to the thematic rallying cry of 700+ startups this year, and now apparently heading for mainstream adoption, Web 2.0 is the topic du jour for the consumers and businesses both. This session attempts to demystify the trend with practical description and examples along with a succinct explanation of Web 2.0. The second half explores how Web 2.0 is being used by companies in various ways to improve internal productivity and innovation, not to mention how to involve customers and reinvent the concept of customer service and marketing."
The New New Internet - Tyson's Corner, Virginia - Tuesday, September 20th, 2006 - I'll be moderating some panels for this Web 2.0 conference which I'm helping organize in the Washington, DC area. We have Michael Arrington's TechCrunch, Harvard Business School's Andrew McAfee (of Enterprise 2.0 fame), Michael Platt from Microsoft's Architecture Strategy Group and a bunch of other great speakers. I do hope to see you there, it'll be the first major Web 2.0 in the enterprise conference on the East Coast and at the tony Ritz-Carton right in middle of the Silicon Beltway.
CIPS (Canadian Information Processing Society) Winnipeg ‘06/’07 Kick-Off Dinner - Winnipeg, Canada - Wednesday, September 21st, 2006 - I'll be the evening's speaker and the subject will be Social Computing, a topic about how the widespread use of Web 2.0 technologies tends to create long term changes in the societies that apply them, particularly a shift from push to pull-based models for management and self-organization. Fascinating stuff. This is a topic I've covered quite a bit in recent months including most notably on ZDNet, where it was Slashdotted, and at the SOA Web Services Journal.
AjaxWorld University Bootcamp - Santa Clara, California - Monday, October 2nd, 2006 - I'll be leading up an all-day, high-intensity bootcamp on how to build "addictive" Ajax applications using the latest frameworks and design patterns. We'll cover Dojo, Prototype, Script.aculo.us, as well as learn how to design irresistible user interfaces with concepts inspired by Kathy Sierra's Addictive User Experiences material. It will be the fastest way to learn Ajax best practices you can find anywhere. I hope to see you there.
AjaxWorld Conference and Expo - Santa Clara, California - Tue.-Wed, October 3rd-4th, 2006 - Not only will the top 70 or so folks in Ajax and RIAs be there but a mini-conference on Ruby on Rails and Flex will be on site too. The list of well-known folks speaking is amazing and combined with the bootcamp above, should be the biggest Ajax event of the year. Disclaimer: I'm the conference chair of AjaxWorld this year. I'll also be speaking at a session titled Leveraging the Web 2.0 Movement. Description: "The emerging Web 2.0 movement is emphasizing the same things as SOA -- which is rapidly emerging as the best practice for building services, reusing IT assets, and providing open access to data and functionality. Web 2.0 and SOA are in fact beginning to converge. Web 2.0 describes the Internet as a "Global SOA" while at the same time laying out practices for building lighter weight SOAs based on REST and RSS. Furthermore, Web 2.0 and online services are expected to become the dominant application model with behemoths like Microsoft and Google fighting for dominance. More interestingly, this phenomenon is actually happening now and helping businesses deliver more value to their customers every day. This fact-filled session will discuss the latest trends citing industry sources, case studies, and other well-known examples."
Office 2.0 Conference - San Francisco, California - Tuesday, October 12th, 2006 - Ismael Ghalimi has organized a terrific conference with a leading cast of speakers on the topic of the next generation of Web-based business software. Ismael calls this concept Office 2.0 and it's rife with important SaaS, Ajax, and Web-based implications for the software business. I'll be moderating a panel on Tuesday morning titled Making Office 2.0 Enterprise Ready.

That's it for now, expect some more WOA/Client patterns soon, particularly a rigorous exploration of contract checking with a sample REST service, instead of the SOAP one we did last time. I'll also be updating my publishing schedule soon; we have some exciting material on SOA/Web 2.0 converence as well as REST coming in a couple of months as well.

Agile and Lean Approaches to Enterprise Architecture, SOA, and Web 2.0

Dion Hinchcliffe — Tue, 16 May 2006 07:28:00 GMT

The face of software development seems to be changing before our very eyes. As an increasing number of our tools, languages, architectures, and even ways of thinking continue to gel around the software model of the Web, we seem to be finally approaching a kind of detente in the software industry.

That's not to say that there isn't a long way to go but the rate of expected change in software systems continues to go up and it's driving radical changes in our processes and technologies. Web 2.0, advent of the next generation of the Web, has heralded many of these changes and I've written in the past about how many of the new best practices that are helping tiny development shops quickly turn out and maintain systems that are used by tens or hundreds of thousands of people. The rise of game-changing dynamic languages and applications stacks are further pushing things to extremes with Ruby on Rails being the pre-eminent example. A terrific article about this phenomenon here on IBM's Developerworks.

This highlights an important trend: That Web devleopment technologies and techniques are increasingly bleeding into the enterprise as a model for how things should be done. In my ZDNet article on enterprise process 'angioplasty' I highlighted how agile and lean techniques are actually middle-of-the-road approaches compared to what's happening out in the Darwinian Petri dish of the Web.

On the architecture side, the collision of Web 2.0 and SOA is starting to happen along with the push towards using Web-scale architecture models like Web-Oriented Architecture. These new lightweight, low-barrier ways of thinking about software architecture are changing our conceptions of how to connect systems and components together. Furthermore, these architectural models directly support agile and lean processes with high change rates because they come from a world where rapid software change and evolution is the norm.

Along these lines, alternative methods like lean are showing how we can drop massive amounts of waste out of existing techniques and processes without necessarily throwing them all away.

As part of this, Sphere of Influence, the company I work for as CTO, will be co-sponsoring a chapter of the Agile Process Leadership Network this week in Northern Virginia. On hand will be Marry Poppendieck, the world leader in lean processes. We invite you to come and hear Mary keynote on agile contracts on May 18th in Tyson's Corner. DETAILS HERE

Note: I'm still trying to post here once a month but my Web 2.0 Blog, ZDNet Enterprise Web 2.0 blog, and my Editor-in-Chief positions at the Web 2.0 Journal and AjaxWorld have kept me busy indeed (besides my day job). I also have an article coming out in the June issue of the Microsoft Architecture Journal that is the culmination of the public work on loosely-coupled Web services that I posted here last year. I'll do a summary and review of the material next month on this blog.

Update on SOA, Web 2.0, and Agile Methods

Dion Hinchcliffe — Sat, 11 Mar 2006 03:14:00 GMT

It's been an extremely busy six months and I haven't posted here much lately with my blogs at the SOA Web Services Journal and Enterprise Web 2.0 on ZDNet demanding most of my blogging output. Since this is my professional site however, I'm going to try to commit to posting here once a month or so. As far as I'm what I'm doing these days, I do keep my current public appearance calendar up to date on my SOA Web Services Journal blog. I'm also involved in a bunch of exciting events coming up including next week's DC 2.0, SYS-CON's jam-packed Ajax Seminar (with an incredible line-up including father of Ajax Jesse James Garrett and Ruby on Rails' David Heinemeier Hansson), and my talk at the SOA Web Services Edge in New York in June to name just a few.

I've been been spending a lot of time working on Web 2.0 related concepts lately, including its convergence with Service-Oriented Architecture. My article on this was the year-end cover story of the SOA Web Services Journal last year. This article, discussing Web 2.0 as the Global SOA, got a considerable amount of attention and ultimately got me invited to SPARK, a private software architecture conference right before MIX 06 and hosted by Microsoft. SPARK will explore the latest emerging ideas in software architecture and how to reconcile the interrelated forces and constraints of SOA, Web 2.0, and Software as a Service (SaaS.)

Another fascinating topic for me has been the SOA/Client, a way of delivering zero-footprint SOA clients using plain old browser technologies. I discussed this recently on ZDNet and got lots of interesting responses including from InterArbor's Dana Gardner who invited me to do a podcast on the subject, which we did late last month.

Finally, I often get quizzical looks when I tie agile methods together with Web 2.0 and even SOA. But they are actually highly interrelated, particularly on the Web 2.0 side. I wrote about this recently in Giving enterprise software practices an 'angioplasty' and I think it's one of the more important aspects of building software using highly responsive techniques that focus on delivering working value to the end-user. And based on real field-observations, the leading edge of best practices in Web 2.0 development really are changing the way the software is getting built.

In any case, I've written a ton of material about all the exciting things going on in software these days, here's a run down of the more popular bits recently:

How Simple Sharing Extensions Will Change the Web

The SOA Tolerance Continuum

10 Issues Facing Web 2.0 Going Into 2006

Making The Writeable Web A Responsible Place

A Timeless Way of Building Software

Web 2.0 and the Enterprise

Balkanizing RSS and the risks to the information ecosystem

Some Predictions for the Coming World of Mashups

There's lots more on the blogs, be sure to check them out. I also have some more podcasts and TV shows coming up, and I'll post them here as time permits.

The Web 2.0/SOA Power Panel - December 7th, 2005

Post-Modern Enterprise Architecture: Service-Oriented, Agile, Decentralized

Dion Hinchcliffe — Sun, 17 Apr 2005 08:42:00 GMT

Are enterprise architecture teams hopelessly outdated and anachronistic in an agile and service-oriented IT world? The verdict is in, and the answer is yes. It's time to pull the rip-cord, shut them down, and send the folks into the trenches. Like centralized planning in a Stalinist regime, most folks now see that heavyweight enterprise architecture was horrendously hide-bound, autocratically inflexible, almost comically out-of-sync with the business, and worse, far too slow to ever respond to the real needs of business units and their unvariably unique requirements. It's perfectly clear that enterprise architecture won't survive in its present form much longer; sucking dollars out of the IT budget and taking the best talent while producing little in the form of actual value.

However, we also can't forget that we have to practice enterprise architecture in a world where businesses themselves are now often defined primarily by loosely coupled federations with other businesses via automated processes that use service-oriented techniques to form their value chains (i.e. like Amazon and its 50,000 partners linked together via web services, ADP and their payroll services, etc.). In this new service-oriented world, companies now realize they can seriously cut costs and increase overall value by building new composite systems out of their own services, and that of other companies, instead of building or buying yet another application silo. Service-oriented architecture achieves the long-sought after goal of enterprise application integration, provides real reuse of value, and solves myraid other classical IT problems in one fell swoop. Does this emergent focus on service-orientation and enterprise architecture give EA teams one final new lease on life as the enablers of the service-oriented enterprise. Or does it finally make them completely irrelevant?

Figure 1: The future of enterprise architecture: Agile, Non-Centralized, Accountable, Service-Based

Certainly, some folks, like Scott Ambler, have recently been advocating a more agile version of enterprise architecture. In most realizations of agile enterprise architecture, there is a considerably reduced central planning aspect, which was formerly concerned mostly with authoring master models of the enterprise architecture and creating piles of write-only documentation. Agile EA has more to do with working with people, coaching, and getting out in the trenches and mentoring developers on practical architecture techniques and developing architecture skills. Stephen Cohen, for example, has been doing an admirable job of describing modern EA recently, but you can clearly see the us vs. them mentality with his safari approach.

In the end, the common complaint with enterprise architecture that most stakeholders outside the EA team have is that EA is notoriously unresponsive to specific needs, too difficult and expensive to comply with, is needlessly complex, and too general purpose. EA also frequently prescribed inapproriate, outdated or excessively bleeding edge technologies. There is also the realization that the growing adoption of agile processes may not be compatible with traditional EA and might never support the use of formalized enterprise architecture. Finally, many folks in the organization just don't have the skills or tools to support what EA groups usually pump out: ivory tower architectures that were never validated in the field other than a pilot or two.

In any case, the answer is that service-orientation won't give EA teams one more chance, in fact, I think service-orientation will likely finish them off. Coupled with the widespread perception that central enterprise architecture groups have been a rampant failure and don't deliver – and with central control essentially being eliminated by pervasive service-orientation – and we'll probably watch EA evaporate as going concern before our very eyes. The only wrinkle comes in with the ever-growing demand that companies build open service-oriented organizations. A demand that is pushed by increased competition, globalization, and need for more efficient (re)use of resources. Over the next few years, to fuel increasing collaboration, supply-chain integration, demand for web-facing delivery of services, we will find that many organizations will be forced to deliver their enterprise architecture in the form of easily consumable, non-visual, and open services. By looking at some of the emerging models for these efforts, I think we can begin to see the outlines of what enterprise architecture organizations will look like for the rest of this decade. We have to remember the classic IT antipattern: Not invented here. All too often our industry comes up a priori solutions and either ignores or fails to recognize existing, successful models unless they fit preconcieved notions. In other words, if enterprise architecture, 90's-style, just doesn't work, then let's look at what does.

The best enterprise architecture I've come across hasn't come out of any ivory tower group. Invariably, the best architecture I see comes naturally from self-organizing thought leaders in an organization that seek each other out and collaborate on common solutions to their problems. Rather than the us vs. them mentality of old-world enterprise architecture, there is only an us mentality. Instead of proscribed standards, designs, technologies, and tools there is real consensus and immediate buy-in. Sure, there are political camps in any organization that don't like playing with other folks but the inexorable drive towards service-orientation in the enterprise makes it so that people have to play nice together like never before. Or they can't get access to or provide the services they must in order to survive. CIOs, chief architects, and other technical officers must foster this collaboration and seek out the folks that are 1) technically competent, 2) have ground truth in what the business actually does, and 3) have great people skills. The truth is that enterprise architecture has always been much more about people and business than about technology. Making enterprise architecture happen today is about building agile networks of people in your organization that have the big picture, the local political power, real understanding of technology, and have a stake in the final outcome.

Technorati: Enterprise Architecture, Computers and Internet, Agile Methods

Domain-Driven Design: Model Driven Architecture Done Right?

Dion Hinchcliffe — Sun, 20 Mar 2005 18:36:00 GMT

My colleagues and I have been making an in-depth examination of Eric Evans' intriguing, and deep, treatise on the Domain-Driven Design approach over the last few weeks, as have others recently. A lot of progress has been made in the last couple of years in trying to achieve the software development industry’s equivalent of a solo ascent of Mount Everest: models of software that can easily be turned into robustly designed working systems at the push of a button.

Since the release of his book last year, Evans has been getting an increasing amount of attention for his ideas, which represent what appears to be a surprisingly hard-headed, practical, and realistic view of the model-driven architecture vision. Evans may just be prescient enough and smart enough to be describing the next major vision of software development. Even more interesting, in our practice, as well as in the software development community, we are starting actually see projects seriously trying to implement forms of Domain-Driven Design.

Evans has an extensive background in delivering large systems and he understands the difficult problems involved with it. Most model-based development methods involve combinations of new tools and processes that don't fit well with the way that programmers actually work, or are too pure and don't address the messy realities involved in actual software development. Evans seems to understand all of this and more and paints a coherent, interlocking vision of how real model-driven development should work.

The Elements of Eric Evans’ Domain-Driven Design

Citing a hard-won set of largely old-but-new-again design guidelines, plus some fascinating new ones, Evans articulates a vision he calls domain-driven design, or DDD. It's worth reviewing these briefly because I am fascinated with the potential that Evans offers: software that meets requirements because there is one artifact everyone refers to, the model. In addition, Evans emphasizes uncompromising use of ubiquitous language of the domain, and a neat approach to maturing designs to be elegant and not-overengineered, which he calls supple design.

But describing this doesn’t convey the quiet self-assurance about these ideas that Evans consistently maintains throughout his story. On top of this, Evans is also acquainted with one of the key values of agile software development: delivering working software of high value to the customer. Awareness of this is present throughout his writing and it seems to add the edge of realism that is essential, in my mind, to making abstract techniques effective in the real world.

Domain-Driven Design is much more than just modeling however, it’s a carefully thought out set of techniques and approaches that this blog entry can’t hope to describe in detail. But I can say that they are a compelling and fascinating read. Like all genuinely new approaches, Evans ideas are sure to stir controversy, but their feeling of “rightness” is undeniable to anyone who has been in the software architecture trenches for any amount of time.

I encourage you to spend an hour or two study his ideas the next time you have a chance, you may very well be preparing yourself for the next major movement in enterprise architecture and software engineering.

Technorati: Enterprise Architecture, Computers and Internet, Programming

Agile Development Needs Good Memes

Dion Hinchcliffe — Sun, 20 Feb 2005 09:30:00 GMT

I get asked fairly often to describe what agile development is. My consulting company's most popular seminar series is about agile/lean processes. The folks that usually attend invariably have little or no understanding of agile when they first come in.

This always comes as a surprise to me since it's been 10 years since XP began its ascent into the mainstream. And it's clear that the agile approach still isn't understood by many folks in IT, despite the fact that they are almost always quite curious about it! I view this primarily as an indirect failure of the agilists; no one has yet hit on the right set of memes to distribute this much needed evangelism.

So it's great when I come across artifacts that accurately capture the spirit of agile concepts in a single snapshot. And today, while surfing the blogosphere, I came across Sam Newman's excellent Agile Release Process post that includes a diagram that is sure to ultimately cause a bit of heated discussion. But it also does a very commendable job of conveying the essence of most agile processes. I especially like that it specifically emphasizes the delivery of value to the customer. I've included a slightly modified version below with notes that I hope will address misperceptions that the diagram can convey to the uninitiated.

Figure 1: The Agile Release Process - High Level Flow

A lot of folks have already chimed in on other blogs and said this should actually be called the Agile Development Process, but that's not entirely accurate. The development process cuts across this for sure, but there are important elements of agile development mechanics that aren't conveyed here. Instead, this diagram clearly shows how end-user value is taken in the form of stories (requirements) and transformed step-by-step into a release that delivers the functionality to the stakeholders, all in the framework of (hopefully) rapid iterations.

My two quibbles with the original release diagram was that:

1) It didn't make clear that end-user functionality is made available for early examination by users, either in the development shop, or my personal favorite, deployed to a group of early-adopters. These are trusted users that don't mind working with beta releases and are representative of the user community. This should happen during the deploy step (in the above diagram), where newly implemented stories are forked off to the early-adopters and the QA process at the same time.

And 2) it didn't describe the time-boxed gating process at the end of the iteration. This ensures stories that don't cut the mustard, are too buggy, or determined not to be needed due to low return for the investment, are cut or moved to the next iteration. Each iteration should start at a predetermined point in time. And since the product should always build and run, time-boxing is not only possible to do rigorously with agile methods, it's essential to keep things moving along.

Finally, since I do worry that workable agile development memes are in short supply, here's a brief recap of the values that drive it, in what I hope is catchy sound-bite form:

Agile Development Values:

- Satisfy the customer, through early and continuous delivery of valuable software.

- Welcome changes, especially in requirements, up to the last responsible moment.

- Deliver working software frequently and maintain a working build at all times.

- Stakeholders and developers must work side-by-side to achieve a common end.

- Construct the project around motivated people and give them what they need to succeed.

- Face-to-face communication within the project is to be preferred above all other forms.

- Working software is the measure of progress. Working stories that are in production is the only success criteria.

- Sustainable but continuous pace. Schedules and staffing must allow a constant pace indefinitely.

- Architecture, requirements, and design should come from self-organizing groups on the team.

- At intervals, the team must rigously reassess itself to tune and improve its behavior.

Note: Freely adapted, with thanks, from the Agile Manifesto principles.

Lean development, which has the elimination of waste as its key driving factor, is also trying to make inroads in almost the same space as agile methods. Lean development also seems to be suffering from a similar lack of effective viral marketing. Although I have seen numerous projects, especially large formal ones, frequently try to introduce elements of agile processes into their hide-bound playbooks, I still think we're an epiphany or two away from the aha! required to get agile/lean methods truly established in the traditional IT world.

Technorati: Agile Methods, Computers and Internet, Programming

Agile Methods Supported by a New Wave of Project Management Tools

Dion Hinchcliffe — Sun, 30 Jan 2005 08:21:00 GMT

These days we see a lot of development shops looking to add agile methods to their daily development efforts. I've also personally observed larger, more hide-bound organizations, with (what they consider to be) proven methods, talk seriously about adding agile elements to their processes. There are a variety of reasons for this; sometimes it's just related to the excitement and buzz around agile, but I've also consistently detected a serious note of pragmatism when agile is considered.

Agile methods seem to offer what so many developers and managers want: less bureaucracy, an unrelenting focus on results, close involvement with the customer, and frequent course corrections that eliminate waste. With the possible exception of customer involvement, it's interesting to see that these goals seem to be shared both at the technical and management level. Furthermore, there seems to be a real sense these days that agile methods actually work.

However, actual adoption of agile methods poses challenges for project managers as they struggle with everything from figuring out how to track earned value against their schedules to finding project management tools that were designed with agile methods in mind.

This last part, lack of PM tools for agile methods, finally seems to be addressed. Now, personally, open source seems to have turned a major in corner in 2004 and I now routinely search SourceForge for the software I need before looking for a commercial product. A recent search turned up a fascinating list of new open source tools for APM (agile project managers as coined by Sanjiv Augustine.)

XPlanner and Project Planning and Tracking System (PPTS) are two emerging open source APM tools that seem to have a lot of promise. XPlanner is the most generic, most advanced, and most popular of the two. But they both offer compelling features to APMs who don't find Microsoft Project very useful when dealing with user stories, emergent requirements, self-organizing teams, and so on.

Both tools are listed as Production-Stable by SourceForge and offer a range of features specifically tailored to the needs of agile projects. And XPlanner and PPTS each have features to support iteration tracking, metrics (a very tough issue with agile projects), reporting, time tracking, and resource management. XPlanner seems to have the edge overall however and it even includes wiki-style word linking, export of data to traditional tools like MS Project/Excel, web service interfaces to integrate with build/test harnesses, and more.

If you want yet another alternative, JS Greenwood has been posting on his blog recently about XP StoryStudio, which is also free APM tool, though not hosted on SourceForge. It is a portal-based tool with general support for agile methods and XP in particular.

The nice thing is that all these tools are free, are built on open standards based technologies, and can be downloaded and compared with each other before use and even during use, something that is often not genuinely possible with commercial tools.

For organizations that prefer commercial products, there are also a few emerging APM tools in the commercial space including TargetProcess, which is entering it's second iteration and has integrated bug-tracking.

It's a much better time to be an APM than it was a few years ago; increased acceptance of agile methods and much more robust tools support only makes getting your project done on time that much easier.