Technorati Profile Blog Flux Local IT Transformation with SOA
>

Friday, August 7, 2009

SOA as the Solution Architecture

It’s time to move on to the more detailed Level II architecture. There are myriads of variations in Level II architectures and most have probably been tried in the past. Every few years or so, you wake up to find new technologies that promise to solve all your IT ailments: high level languages, structure programming, fourth generation programming, CASE Tools, Object Oriented Programming and an ever-expanding list of software development methodologies. However, given that this is the dawn of a new millennium and that we have over six decades of commercial computing under our belts, I would suggest that not planning to adopt SOA (Service oriented Architecture) for a new system would be like planning to build a house using straw and mud. Yes, there might be reasons for preferring to build a primitive hut (as a part of a movie set, perhaps?), but in general I’d rather build a house using modern construction materials. Wouldn’t you?

Defining a high level architecture these days is all about adapting SOA precepts to support the chosen architecture. Even though SOA can aid in simplifying the definition of the Level II architecture, you and your team still have to make the key decisions related to SOA-specific choices. Remember, this is the stage where the architecture moves from abstract to more pragmatic levels. With Level II you will still be high enough up so that you won’t need to worry about negotiating the ground level, but at thirty-thousand feet you have to keep a watch out for weather patterns. It is in this stage that you can better discern the horizon and where true innovation can be applied with unique solutions that can ultimately serve as key success differentiators in the ultimate deliverable.

The first thing to keep in mind is that SOA is about simplifying the business process automation and not about introducing technology for technology’s sake.

A friend of mine related this anecdote after attending a ceremony celebrating the activation of the first automated phone exchange in a small town in Mexico. As the mayor gave a glowing discourse on how the town was finally “entering the 20th century” and people would now be able to automatically place calls simply by dialing the numbers on their phone, an elderly woman sitting next to him complained, “Automatic? This ain’t automatic! Automatic was when I lifted the receiver and asked Maria, the switchboard lady, to connect me to my daughter!”

She was right. From a user’s perspective, all that matters is the ability to articulate a need in a simple way, and then have the need satisfied by the appropriate service. Alas, this woman will have to wait many years before she once again receives the same level of service she received from Maria. Maybe if she (or her grandkids) programmed her cell phone, she could once again connect to her daughter simply by speaking her name. However, if she ever wanted a connection by voicing something like, “connect me to someone who can fix my stove”, this would require the emergence of software that can truly understand natural-language and take intelligent action (don’t get me started with the so-called Interactive Voice recognition systems of today!).

The beauty of thinking in terms of services is that you can avoid getting bogged down by how the service is provided. The manner in which the service is provided should be, in the end, immaterial to the person requesting the service. What it does matter is having a well defined interface to the service be well defined. If you order a meal in a language that is not understood by the waiter, then you can be assured the request either won’t be met or that you will get served a dish full of proteins and fats of unknown origins (something like this actually happened to me while in Hong Kong, after wrongly assuming I had ordered chicken!)

How then do we define SOA? Simply stated, SOA deals with the ability to ask a system to do something (typically a coarse-grained business or system process) without having to tell it HOW to do it. Think about it, when you go to a restaurant and order a dish from the menu in the correct language, you are applying SOA principles. SOA is about abstracting the request so that the business need can be posed directly to the system via the use of a proper interface request.

In fact, SOA is nothing new. From my perspective, Service Oriented Architecture was actually invented more than ten thousand years ago with the advent of modern civilization. The SOA inventor is unknown, but most assuredly was some lazy bum trying his best (let’s face it folks . . . it was a he!) to avoid work and pass on responsibilities to others. Specialization resulted in people becoming more competent in their chores and the framework of rules and processes needed to facilitate this delineation of responsibilities became part of the societal laws we have today. Back then you had a merchant asking a scribe to log a transaction, or a king requesting a priest to plead his case to the gods, or a man of commerce paying someone to carry his produce. Agriculture, war, religion, the construction of temples and edifices, all the core activities we associate with modern human endeavors, are the results of someone doing another one’s bidding along the concepts we now refer to as Service Oriented Architecture. Once SOA became firmly entrenched, there was no turning back. SOA became the paradigm of civilization. As it expanded, it created the specializations and professions we see in today’s world.

So, why wasn’t SOA used in IT systems from the start?

Earlier generations of computer technology did not have enough “juice” to support SOA. RAM was too expensive, disks were too slow, and communication speeds were hilariously sluggish (300 bauds[1] was super-fast, and data at that speed would have taken you something like twenty-four hours to download just one song from, say, The Gabe Dixon band—one of my favorites). Still, information systems had to support business systems and business needed IT; so an implicit compromise was struck. When it came to IT, SOA was abandoned in favor of an approach that forced business to adapt to computers rather than the other way around. For example, computers did not have sufficient storage space to store dates, so only the last two digits of a year were stored. Computers didn’t have the ability to present information in plain English. No problem, only abbreviated codes, upper-case text, and cryptic commands were used.

The result is that traditional IT quickly devolved into an assemblage of monolithic processes, inflexible data schemas, and unfriendly interfaces. Eventually, as a consequence of Moore’s Law, computers became more and more powerful, and more capable of tackling increasingly complex tasks. We have now come full circle. Instead of having to adapt to the computer’s limitations, it is the computers that are now expected to adjust to human ways of interaction and even to handle high-level questions and processes. Computers can now provide natural interfaces, follow complex heuristic-driven reasoning logic, and seamlessly tap large amounts of stored information; all in real-time. SOA is the natural way to architect systems, and its use in computer systems is the result of finally being able to effectively mirror business processes with technology thanks to the arrival of powerful computers, cheaper storage, and faster networks. Now, effectively doesn’t mean efficiently. Applying SOA conveys a certain acceptance that we can afford to “waste” computer resources to achieve the flexibility and transparency advantages SOA provides (more on this next week)—something similar to the way we have accepted the performance impact of using a higher level language over machine language programming.

Now, don’t get me wrong, badly implemented SOA can result in major costly failures. Just because we can now afford computational power to better mirror business processes doesn’t mean that resources are infinite, that budgets are boundless, and that the laws of physics can be suspended. In fact, SOA gives you flexibility, and we all know that with flexibility comes plenty more ways to screw up! SOA is not a panacea, but when properly applied, our computer systems can become part of the SOA future, just as SOA has always been a part of our past.




[1] In old modems this was equivalent to 300 bps.

Labels: , , , , , , ,

Friday, June 5, 2009

Taming Complexity

The complexity of a project is an intrinsic variable ultimately determined by the business processes. Keeping in mind the key dictum that a technology solution can never be of lesser complexity than the underlying problem it solves, you should first ensure that prior to automation, the business processes are as optimized as possible[1]. Many technology projects end up being unyieldingly complex because they try to tackle what is in essence a mash up of chaotic business rules.

Having said this, there is no reason why the technology solution should be more complicated than necessary. For instance, poor technical decisions or the use of inappropriate software tools can add unnecessary complexity to a project.

Recognizing that a project has an intrinsic complexity framed by the business requirements, your challenge is to efficiently tame this complexity within the realm of the technology and budget available to you. Here we have the “Law of Preservation of Complexity”: N = S + U; where N is the total complexity of the solution (as determined by the complexity of the business problem), S is the technology system complexity, and U is the complexity that the user faces when interacting with the system. Given that the intrinsic complexity of a project is “N”, it is your decision how to apportion the complexity between the system and the user.

Thinking about it, the simpler the user interface, the more complex the technology backend that supports that iteration is bound to be. I’m fairly certain that the underlying software supporting the Apple iPhone is quite complex, given that it is a device universally recognized as providing an extremely-intuitive interface. Alternatively, simple technology implementations tend to deliver solutions that are more manual in nature and hence more complex for the user. A complex solution that delivers an easy-to-use implementation is bound to cost more to develop, but its operational costs, due to reduced support and training costs will be lower. A simple solution that places the burden of complexity on the user can typically be developed more rapidly and at lesser expense, but its operational cost will be higher.

Taken to the extreme, the simplest automation solution is one in which automation ceases to exist and all business processes are manual. The most complex automation solution will probably involve the use of a sophisticated, artificial-intelligence system that is able to take on all automation burdens[2]. This capability, however, has yet to be invented.

In the end, it behooves you as technology leader to ensure that no unneeded technical complexity is introduced into your projects (watch out for that young bright, software maverick who pushes for that “cool”, dynamic rules engine to do the payroll!). Always keep in mind that the single most important independent variable in apportioning complexity is the understanding of the key business priorities and the true capabilities of technology.

The trick is to strike the proper balance in the deliverable. A good technical delivery will endeavor to place the bulk of complexity inside the solution black-box, making the interaction with the software simpler and more user friendly, but not at the cost of creating an un-implementable solution.

Many people confuse complexity with over-automation. If the complexity of the technology solution does not ultimately simplify the interaction with the user then you are not improving things. A personal peeve of mine is the recent “popularity” of so-called voice-response systems. If they were used only for simple voice driven selections, then there would be no problem. The issue arises when these interactive-voice-response (IVR) applications are forced to carry more complex interactions. The fact is that technology is not yet at the point where it can support these interactions in a seamless manner. In the end, the only purpose these “voice-recognition” systems accomplish is to weed out customers’ access to the support staff, at the price of lowered service levels, and also heightened customer dissatisfaction. The manner in which companies use these systems is a typical example of adding complexity for the wrong reason.


[1] In fact, one of the intrinsic benefits of IT Transformation is that, just like the act of smiling can make people feel happier via backward neural influence, this transformation can also serve as a trigger for much-needed business transformation.

[2] As I write this, I’ve come across the announcement that IBM is starting a project funded by DARPA to build a new type of computer that simulates the way the brain operates. A multi-disciplinary working team that includes Neuroscientists and other disciplines will work on this project. I doubt this type of approach will work (men finally invented airplanes when it stopped trying to emulate the way birds fly), but who knows?

Labels: , , , , , ,