IT Transformation with SOA

Advantages of the Service Oriented View

If SOA is so “old” why do we still have all this excitement about service oriented architecture applied to IT? Isn’t SOA an obvious choice anyway? In fact, the history of computer science has always been about movement from the very complex to the obvious. While in the pioneering days of computing it took a John von Neumann to work out programming concepts, these days even an Alfred E. Neuman could easily write a decent program. The reason the “obvious” solution was not used in earlier decades is because “obvious” solutions require more complex technologies (recall my earlier discussion on the preservation of complexity). Think of the electronic spreadsheet which was invented in the early 80’s (Visicalc). In hindsight this invention is something obvious. Why then wasn’t the electronic spreadsheet invented even earlier? If we take a look at the character oriented computer screens prevalent in the decade of the 70’s, it’s clear that a spreadsheet model would not have been adequate for the teletype-like devices of that era. It took cheaper bit-mapped displays for the electronic spreadsheets idea to become viable.

What’s obvious these days is that implementing systems with SOA can lead to better solutions as long as the inherent issues of SOA are tamed and appropriate service interfaces and service management governances are established. In this context, a service-oriented view provides many advantages:

1. Allows a direct mapping to a business perspective. SOA allows the implementation of solutions that can directly mirror the business processes. In the past, system designers had to translate business processes into computer driven structures. System implementations based on awkward mappings of business requirements did work. However, given that the most natural way to describe business processes and organizational flows is through a consumer view and given the fact that older computers couldn’t cope with these views, the resulting IT systems almost always ended up being difficult to use.

In fact, SOA’s facilitation of a direct mapping with the business is key to the emergence of higher-layer business tools such as Business Process Modeling (BPM). BPM represents an even higher level of abstraction of automation which can be still be used to dynamically generate software solutions. The capability to give the definition of business processes to actual business users and then have these definitions used to generate actual applications is only feasible when these processes can call predefined services.

Without SOA there is no BPM.

1.

2. Enables Reuse of well-defined logic blocks. This is a Lego approach. Just as a building contractor assembles the skills needed to build a house, services can, in turn, call services. A general-purpose service can be re-used by several applications.

Admittedly, reuse is not a new concept. It has been the holy-grail of computer science for many years. For example, in the early days code reuse was sought via definition of macros or subroutines—chunks of source code that could be embedded into the application.

While this approach had the advantage of reusing source code providing generic functions (“Convert Data”, “Hash a Table”, etc.) it did have its issues. First of all, programmers would sometimes tweak the library code to better match their requirements; making the code non-reusable. Also, as new programming languages emerged, these libraries became outdated and could no longer be used. This is not to say the use of macros is no longer valid. The use of macros or functions for repetitious snippets of code is still a recommended best practice, but only within the confines of a single application.

In time, other code reuse techniques emerged. The most important of these were the linkable libraries. Unlike macros, the programmer could use the library without having to know the source code; thus giving some degree of protection against improper changes. However, as with macros, these static libraries become embedded in the executable code, using more memory and ultimately forcing the updating of programs whenever the libraries changed.

Enter the Dynamic Libraries. The familiar Windows DLL (Dynamic Link Library) is basically a library that becomes dynamically linked to a program, pretty much on demand. However, dynamic libraries require the use of a specific running environment (i.e. MS/Windows) making them viable only when executed in the same system as the calling application.

Dynamic Libraries represented a good step forward, and indeed they have been extremely popular as commercially available add-ons for development tools such as Visual Basic and more broadly under MS/Windows frameworks. Still, what if you wanted to use a dynamic library from a different platform? What if you could place dynamic libraries into any system and be able to call them from any platform?

Enter the concept of Services!

Just as with DLLs, you can acquire and use external tools and services, but more fundamentally, because of the loose coupling, you can run your service in a system completely different from your own. You can call a Linux service from a Windows application, or call a service located somewhere in a cloud.

SOA is all about transparency . . .

3. SOA is the foundation for transparency. If you call a help desk these days, chances are that the person at the other end of the line is based in a foreign country. Thanks to the lower communication costs and the benefits provided by educational standards and globalization, companies benefit by sourcing the services wherever they are the most cost effective. Likewise, you can run services anywhere and have them accessed by any authorized user. SOA allows you to place a function where it makes the most sense. But, since what makes sense today may not make sense tomorrow, SOA is also about allowing change with a minimum of effort. We can finally decouple the way we logically partition functions from the way we deploy physical computer systems. Service Oriented Architectures should provide as many of the following transparency tenets as possible:

· Access Transparency. Provide the ability to access the system from different devices and mechanisms.

· Failure Transparency. Designing the systems with automatic service failure fallback, without affecting the application.

· Location Transparency. The ability to deploy the system in any location.

· Migration Transparency. Allow minimum or no impact to the existing system when upgrading service implementations.

· Persistence Transparency. If the desired service has not been used, you can load it automatically. If it has been used before you can reuse the code already resident in memory.

· Relocation Transparency. The system should allow you to movie a service from machine A to machine B without impacting clients.

· Replication Transparency. The system should be able to provide the same service from different locations. This supports failure transparency and it can also be used to increase performance via horizontal scalability.

· Technology Transparency. As long as you get the service to do what the service is meant to do, the client shouldn’t care about the technology used to implement the service.

These transparency attributes facilitate legacy integration with new technologies. Since the implementation of the service is hidden from the service consumer, the service-oriented approach enables the integration of older legacy software with emergent software. Once older applications are properly encapsulated under the guise of services, it is also possible to gradually transition a system by re-implementing services one step at a time. This removes the necessity to incur in risky “big-bang” system migrations. Also, these transparency attributes are what makes emerging technologies such as Cloud Computing possible. Without transparency there are no clouds!

4. Simplifies software development by decoupling business processes, decisions, and data. The agility gained from SOA comes from the inherent simplification of the software design. It becomes easier to assign development of different modules to different groups and isolates the way the program accesses the data. More importantly, because SOA can mirror the business processes, you can create organizational structures in IT that truly mirror the business structures.

The diagram above shows this concept pictorially. It’s far easier to decompose and assign each of the various business processes inside the box to the right, using external services, than having to adopt responsibility for the way data is accessed or manipulated.

Now that I have sung the praises of SOA, I want to bring us down to reality just a bit. There are certainly many traps and difficulties in using SOA, and it makes sense to be aware of them so that you can use the well-travelled roads of successful past experiences as much as possible. This means that designing SOA is all about figuring out the concept of patterns and the way services fit into these patterns.

That’s next.

Labels: Advantages, BPM, Cloud Computing, DLL, Macros, SOA, Transparency, Visicalc Microsoft, Visual Basic, Windows

posted by IT Transformation with SOA @ 2:26 PM

Musings about Innovation

Thus far I have focused primarily on aspects related to the corporate culture, the business requirements, and the funding considerations that serve as a foundation for an IT Transformation initiative. Soon I will be delving into the more technically focused steps required to provide the strategic technology framework for this transformation, such as the gathering of specific requirements, setting the program scope, and defining the high level architecture and technology transformation tenets. However, before I proceed with this technical nitty-gritty, I wanted to highlight one important element that I believe must be at the core of any IT transformation initiative: innovation.

Surely, one of the sweetest and most rewarding outcomes of a true renovation program is the introduction of truly innovative features, technologies, and processes. A transformation program that results in more of the same tired old solutions, except for the “we’re now running a more modern platform,” is bound to excite no one and is less likely to meet emerging requirements. Without Innovation, IT Transformation would be like pimping a Ford Pinto with velvet seats and a “La Cucaracha” horn.

Innovation can and should take place at each step of the transformation, from the identification of business processes suitable for improvement, to the manner in which you lay down the technology strategy, to the methods and processes you follow, and certainly to the specific elements and inventions of the solution.

Then again, this may be easier said than done. The creation of innovation is not the sort of thing that can be scheduled in a plan. Innovation is about the “how” we solve problems and not about the “what”. There are plenty of white papers attempting to formalize innovation processes as if innovation were something that can be created by simply following predefined methodologies: “Put processes here, structure this, add water, and bingo! Invention will happen!” Forgive my skepticism. It just doesn’t work that way.

After all, the genesis of innovation is perhaps one of the most intriguing aspects of all human endeavors, and its appearance can often only be explained as the result of the combination of the right circumstances and a little bit of luck. Innovation is more about atmospherics than about process. Why? I don’t believe an innovation process can be formalized with the kind of structure and targeted predictability that is applied to other methodology-rich disciplines such as auditing or quality assurance.

The truth of the matter is that the one proven way to foster innovation is to secure resources with the smarts and ability to generate ideas, all the while ensuring they enjoy an environment that supports and encourages the emergence of their ideas. Even then, new ideas may or may not occur, depending upon the presence or absence of a third element: serendipity.

Serendipity matters. Think of the invention and use of the wheel. Depictions of Sumerians carriages pulled by horses confirm that the wheel was first invented at least 5,000 years ago. On the other hand, look at the Americas. The Aztecs never used the wheel for transportation, even though they had invented it and used it in children toys. Why? The most common explanation is that the wheel was impractical to them because they lacked horses, oxen, or any other animal capable of pulling a carriage. Fair enough, as far as explanations go this is not a bad one. It’s just that I don’t buy it. Even though the Aztecs did not have horses or oxen, they did have access to readily available beasts of burden: enemy prisoners and slaves. Given that human rights was not high on that society’s agenda, one would imagine that instead of extracting the hearts of thousands-upon-thousands of their war prisoners, the Aztecs could have put some of those folks to a more practical use by having them pull wheeled carriages. No, a simpler explanation is this: the idea to use carts with wheels as a form of transportation simply did not occur to them! In fact, it’s just possible that the one individual that might have come up with the revolutionary idea to use the wheel for transportation was one of the unlucky fellows whose heart was pulled out for the benefit of the gods! With their pervasive human-sacrifice program, the Aztecs definitely failed to provide the right innovation-fostering environment.

Now, before you begin to argue that perhaps the Aztecs were just not all that smart, think of wheeled-luggage. Why wasn’t the idea to attach wheels to luggage invented much sooner?

I’m amused every time I view a fifties movie classics with people carrying immense traveling trunks. Most of us old enough to have used vinyl records had to go around airports, train stations, and bus stations hauling heavy luggage. And yet something so seemly obvious and technologically viable, such as attaching wheels to luggage, did not occur until very recently!

Again, why is this?

It wasn’t that the technology to attach wheels to luggage didn’t exist, or that train stations lacked even terrain, or that travelers of yore had bigger muscles, but quite simply the idea had not occurred to anyone!

A caveat regarding innovation is this: beware of the gimmick. Gimmicks are to innovation what margarine is to butter. They only give the appearance of innovation but fail to provide real value and benefits. They shimmer with the glow of vanity that hides their ultimate banality, and quickly become anchors that usually add cost and obfuscation to the deliverables. Remember that annoying Windows paper clip “assistant”? Gimmicks often occur when there is an artificial pressure to innovate; to come up with “new stuff” no matter what. The potential for their embarrassing existence is another good reason to avoid the establishment of inane innovation quotas.

You don’t need gimmicks. Perhaps it is true that the best ideas are often the simplest ones—like the wheel. However, if you plan to develop a new system that includes a fair share of innovation, you could do more than simply hope for the best. A good environment for innovation is made of constructing appropriate facilities (offices, labs, and equipment), providing individual and team recognition, establishing open communications, instigating flexibility and, most importantly, ensuring the team is fully aware of the shared objectives and passionately committed to success.

Then again, a little bit of luck doesn’t ever hurt.

Serendipity awaits.

Labels: Aztecs, Gimmicks, Innovation, Pinto, Serendipity, Strategy, Sumerians, Technology, Windows

posted by IT Transformation with SOA @ 4:44 PM