IT Transformation with SOA

Back in the XIX century (that’s the 19^th century for all you X-geners!), there was a composer who didn’t know how to play the piano. In fact, nor did he know how to play the violin, the flute, the trombone, or any other instrument for that matter. Yet, the man managed to compose symphonies that to this day are considered musical masterpieces. The composer’s name was Louis Hector Berlioz, and he achieved this feat by directing the orchestra through each step of his arrangement and composition. His most recognized work is called “Symphonie Fantastique” and, according to Wikipedia, the symphony is scored for an orchestra consisting of 2 flutes(2nd doubling piccolo), 2 oboes (2nd doubling English horn), 2 clarinets (1st doubling E-flat clarinet), 4 bassoons, 4 horns, 2 trumpets, 2 cornets, 3 trombones, 2 ophicleides (what the heck is an ophecleide? A forerunner of the euphonium, I found out. What the heck is a euphonium? Well, check it out in Wiki!), 2 pairs of timpani, snare drum, cymbals, bass drum, bells in C and G, 2 harps, and strings.

By now, you probably get the idea. Mr. Berlioz fully exemplifies the ultimate back-end composite services element: The Orchestrator. Berlioz composed some pretty cool stuff by knowing a) what he wanted to express, b) what specific set of instruments should be used at a particular point in time, and c) how to communicate the notes of his composition to the orchestra.

Every SOA-based system needs its Berliozes.

There are several dimensions involved in defining the role of an orchestrator for SOA. First, as discussed earlier, most orchestrator roles will be provided within the context of an application; not as a part of a service. That is, the orchestration is what defines an application and makes one application different from another. The orchestration is the brain of the application, and it is the entity that decides the manner and SOA services calling flow.

In some instances, you might even be able to reuse orchestration patterns and apply them across multiple applications. Better still, you can build orchestration patterns by utilizing the emerging Business Process Modeling technologies (BPM). BPM simplifies the work of creating orchestration logic by providing a visual and modular way of assembling orchestration flows. A small commentary of mine is this: BPM is not SOA, but BPM requires SOA to work properly.

An apropos question is to ask how much orchestration should be automated in the SOA system as opposed to letting the user manually orchestrate his or her own interactions. To answer this question it is best to remember the complexity rule I stated earlier:  the simpler the user interaction; the more complex the system, and vice-versa. 

Then again, there are limits to the complexity of an orchestration. A full-fledged Artificial Intelligence system could become the ultimate orchestration engine but, unfortunately, such a machine remains in the realm of science fiction.  Cost-Benefit compromises must be made.

Say we have a travel oriented system and need to find the coolest vacation spots for the month of September. Should we let the user manually orchestrate the various steps needed to reach a conclusion? Each step would indirectly generate the appropriate service calls for searching destinations, filtering unwanted responses, obtaining additional descriptions, getting prices, initiating the booking, and so forth. Or we could consider developing a sophisticated orchestration function that’s able to take care of those details and do the hard work on behalf of the prospective traveler. But should we do it?

The answer lies in the size of “the market” for a particular need. Clearly, there is a need for a travel orchestration capability that can take care of all the details mentioned. After all, isn’t this why Travel Agencies emerged in the first place? If the orchestration is need by only a few users, then it is best not to spend money and effort attempting to automate something that is too unique. On the other hand, if the request becomes common, then it is preferable to create an automated orchestration function that organizes and integrates the use of SOA services.

The orchestrators design should always accommodate the transparency tenets in order to allow horizontal scalability. In other words, if you provide the orchestration via servers located in the system membrane, you will need to design the solution in such a way that you can always add more front end servers to accommodate increased workloads, without disrupting the orchestration processes in existing servers. Because orchestration usually requires the server to maintain some form of state, at least for the duration of a transaction, you will need to incorporate some form of session-stickiness in the orchestration logic. Later on, I will write more about why I recommend that this is the one and only area where a “session state” between the user and the orchestration should exist, even as I still advice to keep backend services discrete and sessionless.

Pursuing the analogy of SOA services mirroring the way civilization is structured around individuals and institutions that provide services to others can help us to understand the value of classifying the classes and types of services. After all, if you think of a repair shop, for example, one can see that the services provided by the receptionist are not the same as the services provided by the repair technician in the backroom, or even the services of the cashier who will collect your payment later on.  You would not normally consider putting the receptionist in a repair role, or the repair technician in the role of cashier. It’s best when every individual performs the role for which they are optimally qualified. 
Then there is at the manner in which the individuals render their services. Some have a role as orchestrators of other people’s work. The shop manager is someone who offers a coordination service that makes the business run coherently. Others perform a specific, specialized role. They do what they do without requiring others help (like those who order you out of the kitchen!). Then there are those whose main role relies on accessing a data repository of sorts. They front end actual information resources.  How much salary you afford and how much value and attention you confer to each, ultimately depends on a combination of all these attributes.

Similarly, with SOA we can define three service classes: Access Services, Enterprise Services, and System Services. 

Access Services are often implemented as wrappers for legacy applications and encapsulate the internal business service logic while taking a role as proxies to the clients. This role usually involves keeping and managing the state of a business flow on behalf of the external client. This is often needed in order to hide proprietary logic from external users. As a consequence, interfaces in this class of “services” will be extremely coarse and therefore somewhat verbose. In travel, for example, the Open Travel Alliance XML protocols tend to be extremely elaborate because of the very high level of service interface abstraction required for diverse companies to interoperate.

If you have control of the actual calling application, then you can implement the orchestration logic directly within them and provided the application runs internally, inside your DMZ scope. They can bypass the Access Services by calling the backend enterprise business services directly. 

Access Services and Enterprise Business Services classes are best defined in terms of their business value—as something easy to explain in terms of the business services they provide. But just as there are business services, there are also system services. 

System Services are the services that support the system and don’t usually have a direct business mapping, even though they indirectly support the business.   Still, remember that, as per our definition, you ought to be able to explain what these services offer to a layman, even if the services are not providing direct business functionality. Just because a service has a system focus, as opposed to a direct business focus, does not mean that the service should be so fine–grained that it simply serves some obscure function which could be better handled via a library or a subroutine.  

All service classes should comply with the guidelines and standards established for service life-cycle, but I suggest that the specific elements of the lifecycle will be different for each service class. Access Services will probably be public and should be normalized to industry standards as much as possible, while Enterprise Services should be governed by your internal architecture group. The likely user of the System Services will be your company’s operation team.

Now, let’s talk about Service Types. . . Just as services can be divided into Access, Enterprise Business, and System Services classes, they also need to be classified based upon their intrinsic roles and in the way they are internally structured so that you can assign their maintenance to the proper development organization.

Services that implement functionality requiring access to other services are known as Composite Services. Services of this type may at times keep a session state, but only for the duration of the execution (having services keep state across multiple service calls is generally not recommended, unless the service is an Access Service as discussed above).  On the other hand, services that provide function without needing to call upon other services are known as Atomic Services.  These services provide coarse-grained functionality in a single-shot.  A specific type of Atomic Services is Data Access Services.  The latter supports one of the key principles in SOA: avoidance of data base visibility from functional services, whether Composite or Atomic, and the interfacing of all interactive data requests via service interfaces. The diagram below gives an example of the kind of service classes and types you would see in a reservation system. 

Next week I will cover the service delivery patterns and I will provide an example of how all these service categories fit into an actual system design. Till then!