A number of new needs have emerged with the advent of SOA. First of all, there was no standard way for an application to construct and deliver a service call. Secondly, there was no standard way to ensure the service would be delivered. Thirdly, it was not clear how this SOA environment could be managed and operated effectively. Fourthly . . . well, you get the idea; the list goes on and on.
SOA demands the existence of an enabling infrastructure layer known as middleware. Middleware provides all necessary services, independent of the underlying technical infrastructure. To satisfy this need, vendors began to define SOA architectures around a relatively abstract concept: the Enterprise Service Bus, or ESB. Now, there has never been disagreement about the need to have a foundational layer to support common SOA functions—an enterprise bus of sorts. The problem is that each vendor took it upon himself to define the specific capabilities and mechanisms of their proprietary ESB, oftentimes by repackaging preexisting products and rebranding them to better fit their sales strategies.
As a result, depending on the vendor, the concept of Enterprise Service Bus encompasses an amalgamation of integration and transformation technologies that enable the cooperative work of any number of environments: Service Location, Service Invocation, Service Routing, Security, Mapping, Asynchronous and Event Driven Messaging, Service Orchestration, Testing Tools, Pattern Libraries, Monitoring and Management, etc. Unfortunately, when viewed as an all-or-nothing proposition, ESB’s broad and fuzzy scope tends to make vendor offerings somewhat complex and potentially expensive.
The term ESB is now so generic and undefined that you should be careful not to get entrapped into buying a cornucopia of vendor products that are not going to be needed for your specific SOA environment. ESBs resemble more a Swiss army knife, with its many accessories, of which only a few will ever be used. Don’t be deceived; vendors will naturally try to sell you the complete superhighway, including rest stops, gas stations and the paint for the road signs, when all you really need is a quaint country road. You can be choosy and build your base SOA foundation gradually. Because of this, I am willfully avoiding use of the term “Enterprise Service Bus”, preferring instead to use the more neutral term, “SOA Fabric.”
Of all the bells and whistles provided by ESB vendors (data transformation, dynamic service location, etc.), the one key function the SOA Fabric should deliver is ensuring that the services and service delivery mechanisms are abstracted from the SOA clients.
A salient feature that vendors tell us ESBs are good for is their ability to integrate heterogeneous environments. However, if you think about it, since you are going through the process of transforming the technology in your company (the topic of this writings after all!), you should really strive to introduce a standard protocol and eliminate as many of legacy protocols as you can.
Ironically, a holistic transformation program should have the goal of deploying the most homogeneous SOA environment possible; thus obviating the need for most of the much touted ESB’s transformation and mapping functions. In a new system, SOA can be based upon canonical formats and common protocols; thus minimizing the need for data and service format conversion. This goal is most feasible when applied to the message flows occurring in you internal ecosystem.
Now, you may still need some of those conversion functions for several other reasons, migration and integration with external systems being the most obvious cases. If the migration will be gradual, and therefore requires the interplay of new services with legacy services, go ahead and enable some of the protocol conversion features provided by ESBs. The question would then be how important this feature is to you, and whether you wouldn’t be better off following a non-ESB integration mechanism in the interim. At least, knowing you will be using this particular ESB function only for migration purposes, you can try to negotiate a more generous license with the vendor.
There are cases whereby, while striving for a homogeneous SOA environment, you may well conclude that your end state architecture must integrate a number of systems under a federated view. Your end state architecture in this case will be a mix of hybrid technologies servicing autonomous problem domains. Under this scenario, it would be best to reframe the definition of the problem at hand from one of creating an SOA environment to one of applying Enterprise Application Integration (EAI) mechanisms. If your end state revolves more around integration EAI, it would be better suited to performing boundary-level mapping and transformation work. In this case, go and shop for a great EAI solution; not for an ESB.
If the vendor gives you the option of acquiring specific subsets of their ESB offering (at a reduced price) then that’s something worth considering. At the very least, you will need to provide support for service deployment, routing, monitoring, and management, even if you won’t require many of the other functions in the ESB package. Just remember to focus in deploying the fabric that properly matches your SOA objectives and not the one that matches your vendor’s sales quota.
A quick word regarding Open Source ESBs. . . There are many, but the same caveats I’ve used for vendor-based ESB’s apply. Open Source ESBs are not yet as mature, and the quality of functions they provide varies significantly according to the component. Focus on using only those components you can be sure will work in a reliable and stable manner or those which are not critical to the system. Remember you are putting in place components that will become part of the core fabric. Ask yourself, does it make sense in order to save a few dollars to use a relatively unsupported ESB component for a critical role (Service Invocation or Messaging, come to mind), versus using a more stable vendor solution?
In the end, if you are planning to use the protocol conversion features packaged in a vendor-provided or open source ESBs, I suggest you use them in a discrete, case-by-case basis, and not as an inherent component of your SOA fabric. This way, even as you face having to solve integration problems associated with the lack of standards, at least you won’t be forced into drinking the Kool-Aid associated with a particular vendor’s view of ESB!
These are the types of tenets you can laminate in plastic and frame for the office wall next to the water cooler. They don’t often change, but change they do.These tenets usually define the current state of the industry preferences for specific technologies. While usually applicable to current environments, there is no guarantee they won’t eventually need to be revised. Take a trip down memory lane and imagine a return to the year 1969. Yes, Woodstock happened and you probably don’t remember it, either because you hadn’t been born yet or because you were in fact there. After listening to Janis Joplin, Santana, and Jimmy Hendrix, you show up to the office and are given the task of defining the tenets for a data processing application. You clear your mind and readjust to the fact you are back at work. You define a tenet that states customer records shouldn’t exceed 80 characters. Why? Because each record must fit on a punch card, and a punch card holds a maximum of 80 columns. Clearly, when there’s a need to code for the year field, you’ll opt for conciseness and drop the “superfluous” 19 from the year, etc.Another tenet might have been to make certain that all job submissions are hand-delivered to the “Batch Service” window with the user signing the log sheet.
Unlike Category I tenets that tend to be industry-agnostic, Category II tenets have a more focused applicability. Some of these tenets will be more appropriate for a particular industry segment, geography, or IT situation.If you are in a service industry, for example, you will have tenets that emphasize functionality over other attributes; if you are in the manufacturing industry, you will probably focus on tenets geared to reducing costs, and if you are in the videogame industry, your tenets will highlight usability and playability.
The list of Category II Tenets can therefore be extremely large. Below, I show some such tenets as examples. Again, when it comes to creating your own tenets, you’ll need to assemble your team and map those tenets that best apply to your specific conditions and that contribute to meeting the category I tenets previously defined:
·Move to a common development environment to maximize use of common support processes and tools within any of the deployed platforms.
·Object-oriented methodologies and approaches will be pursued whenever possible. Percentage of reused code will be a success factor.
·Use state-of-the-art programming tools and techniques to ensure maximum development productivity. Use Life Cycle and Service Studio components to facilitate service reuse. Optimize the development cycle by using design methodologies that permit analysis of the system under a business view and rapid prototyping for proof of concept. Use UML whenever possible.
·Ensure reliability through redundant clustering, either purchased or built.The strategy is to seek system level resilience via n-plus one redundancy with automatic fallback/recovery.
·Consider security requirements from the start. Establish security, logging and billing mechanisms based on service-oriented constructs.
·Message queuing is the strategic middleware standard. Mapping to other middleware will be supported as required, but the canonical service distribution protocol will be SOAP based.
·Unify connectivity protocols and communications interfaces end-to-end.Reduce the number of different access methods and protocols used inside the central complex. Use open, off-the-shelf protocols and systems. TCP/IP is the preferred connectivity protocol.
·Hide the complexity of the DBMS from the applications, users, and processes that access the data, by encapsulating these database accesses with service oriented APIs.
·Enable distributed data access with connectivity to heterogeneous DBMS within one unit of work.
·AvoidDBMS access across the wide area network. Propagate remote data queries via a service-oriented approach.
·Portals will deal with user sessions and presentation and will orchestrate the user interaction, including content presentation, but the business services will be provided by backend engines and will not reside in the front end portals.
·All systems to be adapted, built, or bought must provide the following features:
oProactive and reactivemonitoring capabilities
oFault tolerant, load balancing, and fail-over capabilities
oPerformance measurement, logging, and system health reporting
oIntegration with system administration and management standards.
In addition, Category II Tenets are the natural realm for all SOA-related tenets. Those tenets deal with desired levels of service-granularity, SOA governance, SOA management approaches, etc. Because the grand theme of IT Transformation is the use of SOA, I will later cover these tenets in detail.
The Category III Tenets
These are the tenets that are defined at the departmental level. They describe naming conventions, documentation standards, reporting mechanisms, and pretty much anything dealing with the nitty-gritty of running a project and are therefore as important as the other categories.It is a good idea to keep a formalized track of how these tenets are defined, communicated, and enforced.
A typical challenge is that sometimes, tenets are created as Level III when in fact they ought to be vetted with a Level II scope. The opposite is also true. A sure sign of an organization trying to over-control a project is having Category II tenets that should be left to a subsidiary unit. Remember that allowing a degree of departmental freedom in choosing tenets at this level is actually a good way to ensure fluid dynamics and engagement of the base in the overall standardization process. In the end, you may have something akin to British Common-Law whereby Level III tenets become so widely adopted that in time they emerge as de-facto Level II tenets!
In my experience, developers tend to converge around novel tools that, due to their emerging nature, have not yet come onto the radar of the corporate bigwigs. Sometimes this results in conflicting viewpoints between the directives driven from the top and the preferred technologies driven from the grassroots.When thishappens you will need to open the floor to debate and be prepared to make some compromises.
An example of this situation is the manner in which TCP/IP was adopted as the preferred networking protocol. It is a fact that TCP/IP emerged from the grassroots, even as corporate directions coming from the top were trying to enforce the OSI or SNA protocol stacks. The ‘can-do-now’ and ‘can-do-cheaper’ pragmatism espoused by departmental gurus won over the ‘comprehensive’ and ‘structured’ OSI dogma espoused by the architects.
Because the OSI standard was a camel (i.e. a horse designed by committee), it was practically impossible to implement, and the grassroots’ preference for TCP/IP did turn out to be the right choice. To be fair, there are examples where grassroots-driven adoptions ended up being counter-productive. Take the wildfire adoption of the distributed Novel servers in the latter part of the eighties that ultimately led to the organic emergence of extremely complex environments; environments with a high degree of data heterogeneity and processes that companies had a hard time extricating themselves from.Now, I am not suggesting that the Novel corporation or its technology was at fault (after all, it was in fact a fairly leading technology for its time). Rather the issue was that much of the deployment of this technology happened pretty much ‘under-the-radar’.Most companies ended-up with duplicate data bases, data bases that lacked the necessary security, badly managed data, and in many instances, application code that was written for the very proprietary vendor environment and which could not be reused.
So what are examples of Category III tenets? The list can become fairly detailed and nuanced. I suggest breaking down these tenets into logical groups so that you have at least a means to compare and contrast tenets coming from different groups. The following groupings are recommended:
Data Tenets. Specific to the data, these tenets would include naming conventions, partitioning approaches, etc.
Foundational Tenets. Tenets dealing with system infrastructure and management would come here. Network IP standards, server naming standards, etc. would come here.
Presentation Tenets. These cover usability standards, use of presentation elements, navigation, etc.
SOATenets. XML tags, repository placement, service levels, etc.
General Tenets. Tenets dealing with general governance and project processes, including vender-selection strategies, evaluation and testing methods, etc..
Also, as a matter of fact, if you can also group the Category II Tenets along these lines, it would be a extremely useful way to ensure coherence between the Category III and Category II Tenet narratives.
That’s it. . .If you are “teneted-out” I don’t blame you. It’s time to dive into the Level II Architecture stage and, in particular, into SOA as the solution for that architecture level.
This blog covers the practical techniques, trials and tribulations associated with the transformation of IT systems from legacy technologies to systems using SOA and modern open systems. I also include the occasional interlude with rants about technology in general.
About Me
Name: Israel del Rio
Location: Atlanta, GA, United States
Israel has been recognized by Computerworld as one of their Premiere 100 IT honorees. Israel is a business and technology leader who has contributed the technology vision as key strategist and designer behind the enterprise technology roadmaps of large hospitality and travel companies. Israel has also have developed and deployed various mission-critical systems and in the process, he's been instrumental in creating and building effective and skilled development organizations.
