Sooner or later it happens to most of us. We grow up and no longer can continue to live in the cocooned environment created by our parents—the comfort and coziness of our youth is gone (except if as a result of the Grand Recession you are obliged to return to your parents home and are forced to experience the George Constanza-like awkwardness of adulthood, but I digress). Either way, we have to enter the real world, a world where people speak the language of credits and debits and where behaviors are no longer governed by Ms. Manner’s etiquette or Mom’s nagging but rather by a set of complex social rules that help us interface with the world. The way we engage with the world, the set of rules we follow, the processes and mechanisms we use to interact with others, the whole cultural context of how to say “please”, or “keep the change”, are equivalent to a boundary layer between us and the rest of humanity.
Having created a suitable SOA system (either homogenous or federated via Enterprise Application Integration tools), we need to enclose it in its own protective cocoon, lest the reckless world outside trample with its internal fabric. The trick is to prevent what is not wanted from getting in while allowing what is wanted to access the system. Here, biology provides us with an ideal model in the workings of the living cell. Just as the membrane of a healthy cell acts as a barrier against harmful agents and judiciously allows the exchange of the enzymes needed to make the cell work in concert with the rest of the organism, we must maintain an SOA membrane that allows the necessary information exchange to take place while keeping the bad guys out of the system.
In IT terms, the membrane is known as the DMZ (Demilitarized Zone). Frankly, I never cared for this term. A DMZ is a buffer zone designed to keep warring factions apart—a zone void of hostilities. The term is deceiving because, in reality, the DMZ is the area where all access battles are fought. Also, the layer’s role is not to keep warring factions apart but to allow the controlled exchange of participating partners. With the emergence of virtualization approaches such as Cloud Computing, we should take the perspective that the membrane is the region where safe trade occurs. In this area the presentation logic is placed alongside a number of public applications. This is the layer that deals with the Business-to-Consumer (B2C) and the Business-to-Business (B2B) interactions. In this layer you also must perform data transformations for data exchange with external entities.
In engineering terms the membrane consists of an arrangement of technologies carrying the interaction with the external world in each layer of the computing stack, from the security guard manning the entrance to the Data Center to the application displaying the sign-on screens. In the networking layer you have the protocol convertors, VPN gateways, IP routers and load balancers. Further up in the stack, the membrane includes firewalls with the appropriate system-level access passwords and permissions; including specific field-level encryption. Even higher up, the membrane contains the needed data mapping and conversion services. Moving on to the application space the membrane includes spam filters and user-level authentication mechanisms.
Rather than give a subliminal message, let me state it as loudly and plainly as a used car commercial before a Memorial-day sale: it’s preferable to create the membrane with off-the-shelf technologies rather than to try to develop your own. The capabilities and features needed for this layer are usually standard to the industry, and thus it makes sense to use vendor solutions. In fact, a trend is to have many of the functions needed by the membrane be handled by special-purpose hardware appliances.
Alternatively, if you plan to outsource operations, then let the hosting provider worry about the make-up of the membrane. Still, you have to define the required levels of service and make certain the monitoring tools and processes exist to ensure these levels. Either way, the membrane is a component that’s rapidly becoming commoditized. A good thing too, for this is not where you ought to seek competitive IT differentiation (that is, unless you are one of those hosting providers!).
To sum up, the membrane is not the area to invest in internal development. The challenge is to create and view the membrane as an integrated component that can be managed and monitored in a holistic manner even if it consists of an assemblage of products and technologies. If you are creating a membrane you should focus on sound engineering work and vendor component integration; not software development.
Ultimately, a well-designed membrane should be trustworthy enough to allow some relaxation of the security levels inside the system. Also, a well-designed membrane should be flexible enough to allow support for a variety of access devices. Once you take care of your system’s membrane you can then focus on what happens inside, where the real work takes place, with the Orchestrators.
Because architectures should strive for flexibility, there is always a danger that, if they are not properly scoped, you may end up with a pile of “architecture” documents that no one will care to read, let alone implement. When defining the architecture it is essential for you and your team to define it with the right scope. If you are to keep it real, boiling the ocean is not an option. (“Let’s boil the ocean to get rid of those German U2s, the implementation is up to you.”)
For example, when defining the scope, ask yourself if you are trying to address only the customer-facing automation side of your business, or if you will also include the backend support systems [1]. If the latter, chances are you will need to consider integration to the ERP vendor. In any case, you’ll need to define whether to take an end-to-end customer lifecycle perspective or to focus on specific core business processes only.
Should you support international environments? If your company is a conglomerate, will you cover multiple business units or just one particular division? Are you even empowered to define the backend architecture?
Answering these questions requires a candid, non-delusional understanding of whether you have the kind of organizational governance to define far-reaching enterprise architectures or are limited to specific areas under a narrower span of control. That subsidiary headquartered in Paris might not appreciate your trying to dictate their technology!
Understanding your own limitations is essential. Clearly, it would be nice if the scope of the architecture encompassed as much of the “problem-space” as possible, but the fact is that real-life problem-spaces are highly irregular.
Were you to depict a typical problem space using a shape, chances are you would end up with a splattered blob like the one that follows:
Non-withstanding your governance limitations, you could define the architecture with a scope so broad that it covers all possible points of the problem space—like throwing a stick of dynamite into lake to do some fishing. This ‘solve-world-hunger’ approach will specify a great deal of things that just don’t belong to your specific transformation initiative. While all-encompassing, this ambitious approach will most likely be very expensive and non implementable.
The next diagram depicts a more realistic approach. The architecture scope here represents a compromise between specifying the largest possible conjunction of common elements in the problem set while ignoring the more “far-out” cases. It is this conjunction that forms the core of enterprise architecture—the main area of focus for the strategic architecture group.
Areas of the problem space not covered by the core architecture can always be handled as exceptions on a case-by-case basis. Who cares if content integration with the three person art-department happens with an 8GB USB stick? And yes, you have to accept the branch office in Namibia entering their one daily purchase order on a typewriter and then faxing the batched weekly orders on a Friday night. When the day comes that Internet connections become cost-effective there, you can then bring them into the fold of your architecture.
Such pragmatic exceptions avoid the need to over-architect data bases or data exchange formats and do help keep the architecture simpler.Of course, it is best to minimize these special cases as much as possible. Were you to set the scope in a too-narrow a fashion then most everything will becomes an exception! In addition to assuring yourself hours of the extra work needed by this fly-swatting approach to solving issues, you will also be costing the company more than necessary.
Even though the exceptions should be in the periphery of interest for your architecture team, you should at least be aware of their existence. Better yet, the architecture group should establish preferred guidelines on how best to admit exceptions so that the possibility of their future alignment with the core architecture is not compromised. For instance, establishing the use of a standardized Comma-Separated-Value (CSV) file for “non-traditional” data exchanges (including the art department’s USB stick), or defining the product codes to be typed in the manual purchase orders from Namibia, would at least ensure that exceptions can be better handled in the future.
It’s also worth remembering that the fluid nature of a business requires the ability to append new problem space areas to the original space (imagine when a business merger or company reorganization occurs). Typically, these “out-of-space” problems will have to be first handled on a reactive basis following the rule that new features are always needed “by yesterday”. However, you should have you team work on integrating these to the mainstream architecture as soon as practical. When this occurs, you will need to start the process of redefining the architecture scope all anew.
But what does the term “Scope” means in practical terms?What’ is it precisely that your architecture team should be defining as being either in or outside of scope?
The architecture scope should define the layers of technology to use (off-the-shelf or homegrown). Also, the scope should detail the applicability of the architecture as it concerns your organization’s structure, both horizontally (how many departmental functions and geographies it spans), and vertically (what level of detail will the architecture attain? Will you stop at the regional level or define the departmental elements?). In regard to timelines, your scope should go all the way to the long-term—there is no such a thing as a short-term architecture.
Perhaps you recall that back in the late eighties there was an Open Systems Interconnect standard (OSI) being pushed by the International Standards Organization. The specification reflected a design-by-committee approach that failed to take hold when faced with more effective de-facto standards such as TCP/IP, but one of its most enduring legacies was the definition of the seven-layer interoperability stack.This stack divided the various system elements in accordance to the type of service provided by each layer.
This framework can be useful in showing how the IT industry has been traversing the OSI layers toward commoditization.In the early days of IT, much blood was shed in battles centered on which technologies were to be chosen for the networking layers. Architecture scopes had to deal, for example, with whether to use SNA or DECnet, or even whether to implement something proprietary (it happened; one of my deliverables in a project long ago was for a proprietary communications protocol). Today it would be incomprehensible to debate against the use of TCP/IP as the protocol of choice, unless you are in a research organization testing the next big thing. The industry has converged to standards that moved the battleground upwards in the stack; toward OSI’s highest layers.
The diagram below shows the OSI layers. Shown in the horizontal axis are the trends toward standardization. In consequence, these days the focus is on actual application and service specifications; not on the underlying networking layers that, for all practical purposes are now completely commoditized.
For all the criticisms against it (the OSI specification truly became a boiling-the-ocean example[2]), it at least served as a unifying schema. The problem is that the OSI Reference Model is now outliving its usefulness. Despite the industry trend toward standardization and commoditization, an area of “infrastructure” that remains yet to be standardized is the Service Distribution Fabric[3]. SOA is today in a similar state of maturity as the networking world back in the eighties.With the so-called Enterprise Service Bus (ESB) we have vendors resisting standardization measures in order to sell their own proprietary solutions and get competitive advantage.Just as happened with TCP/IP, the world awaits a vendor independent SOA standards specification.
More work is needed to define the detailed structure standards for the Presentation and Application layers. This burden has been taken up by a variety of organizations such as the W3C, the OMG (Object Management Group), the Java Community Process, IEEE, many others, but we are not yet at a point where we can say that application and presentation layer standards have been universally adopted (Web browsers are still not guaranteed to display content the same way), but I have no doubt the industry will continue to converge towards more standards for SOA and the upper application development layers.
While that happens, your role will be to apply you know-how to best adapt from the range of available technologies and standards the solutions that best meet your specific needs. Defining the scope of the architecture, even when “only” focusing on the highest layers of the OSI stack, remains a challenging and fundamental proposition.
Life is still fun!
[1] As per my definition, a pure backend ERP project does not an IT Transformation project make, but there can be super-duper projects that require front-facing and backend integration.
[2]Ironically, the OSI standard is an example of an architecture scope run amok. Its broadness made it extremely difficult to implement. In fact, the standard became so bloated that various “implementation subsets” were defined. The irony of that compromise was that if you had Vendor A implementing subset X, there were no guarantees this vendor could interoperate with Vendor B! implementing subset Y.The whole point of having a standard in order to ensure vendor interoperability was lost!
[3] This is commonly referred to as Enterprise Service Bus, but since we are not yet at that point where vendors even agree on what the ESB term really encompasses, I prefer to use the more generic term: Service Distribution Fabric.
This blog covers the practical techniques, trials and tribulations associated with the transformation of IT systems from legacy technologies to systems using SOA and modern open systems. I also include the occasional interlude with rants about technology in general.
About Me
Name: Israel del Rio
Location: Atlanta, GA, United States
Israel has been recognized by Computerworld as one of their Premiere 100 IT honorees. Israel is a business and technology leader who has contributed the technology vision as key strategist and designer behind the enterprise technology roadmaps of large hospitality and travel companies. Israel has also have developed and deployed various mission-critical systems and in the process, he's been instrumental in creating and building effective and skilled development organizations.
