Welcome!

Weblogic Authors: Yeshim Deniz, Elizabeth White, Michael Meiner, Michael Bushong, Avi Rosenthal

Related Topics: Weblogic

Weblogic: Article

Enterprise Portal Integration and the Enterprise Service Bus Part 2

Enterprise Portal Integration and the Enterprise Service Bus Part 2

Enterprise portals have become the most popular method of offering a common user interface to a suite of services across the enterprise. Offering business visibility, flexibility, and knowledge management, portals promise users the ability to monitor, search, and manage business activity across the enterprise.

However, as portals mature from basic offerings to more complex sets of services, they will require a robust, agile, and pervasive integration network to provide critical business services to portal users in real time. The Enterprise Service Bus (ESB) not only provides the necessary infrastructure to reliably link services with the portal across heterogeneous environments, disparate organizations, and geographic boundaries - it also has the capability to provide or enable more advanced services, such as integrated business process management and business activity monitoring.

In part 1 of this series (WLDJ, Vol. 2, issue 9), we discussed how the ESB can provide a reliable "services network" for a portal, providing reliable transport, intelligent routing, transformations, and the ability to operate in a highly distributed and federated environment. Strictly speaking, Gartner has defined the ESB has a "lightweight integration platform composed of message-oriented middleware (MOM), transformations, intelligent routing, and Web services." However, once coarse-grained services are accessible from the bus, services can be recombined in multiple ways to form unified business processes that span the enterprise. As business processes execute over days, months, or even years, a new wealth of information can be extracted to provide deeper business awareness and insight. These types of capabilities are the next evolution of the ESB and are critical to providing the "missing link" in a service-oriented architecture (SOA).

In this article, we'll examine two use cases that demonstrate the extended power of the ESB, beyond what a single application server or portal platform can provide on its own. The first use case discusses the notion of a "Worklist Waypoint," a step in a long-running business process that needs manual intervention. A portal offers the user a "human interface" accessible from virtually anywhere, making it a good choice for users to interact with a business process. The second use case shows how portals can provide a window into the operational aspects of the business processes, typically referred to as "Business Activity Monitoring." To illustrate these points, we will once again enhance the "Avitek Medical Records" tutorial shipped with BEA WebLogic 8.1. The tutorial is modeled around a portal (a J2EE Struts application) that allows patients, physicians, and administrators to view an aggregated set of medical records.

Worklist Waypoint
One of the great opportunities of SOA is the ability to create a set of common business processes that bind together services in different applications, technology domains, and even organizational domains. But with this opportunity come significant challenges. What type of infrastructure is required to allow business process execution to operate over long durations (days, weeks, months, or even years)? How will manual steps be intermixed with automated steps, and how will the business process execution interact with my various presentation tiers, both now and in the future?

Let's consider that Avitek would like to automate the process of scheduling lab work for a patient. Through the Avitek portal, a doctor will be able to refer a patient to one of several labs for a test. The business process might look something like Figure 1.

There are a few characteristics that should be observed from this diagram. The first, and primary item of note is that the business process must be long running. The business process combines manual steps (shaded in Figure 1) with automated steps (services on the ESB) that may take an undetermined amount of time. In addition to the latency caused by manual steps, the availability of services themselves may also extend the duration of a business process. Given the long-running nature of this type of business process, the ESB provides a perfect foundation to manage long-running business processes. Leveraging the guaranteed delivery characteristics of the underlying Java Message Service (JMS)-based transport, business process orchestration is a natural extension to the bus.

However, further analysis of the business process reveals a more complex requirement. The manual steps require some type of human interaction to satisfy the conditions of the business process. An open, standards-based, flexible interaction model is required to allow any presentation tier interface to interact with the business process from anywhere on the ESB. For example, Figure 1 shows two different interface types providing input to the process: a portal (in our case, the Avitek portal), and a custom .NET application (designed specifically for the lab technicians to provide results back to the process). The interfaces should be flexible enough to interact with other types as well, such as Interactive Voice Response (IVR), Web services, or even some type of handheld device used by a lab technician. A loosely coupled approach, such as Web services/XML or JMS/XML, is an appropriate choice to satisfy these requirements. The ESB supports both of these natively.

In business process terminology, manual steps are handled through a technique known as a "Worklist." A Worklist is a specialized "inbox" for tasks that need to be accomplished by an "Actor" in a business process. A task correlates to a specific instance of a business process definition. For example, a lab technician may have seven open lab requests that need fulfillment. In this context, a Worklist would contain seven process instances of the "Lab Referral Process" (from Figure 1). The first required service is a "Worklist Summary" that returns the list of tasks for a specific user (see Figure 2).

Once the task lists have been retrieved from the Worklist Summary service, the user may select a task, perform an operation on the task, and submit the task back to the process instance to continue executing. To illustrate the process, the presentation tier first extracts the task (using a process id embedded in the Worklist). The task is represented in XML and returned to the presentation tier (see Figure 3).

The presentation tier would then allow the user to operate on the task (XML). Once the user has completed the necessary editing, the presentation tier submits the modified task (XML) back to the process instance to continue. It's important to note here that the presentation tier submits only a modified XML document back to the process, and does not specifically invoke operations within the process itself. This simplifies process maintenance by keeping the presentation tier unaware of the definition of the business process (see Figure 4).

One common theme throughout these design patterns is the use of XML to represent the list of tasks, as well as the tasks themselves. XML provides the most widely accepted means of data interchange and will have the most adaptability for different presentation tiers. However, a significant point of concern is the number of different XML schemas that may be used to support the different business processes within the ESB. To understand the full nature of the problem, let's consider the Worklist interaction points from the Lab Referral Process shown in Figure 1:
1.  When a lab technician makes a "Worklist Summary Request," they might need to know the patient ID, the type of lab work, and the due date.
2.  When a lab technician makes a "Worklist Task" request for more information, they might need to know the name of the patient, the doctor, relevant history for the lab, and some detailed information related to the type of lab.
3.  Finally, when the lab technician submits the task back to the process, they may need to fill out data specific to the type of lab work performed.

In this simple discreet use case, it is clear that there is a need to support a very contextual set of information to the presentation tier to provide meaningful interaction. When one starts considering the numbers of potential business processes and Worklist requirements within a real-world ESB, it begins to highlight some critical technical requirements on the infrastructure to provide a flexible, adaptive environment:

1.  The Worklist Inbox Must Efficiently Manage XML
Imagine the prospect of manually constructing relational database tables and implementing marshaling code to move XML between the tables for every business process on the ESB. The manual effort required to implement the solution would outweigh the benefits of a service-oriented approach.

For this reason, a natural extension to the ESB is a native XML database, allowing XML of any schema to be persisted, indexed, retrieved, queried, and aggregated, without requiring support from a database administrator. The XML database features a schema-less design, allowing multiple XML schemas to co-exist in the database without prior knowledge or administration.

2.  The Worklist Interaction Model Must Support a Standards-Based, Open, Flexible XML Query Mechanism
To accommodate the presentation needs of various business processes, allowing any data to be retrieved and manipulated at the presentation tier, a flexible XML query mechanism should be used to automate integration with the presentation tier. In addition to the sheer numbers of business processes that may need to be accommodated, imagine also the different needs of the presentation tiers. An IVR system may request a subset of the information, while a "thick" .NET client may have the ability to request all of the information.

Luckily, a W3C standard known as XQUERY is being developed for this purpose. An XQUERY facility that natively operates with the ESB's native XML database provides a standards-based approach for different presentation tiers on the ESB to interact with any business process, regardless of the XML schemas in use.

The "Lab Results" service could be quickly integrated with the Avitek Portal by constructing a JSP that invokes the service using XML parameters (the principle would be a useful parameter) and extracts the response in XML. The results are returned in XML and rendered in HTML back to the browser. The Java Standard Tag Library could be used to automate the process of parsing the XML and rendering the results to HTML, as well as any other type of XML- to-HTML mapping utility.

The service invocation mechanism could be any loosely coupled type of mechanism, such as JMS or HTTP/Web services. BEA WebLogic 8.1 provides automatic pooling of JMS connections, making the run-time invocation of JMS-based services extremely efficient and fast.

While the XQUERY statement could certainly be coded directly in the JSP and passed as a parameter to the "Lab Results" service, it may be better to embed the XQUERY into the service itself, providing better abstraction of presentation components from the underlying implementation details (see Listing 1).

Business Activity Monitoring
As critical business processes across the enterprise are moved into a loosely coupled, service-oriented framework on the ESB, new information can be unlocked that can have significant business value. Traditional application implementations typically capture the "end result" of a business process with business intelligence reporting tools. The ESB can capture the intermediate steps of a business process, allowing critical usage patterns to be identified and integrated into decision support systems in real time. Usage patterns can be identified to streamline business processes, or even head off problems before they cause irrevocable damage.

To illustrate the idea of business activity monitoring, let's once again consider the "Lab Referral" process outlined above. In this simple business process, what types of information could be derived?
1.  As doctors refer patients to the lab, hospital administrators may want to be notified if the number of referrals in a given week exceeds the capacity of the lab. This might allow administrators to change work schedules for the lab technicians to accommodate the peak in activity.
2.  As doctors refer patients to the lab, the CDC may want to know if the number of referrals of a certain type exceeds a certain threshold. Certain lab tests may indicate some type of new outbreak that needs to be dealt with quickly.
3.  Administrators may want to monitor overall lab processing times and perform some analysis to optimize processes in the future. For example, they may want to determine the processing time or failure rates for individual lab technicians.

These use cases highlight some core capabilities that can be embedded directly into an ESB: XML data generation, capture, aggregation, and triggering. Once these basic facilities are inplace, the portal can harness these core services to provide interactive dashboards using standard XML facilities such as XQUERY and XSLT. It's important to recognize that there is no "one-size fits all BAM solution." Just as business models differ between organizations, it would stand to reason that the BAM solutions must also differ. The ESB takes the approach of providing a core set of building blocks that provide users with the ability to build their own BAM solutions quickly and in the context of their own business models at a lower cost than dedicated stand-alone platforms. To appreciate the infrastructure behind the portal to support these features, let's take a deeper look at the five layers of BAM.

Generate
In order to support a highly flexible BAM environment, the ESB must provide quick and efficient mechanisms to generate the core data that will feed the BAM message store, poviding business users the ability to perform real-time analytics on the data. There are primarily two sources of BAM data:
1.  Business processes: As business processes execute, the ESB must be able to extract intermediate messages between services in the process, service start and stop times, and process start and stop times. The ESB must also provide the ability to determine information on processes in the aggregate, such as the total, average, and min/max values of critical process data.

This is where ESB providers differ in implementation. The ideal ESB infrastructure will allow process tracking and intermediate data capture without refactoring the existing business processes. Tracking of BAM data should be a transparent deployment-time operation to streamline the process of tracking business data.

2.  Systems on the Bus: As services are invoked within systems on the bus, there may be data internal to the systems that should be communicated out in an event-driven fashion. The ESB provides this feature based on its inherent underpinnings of JMS. As a standards-based asynchronous transport, JMS provides the best (and widely accepted) method of generating events from any system on the bus (see Figure 5).

Capture
Once data is generated from anywhere on the ESB, it must be intelligently routed to BAM Collection Points for real-time analysis. While the ESB provides an efficient means of routing through a distributed infrastructure, it must also provide efficient means of storing XML natively. Again, the concept of a native XML storage mechanism is crucial to a flexible BAM infrastructure. The schema-less design of the native XML storage provides the ability to quickly introduce new types of BAM data without the overhead of marshaling the XML into custom-defined relational tables.

Equally important is the ability to distribute the XML storage anywhere along the ESB. This allows deployment engineers to provide BAM collection points close to the point of origination, streamlining the flow of data traffic in the ESB. Imagine a case where the Lab System (from Figure 5) was generating a lot of event data. If a slow network or a WAN separated the portal and the Lab System, it would be better to capture and aggregate the data locally while also allowing aggregation of BAM data across all of the collection points.

Aggregate/Threshold/Trigger
Once the data is efficiently captured, there must be ways of aggregating the XML data to create useful reports of trends over time. XQUERY and XSLT can be used to aggregate information into summary XML documents, which can subsequently be dashboarded on a portal interface.

Once the data has been aggregated, business rules need to be executed on the aggregated data to determine if critical thresholds have been reached. Again, an XQUERY statement can be run to compare the data against critical business thresholds. If the thresholds are exceeded, the XQUERY can construct a "Business Alert" message to the ESB. Once the alert is on the ESB, it can be routed and handled in any method appropriate. For example, the alert can be routed to an e-mail service to notify one or more users of the event.

Event Processing
As Business Alerts are generated from the BAM Message Stores, they can be routed back to the ESB and handled in arbitrarily complex ways. The alerts may even kick off new, long-running business processes that run over the ESB. One interesting application of business alerts with respect to a portal is the notion of "feedback loops." Feedback loops provide the ability for the portal to modify its own behavior (within a set of constraints) based on external events.

Using the example "Lab Referral" process, imagine the case where the overall lab processing times exceeded a specific time threshold. In this case, the portal could provide additional content informing the doctor of the delay in lab processing times, and possibly even reroute the request to a secondary lab for processing.

Dashboard
Once the BAM data is in an agile XML database, the ability to view the data in a variety of ways on any presentation device (including a portal) becomes important. As executives begin to see the business value the BAM data, the requests for more visualization will increase. There are couple of challenges to consider:
1  An open, generic service architecture is important to integrate with different presentation devices. For this purpose, again the XQUERY and XML is a good choice.
2.  For rapid integration, flexible methods of moving the XML to the presentation layer are required. In the case of a portal, XSLT is a great way to quickly move XML data into HTML visualization.

Conclusion
As the notion of SOA gains traction, Enterprise Portals will be expected to rapidly integrate new services reliably with the portal, providing users with instant access to these services. The ESB will be a critical component of the portal infrastructure, providing scalablity, reliability, and the ability to harness the complexity of very large services networks. However, there are still critical infrastructure services necessary to provide integrated business process management and business activity monitoring. As ESB's gain in popularity, core features such as native XML storage and integrated business process management will be considered standard and necessary features for any portal.

More Stories By Hub Vandervoort

Hub Vandervoort is vice president of Professional Services for Sonic Software. He has over twenty years experience as a consultant and senior technology executive in the networking, communications software, and Internet industries. Vandervoort previously co-founded three start-up ventures, including early message-oriented middleware (MOM) leader Horizon Strategies, Inc., which he merged with Momentum Software Corporation. He also co-founded, and served as board member of the Message-Oriented-Middleware Association (MOMA).

More Stories By Matt Rothera

Matt Rothera is a practice manager at Sonic Software (www.sonicsoftware.com), and works with customers to help plan, design, and deploy real-time, service-oriented architectures. With over 15 years of technical experience, Matt has worked with Fortune 100 customers to integrate their services with internal applications, business partners, and enterprise portals.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@ThingsExpo Stories
The 22nd International Cloud Expo | 1st DXWorld Expo has announced that its Call for Papers is open. Cloud Expo | DXWorld Expo, to be held June 5-7, 2018, at the Javits Center in New York, NY, brings together Cloud Computing, Digital Transformation, Big Data, Internet of Things, DevOps, Machine Learning and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding busin...
Smart cities have the potential to change our lives at so many levels for citizens: less pollution, reduced parking obstacles, better health, education and more energy savings. Real-time data streaming and the Internet of Things (IoT) possess the power to turn this vision into a reality. However, most organizations today are building their data infrastructure to focus solely on addressing immediate business needs vs. a platform capable of quickly adapting emerging technologies to address future ...
Nordstrom is transforming the way that they do business and the cloud is the key to enabling speed and hyper personalized customer experiences. In his session at 21st Cloud Expo, Ken Schow, VP of Engineering at Nordstrom, discussed some of the key learnings and common pitfalls of large enterprises moving to the cloud. This includes strategies around choosing a cloud provider(s), architecture, and lessons learned. In addition, he covered some of the best practices for structured team migration an...
No hype cycles or predictions of a gazillion things here. IoT is here. You get it. You know your business and have great ideas for a business transformation strategy. What comes next? Time to make it happen. In his session at @ThingsExpo, Jay Mason, an Associate Partner of Analytics, IoT & Cybersecurity at M&S Consulting, presented a step-by-step plan to develop your technology implementation strategy. He also discussed the evaluation of communication standards and IoT messaging protocols, data...
With tough new regulations coming to Europe on data privacy in May 2018, Calligo will explain why in reality the effect is global and transforms how you consider critical data. EU GDPR fundamentally rewrites the rules for cloud, Big Data and IoT. In his session at 21st Cloud Expo, Adam Ryan, Vice President and General Manager EMEA at Calligo, examined the regulations and provided insight on how it affects technology, challenges the established rules and will usher in new levels of diligence arou...
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, led attendees through the exciting evolution of the cloud. He looked at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering m...
In his session at 21st Cloud Expo, Raju Shreewastava, founder of Big Data Trunk, provided a fun and simple way to introduce Machine Leaning to anyone and everyone. He solved a machine learning problem and demonstrated an easy way to be able to do machine learning without even coding. Raju Shreewastava is the founder of Big Data Trunk (www.BigDataTrunk.com), a Big Data Training and consulting firm with offices in the United States. He previously led the data warehouse/business intelligence and B...
Recently, REAN Cloud built a digital concierge for a North Carolina hospital that had observed that most patient call button questions were repetitive. In addition, the paper-based process used to measure patient health metrics was laborious, not in real-time and sometimes error-prone. In their session at 21st Cloud Expo, Sean Finnerty, Executive Director, Practice Lead, Health Care & Life Science at REAN Cloud, and Dr. S.P.T. Krishnan, Principal Architect at REAN Cloud, discussed how they built...
22nd International Cloud Expo, taking place June 5-7, 2018, at the Javits Center in New York City, NY, and co-located with the 1st DXWorld Expo will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud ...
22nd International Cloud Expo, taking place June 5-7, 2018, at the Javits Center in New York City, NY, and co-located with the 1st DXWorld Expo will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud ...
DevOps at Cloud Expo – being held June 5-7, 2018, at the Javits Center in New York, NY – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real results. Among the proven benefits,...
@DevOpsSummit at Cloud Expo, taking place June 5-7, 2018, at the Javits Center in New York City, NY, is co-located with 22nd Cloud Expo | 1st DXWorld Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait...
Cloud Expo | DXWorld Expo have announced the conference tracks for Cloud Expo 2018. Cloud Expo will be held June 5-7, 2018, at the Javits Center in New York City, and November 6-8, 2018, at the Santa Clara Convention Center, Santa Clara, CA. Digital Transformation (DX) is a major focus with the introduction of DX Expo within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive ov...
SYS-CON Events announced today that T-Mobile exhibited at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. As America's Un-carrier, T-Mobile US, Inc., is redefining the way consumers and businesses buy wireless services through leading product and service innovation. The Company's advanced nationwide 4G LTE network delivers outstanding wireless experiences to 67.4 million customers who are unwilling to compromise on qua...
SYS-CON Events announced today that Cedexis will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Cedexis is the leader in data-driven enterprise global traffic management. Whether optimizing traffic through datacenters, clouds, CDNs, or any combination, Cedexis solutions drive quality and cost-effectiveness. For more information, please visit https://www.cedexis.com.
SYS-CON Events announced today that Google Cloud has been named “Keynote Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Companies come to Google Cloud to transform their businesses. Google Cloud’s comprehensive portfolio – from infrastructure to apps to devices – helps enterprises innovate faster, scale smarter, stay secure, and do more with data than ever before.
SYS-CON Events announced today that Vivint to exhibit at SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California. As a leading smart home technology provider, Vivint offers home security, energy management, home automation, local cloud storage, and high-speed Internet solutions to more than one million customers throughout the United States and Canada. The end result is a smart home solution that sav...
SYS-CON Events announced today that Opsani will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Opsani is the leading provider of deployment automation systems for running and scaling traditional enterprise applications on container infrastructure.
SYS-CON Events announced today that Nirmata will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Nirmata provides a comprehensive platform, for deploying, operating, and optimizing containerized applications across clouds, powered by Kubernetes. Nirmata empowers enterprise DevOps teams by fully automating the complex operations and management of application containers and its underlying ...
SYS-CON Events announced today that Opsani to exhibit at SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California. Opsani is creating the next generation of automated continuous deployment tools designed specifically for containers. How is continuous deployment different from continuous integration and continuous delivery? CI/CD tools provide build and test. Continuous Deployment is the means by which...