Weblogic Authors: Yeshim Deniz, Elizabeth White, Michael Meiner, Michael Bushong, Avi Rosenthal

Related Topics: Weblogic

Weblogic: Article

Failover and Recovery of Enterprise Applications - Part 1

High availability - moving beyond clustering

WebLogic Platform Deployment Architecture
Before diving into specific details of the strategies used to make applications highly available, let us recap the overview of the WebLogic Platform deployment architecture from the documentation. (http://edocs.bea.com).

The basic administrative unit for a WebLogic Server installation is called a domain. A domain is a logically related group of WebLogic Server resources that are managed as a unit. A domain always includes at least one WebLogic Server instance called the administration server. The administration server serves as a central point of contact for server instances and system administration tools. A domain may also include additional WebLogic Server instances called managed servers.

Administrators can configure some or all of these managed servers to be part of a WebLogic Server cluster. A cluster is a group of WebLogic Server instances that work together to provide scalability and high availability for applications. A managed server in a cluster can act as a backup for services such as JMS and JTA that are hosted on another server instance in the cluster. Applications are also deployed and managed as part of a domain.

Failover Scenario
While the details of the failover and recovery procedures would be covered in the next part of this article, a brief overview of what happens when a server instance in the cluster crashes may be useful. Figure 2 represents the possible activities that happen when a WebLogic instance goes down and all of the services are failed over to another available instance.

As indicated in the diagram, one or more of the following operations may be performed to achieve failover if one of the instances goes down.

  • All of the new incoming HTTP requests will be rerouted to the other available servers in the cluster by the Web server plugins. Web server plugins detect that a server instance has faulted and dynamically update their list of available servers. The subsequent requests are then routed to the appropriate server (mostly the secondary server) from among the available servers.
  • All of the new requests for EJB/RMI will be rerouted to the available servers in the cluster by the EJB/RMI client-side cluster-aware stubs. Essentially, the stubs generated by the EJB/RMI compiler are aware of all of the available servers in the cluster. When a request to a server fails, the stub intercepts the exception and, depending on the type of the exception (such as network exceptions), the stub may redirect the request to any other available servers. In the case of stateless session beans the request may be routed to any available server, while in the case of stateful EJB the stub sends the request to the secondary server, which is the location to which the primary server replicates its state.
  • Internal requests for JMS are routed to other available servers in the cluster. In the case of JMS, if the destination is created as distributed destinations and contains physical members on the any of the available servers, the producer of the messages may continue to send messages without any interruption. The consumers, however, may need to reconnect to available members by incorporating logic within the Exception listeners. In the case of an MDB, the container may provide the logic for reconnecting to the destination.
  • The JMS server can administratively migrate to another available server. The migration of the JMS server assists in bleeding messages from the queue that went down with a downed server instance.
  • Any in-flight transactions are handled as per the JTA specifications. Essentially, the administrator can move the transaction logs from the failed instance to another available instance. The Transaction Manager within the application server then attempts to complete those transactions based on the tlog entries.
Highly Available Deployment Strategies
Having read the high-level introduction about a typical failover scenario, let's now discuss the possible high-availability strategies for deploying applications on the WebLogic Platform. It is no secret to architects that the bottom line in achieving high availability is avoiding single point of failure (SPOF). On the face of it, it appears one would always want to avoid SPOF, but in reality there many constraints in the application design, vendor product architecture, and deployment topology that force the infrastructure with one or more single point of failures. The following sections explain what the infrastructure team should do to avoid these in their deployment architecture.

Hardware Load Balancer
One of the first entry points to the application is via the hardware load balancers. Hardware load balancers have gone beyond simple load balancing/distribution of incoming HTTP traffic and now provide sophisticated algorithms to distribute IP traffic more efficiently, and provide a much higher level of fault tolerance. WebLogic clusters can use any of the sophisticated load balancing/failover algorithms supported by the hardware load balancer. Hardware load balancers are generally more fault resilient than Web server plugins. For a detail description of the capabilities of a hardware load balancer, readers should see the vendor-specific documentation and the BEA documentation on configuring hardware load balancers with WebLogic cluster (http://e-docs.bea.com/wls/docs81/ cluster/load_balancing.html#1026240).

Web Server Farm
In some of the conventional Web facing applications, the Web servers front-end the application servers. Web server runs a Web server plugin that redirects/routes the HTTP traffic. While a single Web server can distribute the traffic to multiple back-end application servers, in this case the Web server itself becomes a single point of failure. Therefore, one of the common strategies to avoid this scenario is to create a Web server farm. Typically, the load balancers are configured to maintain a sticky session with the Web server to which the first request from a given client was routed. In addition, the Web server plugin also maintains stickiness to the server that the first request was routed to. The Web server plugin maintains a list of available back-end servers, along with the primary/secondary pair for that particular client. In case of failure of one of the back-end application servers, the Web server plugin routes the request to the appropriate available server. Regardless of which Web server the request gets routed to, the Web server routes the request to the correct application server by inspecting the cookie in the HTTP header.

More Stories By Sudhir Upadhyay

Sudhir Upadhyay is currently with Architecture and Shared services at JP Morgan Chase where he is an application architect. Prior to joining JPMorgan, he was a principal consultant with BEA Professional Services where he helped customers design and implement enterprise J2EE solutions. He is a BEA Certified WebLogic Developer and a Sun Certified Java Developer.

Comments (2) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

Most Recent Comments
Thirunavukarasu 02/04/06 06:31:00 AM EST

A neatly explained technical atricle. Very useful, very crisp. Thanks .

Viktor Lioutyi 08/12/05 10:34:27 AM EDT


How to test failover in automatic manner?

"In several scenarios, administrators end up performing basic failover testing by shutting down the processes and verifying that the subsequent requests succeeded.

Although this level of testing can satisfy the failover requirements for the records, more robust failover testing needs to be performed to ensure a proper recovery if failures do occur."

We did the manual testing and failover worked. But we would like to do automatic testing of failover to make sure that it works for all our 1000+ pages. BEA does not have any tool for such testing.

There are different reasons why someone may want to test all pages for failover.

1) WebLogic only replicates attributes that were modified. Call of session's setAttribute() method is an indication for WebLogic that attribute was modified. This call may be done explicitly or implicitly when jsp tags are used. It is possible that on some pages members of complex attributes were modified but WebLogic was not notified about it, so it will not replicate such attributes.

2) Complex attributes may reference other objects and attributes. After replication these references may be broken. For example, attribute A and B references object C. Only attribute A was modified, so only A will be replicated. After the replication A and B may point to different copies of C and program may not work correctly anymore.

3) Some objects are assumed to be singletons. Developer needs to provide special implementation for serialization to support replication of singleton objects. If this implementation is omitted, then replication may create copies of a singleton object.

4) Transient fields are not going to be replicated but there should be a recovery code that restores values of these fields after replication. Without testing we do not know if all our recovery code works correctly or not.

There are probably other reasons too.

Does anybody know about any tool for automatic testing of failover (or at least just session replication) for WebLogic and/or WebSphere?


@ThingsExpo Stories
"There's plenty of bandwidth out there but it's never in the right place. So what Cedexis does is uses data to work out the best pathways to get data from the origin to the person who wants to get it," explained Simon Jones, Evangelist and Head of Marketing at Cedexis, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Digital Transformation and Disruption, Amazon Style - What You Can Learn. Chris Kocher is a co-founder of Grey Heron, a management and strategic marketing consulting firm. He has 25+ years in both strategic and hands-on operating experience helping executives and investors build revenues and shareholder value. He has consulted with over 130 companies on innovating with new business models, product strategies and monetization. Chris has held management positions at HP and Symantec in addition to ...
Enterprises have taken advantage of IoT to achieve important revenue and cost advantages. What is less apparent is how incumbent enterprises operating at scale have, following success with IoT, built analytic, operations management and software development capabilities - ranging from autonomous vehicles to manageable robotics installations. They have embraced these capabilities as if they were Silicon Valley startups.
In their session at @ThingsExpo, Shyam Varan Nath, Principal Architect at GE, and Ibrahim Gokcen, who leads GE's advanced IoT analytics, focused on the Internet of Things / Industrial Internet and how to make it operational for business end-users. Learn about the challenges posed by machine and sensor data and how to marry it with enterprise data. They also discussed the tips and tricks to provide the Industrial Internet as an end-user consumable service using Big Data Analytics and Industrial C...
René Bostic is the Technical VP of the IBM Cloud Unit in North America. Enjoying her career with IBM during the modern millennial technological era, she is an expert in cloud computing, DevOps and emerging cloud technologies such as Blockchain. Her strengths and core competencies include a proven record of accomplishments in consensus building at all levels to assess, plan, and implement enterprise and cloud computing solutions. René is a member of the Society of Women Engineers (SWE) and a m...
When talking IoT we often focus on the devices, the sensors, the hardware itself. The new smart appliances, the new smart or self-driving cars (which are amalgamations of many ‘things'). When we are looking at the world of IoT, we should take a step back, look at the big picture. What value are these devices providing. IoT is not about the devices, its about the data consumed and generated. The devices are tools, mechanisms, conduits. This paper discusses the considerations when dealing with the...
DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City. Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term.
To Really Work for Enterprises, MultiCloud Adoption Requires Far Better and Inclusive Cloud Monitoring and Cost Management … But How? Overwhelmingly, even as enterprises have adopted cloud computing and are expanding to multi-cloud computing, IT leaders remain concerned about how to monitor, manage and control costs across hybrid and multi-cloud deployments. It’s clear that traditional IT monitoring and management approaches, designed after all for on-premises data centers, are falling short in ...
With privacy often voiced as the primary concern when using cloud based services, SyncriBox was designed to ensure that the software remains completely under the customer's control. Having both the source and destination files remain under the user?s control, there are no privacy or security issues. Since files are synchronized using Syncrify Server, no third party ever sees these files.
Cloud-enabled transformation has evolved from cost saving measure to business innovation strategy -- one that combines the cloud with cognitive capabilities to drive market disruption. Learn how you can achieve the insight and agility you need to gain a competitive advantage. Industry-acclaimed CTO and cloud expert, Shankar Kalyana presents. Only the most exceptional IBMers are appointed with the rare distinction of IBM Fellow, the highest technical honor in the company. Shankar has also receive...
In his session at 21st Cloud Expo, Carl J. Levine, Senior Technical Evangelist for NS1, will objectively discuss how DNS is used to solve Digital Transformation challenges in large SaaS applications, CDNs, AdTech platforms, and other demanding use cases. Carl J. Levine is the Senior Technical Evangelist for NS1. A veteran of the Internet Infrastructure space, he has over a decade of experience with startups, networking protocols and Internet infrastructure, combined with the unique ability to it...
"Cloud Academy is an enterprise training platform for the cloud, specifically public clouds. We offer guided learning experiences on AWS, Azure, Google Cloud and all the surrounding methodologies and technologies that you need to know and your teams need to know in order to leverage the full benefits of the cloud," explained Alex Brower, VP of Marketing at Cloud Academy, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clar...
Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settlement products to hedge funds and investment banks. After, he co-founded a revenue cycle management company where he learned about Bitcoin and eventually Ethereal. Andrew's role at ConsenSys Enterprise is a mul...
Internet-of-Things discussions can end up either going down the consumer gadget rabbit hole or focused on the sort of data logging that industrial manufacturers have been doing forever. However, in fact, companies today are already using IoT data both to optimize their operational technology and to improve the experience of customer interactions in novel ways. In his session at @ThingsExpo, Gordon Haff, Red Hat Technology Evangelist, shared examples from a wide range of industries – including en...
"Space Monkey by Vivent Smart Home is a product that is a distributed cloud-based edge storage network. Vivent Smart Home, our parent company, is a smart home provider that places a lot of hard drives across homes in North America," explained JT Olds, Director of Engineering, and Brandon Crowfeather, Product Manager, at Vivint Smart Home, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Rodrigo Coutinho is part of OutSystems' founders' team and currently the Head of Product Design. He provides a cross-functional role where he supports Product Management in defining the positioning and direction of the Agile Platform, while at the same time promoting model-based development and new techniques to deliver applications in the cloud.
DevOpsSummit New York 2018, colocated with CloudEXPO | DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City. Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of bus...
delaPlex is a global technology and software development solutions and consulting provider, deeply committed to helping companies drive growth, revenue and marketplace value. Since 2008, delaPlex's objective has been to be a trusted advisor to its clients. By redefining the outsourcing industry's business model, the innovative delaPlex Agile Business Framework brings an unmatched alliance of industry experts, across industries and functional skillsets, to clients anywhere around the world.
Business professionals no longer wonder if they'll migrate to the cloud; it's now a matter of when. The cloud environment has proved to be a major force in transitioning to an agile business model that enables quick decisions and fast implementation that solidify customer relationships. And when the cloud is combined with the power of cognitive computing, it drives innovation and transformation that achieves astounding competitive advantage.
Headquartered in Plainsboro, NJ, Synametrics Technologies has provided IT professionals and computer systems developers since 1997. Based on the success of their initial product offerings (WinSQL and DeltaCopy), the company continues to create and hone innovative products that help its customers get more from their computer applications, databases and infrastructure. To date, over one million users around the world have chosen Synametrics solutions to help power their accelerated business or per...