Welcome!

Weblogic Authors: Yeshim Deniz, Elizabeth White, Michael Meiner, Michael Bushong, Avi Rosenthal

Related Topics: Weblogic

Weblogic: Article

Failover and Recovery of Enterprise Applications - Part 1

High availability - moving beyond clustering

WebLogic Platform Deployment Architecture
Before diving into specific details of the strategies used to make applications highly available, let us recap the overview of the WebLogic Platform deployment architecture from the documentation. (http://edocs.bea.com).

The basic administrative unit for a WebLogic Server installation is called a domain. A domain is a logically related group of WebLogic Server resources that are managed as a unit. A domain always includes at least one WebLogic Server instance called the administration server. The administration server serves as a central point of contact for server instances and system administration tools. A domain may also include additional WebLogic Server instances called managed servers.

Administrators can configure some or all of these managed servers to be part of a WebLogic Server cluster. A cluster is a group of WebLogic Server instances that work together to provide scalability and high availability for applications. A managed server in a cluster can act as a backup for services such as JMS and JTA that are hosted on another server instance in the cluster. Applications are also deployed and managed as part of a domain.

Failover Scenario
While the details of the failover and recovery procedures would be covered in the next part of this article, a brief overview of what happens when a server instance in the cluster crashes may be useful. Figure 2 represents the possible activities that happen when a WebLogic instance goes down and all of the services are failed over to another available instance.

As indicated in the diagram, one or more of the following operations may be performed to achieve failover if one of the instances goes down.

  • All of the new incoming HTTP requests will be rerouted to the other available servers in the cluster by the Web server plugins. Web server plugins detect that a server instance has faulted and dynamically update their list of available servers. The subsequent requests are then routed to the appropriate server (mostly the secondary server) from among the available servers.
  • All of the new requests for EJB/RMI will be rerouted to the available servers in the cluster by the EJB/RMI client-side cluster-aware stubs. Essentially, the stubs generated by the EJB/RMI compiler are aware of all of the available servers in the cluster. When a request to a server fails, the stub intercepts the exception and, depending on the type of the exception (such as network exceptions), the stub may redirect the request to any other available servers. In the case of stateless session beans the request may be routed to any available server, while in the case of stateful EJB the stub sends the request to the secondary server, which is the location to which the primary server replicates its state.
  • Internal requests for JMS are routed to other available servers in the cluster. In the case of JMS, if the destination is created as distributed destinations and contains physical members on the any of the available servers, the producer of the messages may continue to send messages without any interruption. The consumers, however, may need to reconnect to available members by incorporating logic within the Exception listeners. In the case of an MDB, the container may provide the logic for reconnecting to the destination.
  • The JMS server can administratively migrate to another available server. The migration of the JMS server assists in bleeding messages from the queue that went down with a downed server instance.
  • Any in-flight transactions are handled as per the JTA specifications. Essentially, the administrator can move the transaction logs from the failed instance to another available instance. The Transaction Manager within the application server then attempts to complete those transactions based on the tlog entries.
Highly Available Deployment Strategies
Having read the high-level introduction about a typical failover scenario, let's now discuss the possible high-availability strategies for deploying applications on the WebLogic Platform. It is no secret to architects that the bottom line in achieving high availability is avoiding single point of failure (SPOF). On the face of it, it appears one would always want to avoid SPOF, but in reality there many constraints in the application design, vendor product architecture, and deployment topology that force the infrastructure with one or more single point of failures. The following sections explain what the infrastructure team should do to avoid these in their deployment architecture.

Hardware Load Balancer
One of the first entry points to the application is via the hardware load balancers. Hardware load balancers have gone beyond simple load balancing/distribution of incoming HTTP traffic and now provide sophisticated algorithms to distribute IP traffic more efficiently, and provide a much higher level of fault tolerance. WebLogic clusters can use any of the sophisticated load balancing/failover algorithms supported by the hardware load balancer. Hardware load balancers are generally more fault resilient than Web server plugins. For a detail description of the capabilities of a hardware load balancer, readers should see the vendor-specific documentation and the BEA documentation on configuring hardware load balancers with WebLogic cluster (http://e-docs.bea.com/wls/docs81/ cluster/load_balancing.html#1026240).

Web Server Farm
In some of the conventional Web facing applications, the Web servers front-end the application servers. Web server runs a Web server plugin that redirects/routes the HTTP traffic. While a single Web server can distribute the traffic to multiple back-end application servers, in this case the Web server itself becomes a single point of failure. Therefore, one of the common strategies to avoid this scenario is to create a Web server farm. Typically, the load balancers are configured to maintain a sticky session with the Web server to which the first request from a given client was routed. In addition, the Web server plugin also maintains stickiness to the server that the first request was routed to. The Web server plugin maintains a list of available back-end servers, along with the primary/secondary pair for that particular client. In case of failure of one of the back-end application servers, the Web server plugin routes the request to the appropriate available server. Regardless of which Web server the request gets routed to, the Web server routes the request to the correct application server by inspecting the cookie in the HTTP header.

More Stories By Sudhir Upadhyay

Sudhir Upadhyay is currently with Architecture and Shared services at JP Morgan Chase where he is an application architect. Prior to joining JPMorgan, he was a principal consultant with BEA Professional Services where he helped customers design and implement enterprise J2EE solutions. He is a BEA Certified WebLogic Developer and a Sun Certified Java Developer.

Comments (2) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
Thirunavukarasu 02/04/06 06:31:00 AM EST

A neatly explained technical atricle. Very useful, very crisp. Thanks .

Viktor Lioutyi 08/12/05 10:34:27 AM EDT

Hi,

How to test failover in automatic manner?

"In several scenarios, administrators end up performing basic failover testing by shutting down the processes and verifying that the subsequent requests succeeded.

Although this level of testing can satisfy the failover requirements for the records, more robust failover testing needs to be performed to ensure a proper recovery if failures do occur."

We did the manual testing and failover worked. But we would like to do automatic testing of failover to make sure that it works for all our 1000+ pages. BEA does not have any tool for such testing.

There are different reasons why someone may want to test all pages for failover.

1) WebLogic only replicates attributes that were modified. Call of session's setAttribute() method is an indication for WebLogic that attribute was modified. This call may be done explicitly or implicitly when jsp tags are used. It is possible that on some pages members of complex attributes were modified but WebLogic was not notified about it, so it will not replicate such attributes.

2) Complex attributes may reference other objects and attributes. After replication these references may be broken. For example, attribute A and B references object C. Only attribute A was modified, so only A will be replicated. After the replication A and B may point to different copies of C and program may not work correctly anymore.

3) Some objects are assumed to be singletons. Developer needs to provide special implementation for serialization to support replication of singleton objects. If this implementation is omitted, then replication may create copies of a singleton object.

4) Transient fields are not going to be replicated but there should be a recovery code that restores values of these fields after replication. Without testing we do not know if all our recovery code works correctly or not.

There are probably other reasons too.

Does anybody know about any tool for automatic testing of failover (or at least just session replication) for WebLogic and/or WebSphere?

Thanks,
Viktor

@ThingsExpo Stories
"Space Monkey by Vivent Smart Home is a product that is a distributed cloud-based edge storage network. Vivent Smart Home, our parent company, is a smart home provider that places a lot of hard drives across homes in North America," explained JT Olds, Director of Engineering, and Brandon Crowfeather, Product Manager, at Vivint Smart Home, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that Conference Guru has been named “Media Sponsor” of the 22nd International Cloud Expo, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. A valuable conference experience generates new contacts, sales leads, potential strategic partners and potential investors; helps gather competitive intelligence and even provides inspiration for new products and services. Conference Guru works with conference organizers to pass great deals to gre...
The Internet of Things will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform. In his session at @ThingsExpo, Craig Sproule, CEO of Metavine, demonstrated how to move beyond today's coding paradigm and shared the must-have mindsets for removing complexity from the develop...
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, led attendees through the exciting evolution of the cloud. He looked at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering m...
"Evatronix provides design services to companies that need to integrate the IoT technology in their products but they don't necessarily have the expertise, knowledge and design team to do so," explained Adam Morawiec, VP of Business Development at Evatronix, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. In his session at @BigDataExpo, Jack Norris, Senior Vice President, Data and Applications at MapR Technologies, reviewed best practices to ...
Widespread fragmentation is stalling the growth of the IIoT and making it difficult for partners to work together. The number of software platforms, apps, hardware and connectivity standards is creating paralysis among businesses that are afraid of being locked into a solution. EdgeX Foundry is unifying the community around a common IoT edge framework and an ecosystem of interoperable components.
Large industrial manufacturing organizations are adopting the agile principles of cloud software companies. The industrial manufacturing development process has not scaled over time. Now that design CAD teams are geographically distributed, centralizing their work is key. With large multi-gigabyte projects, outdated tools have stifled industrial team agility, time-to-market milestones, and impacted P&L stakeholders.
"Akvelon is a software development company and we also provide consultancy services to folks who are looking to scale or accelerate their engineering roadmaps," explained Jeremiah Mothersell, Marketing Manager at Akvelon, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"IBM is really all in on blockchain. We take a look at sort of the history of blockchain ledger technologies. It started out with bitcoin, Ethereum, and IBM evaluated these particular blockchain technologies and found they were anonymous and permissionless and that many companies were looking for permissioned blockchain," stated René Bostic, Technical VP of the IBM Cloud Unit in North America, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Conventi...
In his session at 21st Cloud Expo, Carl J. Levine, Senior Technical Evangelist for NS1, will objectively discuss how DNS is used to solve Digital Transformation challenges in large SaaS applications, CDNs, AdTech platforms, and other demanding use cases. Carl J. Levine is the Senior Technical Evangelist for NS1. A veteran of the Internet Infrastructure space, he has over a decade of experience with startups, networking protocols and Internet infrastructure, combined with the unique ability to it...
22nd International Cloud Expo, taking place June 5-7, 2018, at the Javits Center in New York City, NY, and co-located with the 1st DXWorld Expo will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud ...
"Cloud Academy is an enterprise training platform for the cloud, specifically public clouds. We offer guided learning experiences on AWS, Azure, Google Cloud and all the surrounding methodologies and technologies that you need to know and your teams need to know in order to leverage the full benefits of the cloud," explained Alex Brower, VP of Marketing at Cloud Academy, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clar...
Gemini is Yahoo’s native and search advertising platform. To ensure the quality of a complex distributed system that spans multiple products and components and across various desktop websites and mobile app and web experiences – both Yahoo owned and operated and third-party syndication (supply), with complex interaction with more than a billion users and numerous advertisers globally (demand) – it becomes imperative to automate a set of end-to-end tests 24x7 to detect bugs and regression. In th...
"MobiDev is a software development company and we do complex, custom software development for everybody from entrepreneurs to large enterprises," explained Alan Winters, U.S. Head of Business Development at MobiDev, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, discussed how from store operations and ...
"There's plenty of bandwidth out there but it's never in the right place. So what Cedexis does is uses data to work out the best pathways to get data from the origin to the person who wants to get it," explained Simon Jones, Evangelist and Head of Marketing at Cedexis, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that CrowdReviews.com has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5–7, 2018, at the Javits Center in New York City, NY. CrowdReviews.com is a transparent online platform for determining which products and services are the best based on the opinion of the crowd. The crowd consists of Internet users that have experienced products and services first-hand and have an interest in letting other potential buye...
SYS-CON Events announced today that Telecom Reseller has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
It is of utmost importance for the future success of WebRTC to ensure that interoperability is operational between web browsers and any WebRTC-compliant client. To be guaranteed as operational and effective, interoperability must be tested extensively by establishing WebRTC data and media connections between different web browsers running on different devices and operating systems. In his session at WebRTC Summit at @ThingsExpo, Dr. Alex Gouaillard, CEO and Founder of CoSMo Software, presented ...