Welcome!

Weblogic Authors: Yeshim Deniz, Elizabeth White, Michael Meiner, Michael Bushong, Avi Rosenthal

Related Topics: Weblogic

Weblogic: Article

Smoothing the Hand-Off to Production

Strategies for Bridging the Expertise Gap

It is late Monday afternoon and your application is finally going into production. After a year of development and months of QA, it will be live first thing Tuesday morning. The next several days will be critical as customers interact with the new for the first time.

It would be nice to think developers can kick back, or even get started on new development, but that's not going to happen. Unexpected performance degradation and errors will likely crop up, and application support staff won't be able to address them alone.

Right now, J2EE applications management in production is a mixed bag. Some companies have an experienced WebLogic administrator or specialized application support staff. But many companies don't, with production management falling to some combination of general IT application support staff and developers (with developers shouldering more of the burden because they generally have more J2EE experience). It's not an effective or efficient production management strategy. Companies need to grow the expertise of application support staff so they can handle most simple production issues without requiring assistance from developers.

Performance Assurance Before Production
As a developer, you do all you can to ensure that your application will run well in production. During development, as you design and implement the application's functionality, you also consider performance goals. Time and resources are a challenge in every project. And while functional testing usually takes precedence over performance testing in development and QA, the benefits of discovering performance issues at this stage should not be overlooked. Allocating time and resources to test performance pays for itself as you avoid downtime and maintain customer loyalty. Every development and test plan should identify performance criteria that need to be met prior to production.

But profiling your application's performance in the lab is not the same as subjecting it to real user load. In the lab, you generally don't have the resources for a truly comprehensive stress test, so you test the application on a small cluster of machines handling a simulated user load. This is helpful to identify transactional and system-wide bottlenecks that are in need of attention, and to design how the application will behave when it is overloaded or unable to access some part of the system. For example, if a data source is not available (resulting in some kind of "Error 500") can you inform the user that a piece of data is not available and allow them to proceed in another direction instead of simply halting abruptly?

However, performance testing during development and QA, while important, will not address every possible performance problem. In production, your application will be subjected to loads and usage patterns that you could not anticipate. Your application support team needs to be prepared for the unexpected.

The Expertise Gap
Once the application is ready for production, responsibility for the application should ideally be handed over to a WebLogic administrator or the IT application support staff. Unfortunately, many companies don't have experienced BEA WebLogic Server administrators, so developers are pressed into service to fill the gap.

The reasons for this expertise gap are logical enough when you think about it. In the early days of a new business application platform, it's the developers who have had the most experience working with it. Only when many applications are in production does the application support personnel have enough experience to build their own expertise.

The application support team has two goals: to ensure that application availability is maintained based on established service levels, and to plan for the application's continued availability in the future as required by the business. We'll focus on the first goal here, but be aware that capacity planning is a future activity that requires the expertise application support teams need to develop.

Support teams need to ramp up quickly and have a process in place to deal with the crisis of the day. The expertise they need to develop is quite different from that of application developers - it's more about tuning the application server environment, understanding the resource dependencies of J2EE, and configuring the application servers.

Strategies for Handing Off to Production
Developers can either passively wait to be drawn into firefighting minor performance issues, or play a proactive role in helping the application support team learn to maintain the availability of the application. The amount of support required varies with the size and structure of the team, and the depth to which they are expected to diagnose problems. As WebLogic administration staff gain experience and expertise, they will in time become the experts in running your application as optimally as possible in production.

As part of the transition into production, the development team should produce a support document describing the high-level functionality and dependencies of the application - an overview of the application from a systems administration perspective, if you will. This starting point should give even inexperienced support staff a place to start learning about the application. It should document each tier of the application and external dependencies, and include a list of key error log messages.

While your code resides solely on the BEA WebLogic Server, many other servers provide data and functionality to your application. The support document should outline all Web servers, components developed outside your team, and all back-end databases used by the application for data and customer authentication. A diagram depicting the source of the data and the type of communication used between your system and the external data source would be useful. A good description of all the moving parts in your J2EE environment will help to get them up to speed quickly.

To ensure a stable environment, the WebLogic administrator should become the gatekeeper for all application and configuration changes made to the WebLogic Servers in production. To prepare the WebLogic administrator to manage the server, developers should update the support document with a list of applications that need to be deployed to each server, expected response times, and a list of hotspots and potential bottlenecks for the server and applications.

Once provided with this roadmap, the application support team needs to be able to see how it all works in practice. This is one area in which tools can be beneficial.

Tools to Inform and Educate
As a developer, you have many tools available to help you create and test a large application - basic tools like editors, debuggers, and source control tools; and tools that enhance productivity and enable you to do your job better, such as profilers, memory debuggers, and automated testing tools. Application support teams need their own specialized tools to configure, monitor, and diagnose problems with the application in production. Ideally, such tools not only make for more efficient management, but also educate and help develop expertise.

Tools exist for anything from 24x7 application monitoring to deep problem diagnostics tools for code and database issues. But in terms of helping application support teams develop their J2EE expertise, a good place to start is in the area of WebLogic Server management.

The BEA WebLogic Server cluster is really the front door to the application. It runs the code, pools access to the back-end databases, and manages and coordinates use of the shared resources of the system. To help manage the WebLogic Servers, application support should choose a tool that presents information in a well-organized, real-time interface; highlights trouble areas; and provides insight into the cause of problems (see Figure 1).

 

Tools that provide a domain overview help isolate whether performance issues are specific to one server, or spread across the entire cluster. With the status of each WebLogic Server, the clusters, and each deployed application, the administrator can quickly determine the extent of the performance outage. This initial triage allows the WebLogic administrator to focus on the problem quickly and provides an initial idea about the impact of the situation on customers.

Presentation of information is a key factor in enabling inexperienced administrators to determine the health of their J2EE systems. There are hundreds of metrics available inside WebLogic Server, each a piece of the performance puzzle. Individually these metrics aren't that useful, but when you are able to visualize the relationships, the performance picture comes into focus, and diagnosing the problem is possible. Tools like the BEA WebLogic Console and command-line interfaces that allow you to browse through the metrics are good if you know what the problem is and may even help WebLogic experts find problems. However, while most WebLogic administrators will rely on the console to make changes to a server configuration, they typically need a more intuitive tool to help find which configuration parameter actually needs to be adjusted.

The best way to bring the WebLogic administrator up to speed on the internals of the WebLogic Application Server is to provide them with a real-time performance view of it. Learning an application and finding performance problems is much easier if you can see how the application and WebLogic Server interact. To see the resources usage fluctuate with the process flow between WebLogic components as customer traffic varies is a powerful educational tool. Once visual alarms are added to the picture, the WebLogic administrator is able to detect and diagnose many problems instead of calling development for assistance. Visual alarms should highlight bottlenecks and indicate where the admin should dive deeper into the application or WebLogic server. A picture is worth a thousand words and a real-time performance picture will save hours of investigation.

 

Another useful capability to look for in an application server management tool is built-in expert advice. This information enables the WebLogic administrator to make informed decisions about the problem and options for resolution. Development teams may still need to be consulted before the administrator takes corrective actions but the availability of context-sensitive advice will improve the decision-making process within the application support team. The right diagnostic tool is critical whether you are a WebLogic veteran or just starting to manage J2EE applications, and integrated expert advice will improve the confidence of the veterans and allow some tasks to be off-loaded to junior staff.

Conclusion
Running J2EE applications in production takes expertise that isn't widely available yet. Developers are the most qualified to diagnose performance and scalability issues at the moment, but instead of fighting fires in production, they should be concentrating on helping application support staff develop the expertise they need to manage their applications.

Simple application documentation is one strategy that will help support teams maintain the availability of the application in production. And to plan future capacity, empirical data collected from the production application is needed. For day-to-day management (and education), a real-time performance viewing tool is ideal, one that can show the live application in operation and help gather data for capacity planning.

More Stories By Hugh Docherty

Hugh Docherty is a product manager for Quest Software’s J2EE performance management solutions. Hugh has extensive experience managing development of web-based financial applications, on top of ten years experience building and supporting web applications.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@ThingsExpo Stories
SYS-CON Events announced today that Dasher Technologies will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Dasher Technologies, Inc. ® is a premier IT solution provider that delivers expert technical resources along with trusted account executives to architect and deliver complete IT solutions and services to help our clients execute their goals, plans and objectives. Since 1999, we'v...
Enterprises have taken advantage of IoT to achieve important revenue and cost advantages. What is less apparent is how incumbent enterprises operating at scale have, following success with IoT, built analytic, operations management and software development capabilities – ranging from autonomous vehicles to manageable robotics installations. They have embraced these capabilities as if they were Silicon Valley startups. As a result, many firms employ new business models that place enormous impor...
SYS-CON Events announced today that Massive Networks, that helps your business operate seamlessly with fast, reliable, and secure internet and network solutions, has been named "Exhibitor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. As a premier telecommunications provider, Massive Networks is headquartered out of Louisville, Colorado. With years of experience under their belt, their team of...
SYS-CON Events announced today that TidalScale, a leading provider of systems and services, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TidalScale has been involved in shaping the computing landscape. They've designed, developed and deployed some of the most important and successful systems and services in the history of the computing industry - internet, Ethernet, operating s...
SYS-CON Events announced today that Taica will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Taica manufacturers Alpha-GEL brand silicone components and materials, which maintain outstanding performance over a wide temperature range -40C to +200C. For more information, visit http://www.taica.co.jp/english/.
SYS-CON Events announced today that MIRAI Inc. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MIRAI Inc. are IT consultants from the public sector whose mission is to solve social issues by technology and innovation and to create a meaningful future for people.
SYS-CON Events announced today that IBM has been named “Diamond Sponsor” of SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California.
SYS-CON Events announced today that TidalScale will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TidalScale is the leading provider of Software-Defined Servers that bring flexibility to modern data centers by right-sizing servers on the fly to fit any data set or workload. TidalScale’s award-winning inverse hypervisor technology combines multiple commodity servers (including their ass...
Join IBM November 1 at 21st Cloud Expo at the Santa Clara Convention Center in Santa Clara, CA, and learn how IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Cognitive analysis impacts today’s systems with unparalleled ability that were previously available only to manned, back-end operations. Thanks to cloud processing, IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Imagine a robot vacuum that becomes your personal assistant tha...
Widespread fragmentation is stalling the growth of the IIoT and making it difficult for partners to work together. The number of software platforms, apps, hardware and connectivity standards is creating paralysis among businesses that are afraid of being locked into a solution. EdgeX Foundry is unifying the community around a common IoT edge framework and an ecosystem of interoperable components.
As popularity of the smart home is growing and continues to go mainstream, technological factors play a greater role. The IoT protocol houses the interoperability battery consumption, security, and configuration of a smart home device, and it can be difficult for companies to choose the right kind for their product. For both DIY and professionally installed smart homes, developers need to consider each of these elements for their product to be successful in the market and current smart homes.
Infoblox delivers Actionable Network Intelligence to enterprise, government, and service provider customers around the world. They are the industry leader in DNS, DHCP, and IP address management, the category known as DDI. We empower thousands of organizations to control and secure their networks from the core-enabling them to increase efficiency and visibility, improve customer service, and meet compliance requirements.
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
SYS-CON Events announced today that mruby Forum will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. mruby is the lightweight implementation of the Ruby language. We introduce mruby and the mruby IoT framework that enhances development productivity. For more information, visit http://forum.mruby.org/.
Digital transformation is changing the face of business. The IDC predicts that enterprises will commit to a massive new scale of digital transformation, to stake out leadership positions in the "digital transformation economy." Accordingly, attendees at the upcoming Cloud Expo | @ThingsExpo at the Santa Clara Convention Center in Santa Clara, CA, Oct 31-Nov 2, will find fresh new content in a new track called Enterprise Cloud & Digital Transformation.
Most technology leaders, contemporary and from the hardware era, are reshaping their businesses to do software. They hope to capture value from emerging technologies such as IoT, SDN, and AI. Ultimately, irrespective of the vertical, it is about deriving value from independent software applications participating in an ecosystem as one comprehensive solution. In his session at @ThingsExpo, Kausik Sridhar, founder and CTO of Pulzze Systems, will discuss how given the magnitude of today's applicati...
Smart cities have the potential to change our lives at so many levels for citizens: less pollution, reduced parking obstacles, better health, education and more energy savings. Real-time data streaming and the Internet of Things (IoT) possess the power to turn this vision into a reality. However, most organizations today are building their data infrastructure to focus solely on addressing immediate business needs vs. a platform capable of quickly adapting emerging technologies to address future ...
SYS-CON Events announced today that NetApp has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. NetApp is the data authority for hybrid cloud. NetApp provides a full range of hybrid cloud data services that simplify management of applications and data across cloud and on-premises environments to accelerate digital transformation. Together with their partners, NetApp emp...
In a recent survey, Sumo Logic surveyed 1,500 customers who employ cloud services such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). According to the survey, a quarter of the respondents have already deployed Docker containers and nearly as many (23 percent) are employing the AWS Lambda serverless computing framework. It’s clear: serverless is here to stay. The adoption does come with some needed changes, within both application development and operations. Tha...
SYS-CON Events announced today that Avere Systems, a leading provider of enterprise storage for the hybrid cloud, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Avere delivers a more modern architectural approach to storage that doesn't require the overprovisioning of storage capacity to achieve performance, overspending on expensive storage media for inactive data or the overbui...