Welcome!

Weblogic Authors: Yeshim Deniz, Elizabeth White, Michael Meiner, Michael Bushong, Avi Rosenthal

Related Topics: Weblogic, Java IoT

Weblogic: Article

Approaches to Performance Testing

A best-practices approach to maximize your performance test effort

Now comes the part where you actually run your capacity-planning test. The next question is, "How do I load the users to simulate the load?" The best way to do this is to try to emulate how users hit the server during peak hours. Does that user load happen gradually over a period of time? If so, a ramp-up-style load should be used, where x number of users are added ever y seconds. Or, do all of the users hit the system in a very short period of time all at once? If that is the case, a flat run should be used, where all of the users are simultaneously loaded onto the server. These different styles will produce different results that are not comparable. For instance, if a ramp-up run (Figure 5) is done and you find out that the system can support 5,000 users with a response time of 4 seconds or less, and then you follow that test with a flat run with 5,000 users, you'll probably find that the average response time of the system with 5,000 users is higher than 4 seconds. This is an inherent inaccuracy in ramp-up runs that prevents them from pinpointing the exact number of concurrent users that a system can support. For a portal application, for example, this inaccuracy is amplified as the size of the portal grows and as the size of the cluster is increased.

This is not to say that ramp-up tests should not be used. Ramp-up runs are great if the load on the system is slowly increased over a long period of time. This is because the system will be able to continually adjust over time. If a fast ramp-up is used, the system will lag and artificially report a lower response time than what would be seen if a similar number of users were being loaded during a flat run. So, what is the best way to determine capacity? Taking the best of both load types and running a series of tests will yield the best results. For example, using a ramp-up run to determine the range of users that the system can support should be used first. Then, once that range has been determined, doing a series of flat runs at various concurrent user loads within that range can be used to more accurately determine the capacity of the system (Figure 6).

Soak Tests
A soak test is a straightforward type of performance test. Soak tests are long-duration tests with a static number of concurrent users that test the overall robustness of the system. These tests will show any performance degradations over time via memory leaks, increased garbage collection (GC), or other problems in the system. The longer the test, the more confidence in the system you will have. It is a good idea to run this test twice - once with a fairly moderate user load (but below capacity so that there is no execute queue - Figure 7), and once with a high user load (so that there is a positive execute queue - Figure 8).

These tests should be run for several days to really get a good idea of the long-term health of the application. Make sure that the application being tested is as close to real world as possible with a realistic user scenario (how the virtual users navigate through the application), testing all of the features of the application. Ensure that all the necessary monitoring tools are running so that problems will be accurately detected and tracked down later (Figure 9).

Peak-Rest Tests
Peak-rest tests are a hybrid of the capacity-planning ramp-up-style tests and soak tests. The goal here is to determine how well the system recovers from a high load (such as one during peak hours of the system), goes back to near idle, and then goes back up to peak load and back down again.

The best way to implement this test is to do a series of quick ramp-up tests followed by a plateau (determined by the business requirements), and then a dropping off of the load. A pause in the system should then be used, followed by another quick ramp-up - then you repeat the process. A couple of things can be determined from this: Does the system recover on the second "peak" and each subsequent peak to the same level (or greater) than the first peak? And does the system show any signs of memory or GC degradation over the course of the test? The longer this test is run (repeating the peak/idle cycle over and over), the better idea you'll have of what the long-term health of the system looks like.

Summary
This article has described several approaches to performance testing. Depending on the business requirements, development cycle, and life cycle of the application, some tests will be better suited than others for a given organization. In all cases though, you should ask some fundamental questions before going down one path or another. The answers to these questions will then determine how to best test the application. These questions are:

  • How repeatable do the results need to be?
  • How many times do you want to run and rerun these tests?
  • What stage of the development cycle are you in?
  • What are your business requirements?
  • What are your user requirements?
  • How long do you expect the live production system to stay up between maintenance downtimes?
  • What is the expected user load during an average business day?
By answering these questions and then seeing how the answers fit into the aforementioned performance test types, you should be able to come up with a solid plan for testing the overall performance of your application.

More Stories By Matt Maccaux

Matt Maccaux is a performance engineer on WebLogic Portal at BEA.

Comments (3) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
Deepak Batra 04/26/06 09:08:56 PM EDT

Great article by Matt. It definitely explains the process of WebLogic Performance Tuning & Testing well.

Arcturus (www.arcturustech.com) can help make the process of WebLogic Performance Tuning easy. AutoPilot WL, our flagship product completely automates WebLogic tuning process.

AutoPilot uses a wizard based approach, it takes a load script, generates load and monitors j2ee application & WebLogic's behavior (including transactions, WebLogic threads, ejbs, jms, jdbc, various pools and a lot more..). In the process, AutoPilot analyzes WebLogic & application's cumulative behavior after each run and makes appropriate adjustments to get the optimum performance. AutoPilot also automatically bounces WebLogic server as and when needed in the process. AutoPilot continues this process unless desired goals are achieved or certain user defined thresholds are reached. WebLogic tuning is in a way a misnomer, it is actually aligning WebLogic as per your application and environment's requirements. You can download an evaluation version of AutoPilot at the following url and try it for yourself.

http://support.arcturustech.com/downloadpage.do

You can get more details on WebLogic Performance Tuning Process using AutoPilot at the following URL.

http://support.arcturustech.com/AutoPilot_WL_Help/Tuning_Your_WebLogic_S...

It is recommended by Arcturus to use AutoPilot Advisor before tuning. Advisor analyzes complete application environment for any configuration related issues using its in built knowledge and gives recommendations. It is like attaching your WebLogic environment to a computer that runs a 1000 point inspection on it and either fixes issues or advises how to fix them. Advisor is a great way to find out where there is room for improvement even before you get into Performance Testing & Tuning. It even guides with the relevant BEA CRs that can save an outage situation. We recommend CRs based on their applicability (not blindly).

We also understand no matter how much effort one puts into an application, we are bound to find apllication/weblogic issues that cause WebLogic server to hang. AutoPilot Detector and Blackbox handle those situations and provide you with an instant root cause report. Feel free to drop me an email ([email protected]) if you any questions at all.

Deepak Batra 04/26/06 09:08:21 PM EDT

Great article by Matt. It definitely explains the process of WebLogic Performance Tuning & Testing well.

Arcturus (www.arcturustech.com) can help make the process of WebLogic Performance Tuning easy. AutoPilot WL, our flagship product completely automates WebLogic tuning process.

AutoPilot uses a wizard based approach, it takes a load script, generates load and monitors j2ee application & WebLogic's behavior (including transactions, WebLogic threads, ejbs, jms, jdbc, various pools and a lot more..). In the process, AutoPilot analyzes WebLogic & application's cumulative behavior after each run and makes appropriate adjustments to get the optimum performance. AutoPilot also automatically bounces WebLogic server as and when needed in the process. AutoPilot continues this process unless desired goals are achieved or certain user defined thresholds are reached. WebLogic tuning is in a way a misnomer, it is actually aligning WebLogic as per your application and environment's requirements. You can download an evaluation version of AutoPilot at the following url and try it for yourself.

http://support.arcturustech.com/downloadpage.do

You can get more details on WebLogic Performance Tuning Process using AutoPilot at the following URL.

http://support.arcturustech.com/AutoPilot_WL_Help/Tuning_Your_WebLogic_S...

It is recommended by Arcturus to use AutoPilot Advisor before tuning. Advisor analyzes complete application environment for any configuration related issues using its in built knowledge and gives recommendations. It is like attaching your WebLogic environment to a computer that runs a 1000 point inspection on it and either fixes issues or advises how to fix them. Advisor is a great way to find out where there is room for improvement even before you get into Performance Testing & Tuning. It even guides with the relevant BEA CRs that can save an outage situation. We recommend CRs based on their applicability (not blindly).

We also understand no matter how much effort one puts into an application, we are bound to find apllication/weblogic issues that cause WebLogic server to hang. AutoPilot Detector and Blackbox handle those situations and provide you with an instant root cause report. Feel free to drop me an email ([email protected]) if you any questions at all.

news desk 02/18/06 03:08:31 PM EST

Performance testing a J2EE application can be a daunting and seemingly confusing task if you don't approach it with the proper plan in place. As with any software development process, you must gather requirements, understand the business needs, and lay out a formal schedule well in advance of the actual testing.

@ThingsExpo Stories
Organizations planning enterprise data center consolidation and modernization projects are faced with a challenging, costly reality. Requirements to deploy modern, cloud-native applications simultaneously with traditional client/server applications are almost impossible to achieve with hardware-centric enterprise infrastructure. Compute and network infrastructure are fast moving down a software-defined path, but storage has been a laggard. Until now.
Digital Transformation is much more than a buzzword. The radical shift to digital mechanisms for almost every process is evident across all industries and verticals. This is often especially true in financial services, where the legacy environment is many times unable to keep up with the rapidly shifting demands of the consumer. The constant pressure to provide complete, omnichannel delivery of customer-facing solutions to meet both regulatory and customer demands is putting enormous pressure on...
The best way to leverage your CloudEXPO | DXWorldEXPO presence as a sponsor and exhibitor is to plan your news announcements around our events. The press covering CloudEXPO | DXWorldEXPO will have access to these releases and will amplify your news announcements. More than two dozen Cloud companies either set deals at our shows or have announced their mergers and acquisitions at CloudEXPO. Product announcements during our show provide your company with the most reach through our targeted audienc...
DXWorldEXPO LLC announced today that All in Mobile, a mobile app development company from Poland, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. All In Mobile is a mobile app development company from Poland. Since 2014, they maintain passion for developing mobile applications for enterprises and startups worldwide.
"Akvelon is a software development company and we also provide consultancy services to folks who are looking to scale or accelerate their engineering roadmaps," explained Jeremiah Mothersell, Marketing Manager at Akvelon, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
JETRO showcased Japan Digital Transformation Pavilion at SYS-CON's 21st International Cloud Expo® at the Santa Clara Convention Center in Santa Clara, CA. The Japan External Trade Organization (JETRO) is a non-profit organization that provides business support services to companies expanding to Japan. With the support of JETRO's dedicated staff, clients can incorporate their business; receive visa, immigration, and HR support; find dedicated office space; identify local government subsidies; get...
The current age of digital transformation means that IT organizations must adapt their toolset to cover all digital experiences, beyond just the end users’. Today’s businesses can no longer focus solely on the digital interactions they manage with employees or customers; they must now contend with non-traditional factors. Whether it's the power of brand to make or break a company, the need to monitor across all locations 24/7, or the ability to proactively resolve issues, companies must adapt to...
"We view the cloud not as a specific technology but as a way of doing business and that way of doing business is transforming the way software, infrastructure and services are being delivered to business," explained Matthew Rosen, CEO and Director at Fusion, in this SYS-CON.tv interview at 18th Cloud Expo (http://www.CloudComputingExpo.com), held June 7-9 at the Javits Center in New York City, NY.
DXWorldEXPO LLC announced today that the upcoming DXWorldEXPO | CloudEXPO New York event will feature 10 companies from Poland to participate at the "Poland Digital Transformation Pavilion" on November 12-13, 2018.
As data explodes in quantity, importance and from new sources, the need for managing and protecting data residing across physical, virtual, and cloud environments grow with it. Managing data includes protecting it, indexing and classifying it for true, long-term management, compliance and E-Discovery. Commvault can ensure this with a single pane of glass solution – whether in a private cloud, a Service Provider delivered public cloud or a hybrid cloud environment – across the heterogeneous enter...
DXWorldEXPO LLC announced today that ICC-USA, a computer systems integrator and server manufacturing company focused on developing products and product appliances, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City. ICC is a computer systems integrator and server manufacturing company focused on developing products and product appliances to meet a wide range of ...
More and more brands have jumped on the IoT bandwagon. We have an excess of wearables – activity trackers, smartwatches, smart glasses and sneakers, and more that track seemingly endless datapoints. However, most consumers have no idea what “IoT” means. Creating more wearables that track data shouldn't be the aim of brands; delivering meaningful, tangible relevance to their users should be. We're in a period in which the IoT pendulum is still swinging. Initially, it swung toward "smart for smart...
Major trends and emerging technologies – from virtual reality and IoT, to Big Data and algorithms – are helping organizations innovate in the digital era. However, to create real business value, IT must think beyond the ‘what’ of digital transformation to the ‘how’ to harness emerging trends, innovation and disruption. Architecture is the key that underpins and ties all these efforts together. In the digital age, it’s important to invest in architecture, extend the enterprise footprint to the cl...
Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, discussed how from store operations and ...
Headquartered in Plainsboro, NJ, Synametrics Technologies has provided IT professionals and computer systems developers since 1997. Based on the success of their initial product offerings (WinSQL and DeltaCopy), the company continues to create and hone innovative products that help its customers get more from their computer applications, databases and infrastructure. To date, over one million users around the world have chosen Synametrics solutions to help power their accelerated business or per...
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.
We are seeing a major migration of enterprises applications to the cloud. As cloud and business use of real time applications accelerate, legacy networks are no longer able to architecturally support cloud adoption and deliver the performance and security required by highly distributed enterprises. These outdated solutions have become more costly and complicated to implement, install, manage, and maintain.SD-WAN offers unlimited capabilities for accessing the benefits of the cloud and Internet. ...
In an era of historic innovation fueled by unprecedented access to data and technology, the low cost and risk of entering new markets has leveled the playing field for business. Today, any ambitious innovator can easily introduce a new application or product that can reinvent business models and transform the client experience. In their Day 2 Keynote at 19th Cloud Expo, Mercer Rowe, IBM Vice President of Strategic Alliances, and Raejeanne Skillern, Intel Vice President of Data Center Group and ...
Bill Schmarzo, author of "Big Data: Understanding How Data Powers Big Business" and "Big Data MBA: Driving Business Strategies with Data Science," is responsible for setting the strategy and defining the Big Data service offerings and capabilities for EMC Global Services Big Data Practice. As the CTO for the Big Data Practice, he is responsible for working with organizations to help them identify where and how to start their big data journeys. He's written several white papers, is an avid blogge...
Founded in 2000, Chetu Inc. is a global provider of customized software development solutions and IT staff augmentation services for software technology providers. By providing clients with unparalleled niche technology expertise and industry experience, Chetu has become the premiere long-term, back-end software development partner for start-ups, SMBs, and Fortune 500 companies. Chetu is headquartered in Plantation, Florida, with thirteen offices throughout the U.S. and abroad.