Welcome!

Weblogic Authors: Yeshim Deniz, Elizabeth White, Michael Meiner, Michael Bushong, Avi Rosenthal

Related Topics: Weblogic

Weblogic: Article

WebSphere vs WebLogic: IBM and BEA Spar Over SPEC Results

SPECjAppServer2004 Results Published, But What Do They Mean?

Benchmarks can mean whatever you want them to mean, it has always seemed. Although useful as a rough guide to performance, and sometimes price/performance, technology companies are famous for interpreting complex benchmark results as victories over their competition and them employing visual aids such as planes, snails, and automobiles to demonstrate their point.

The latest benchmark brouhaha may be over the newly developed SPECjAppServer2004 benchmark from SPEC (the Warrenton, VA-based Standard Performance Evaluation Corp.) This benchmark is the latest in the "Java Client/Server" category, and replaces SPECjAppServer 2002, which has now been retired. It "is designed to measure the performance of J2EE 1.3 application servers, (and) includes an enhanced workload by adding a web tier, JMS, and other changes to SPECjAppServer2002," according to SPEC.

And it has lent itself to spectacular success by IBM, according to a recent release coming out of Somers, NY. In fact, IBM's WebSphere Application Server "outperformed the competition by 64 percent," according to the release. Competitor BEA was quick to dispute this claim, however, pointing to its own success with its WebLogic on this benchmark.

IBM executive Sandy Carter said "these latest results further demonstrate the industry leading capability of IBM WebSphere Application Server," said Sandy Carter, Vice President, IBM WebSphere Software. And the IBM release went onto say that the "results submission involved more than 22,000 concurrent clients and produced more than 2,921 complex business transactions per second. The IBM submission represented a complete IBM solution bringing together the latest version of WebSphere Application Server software, DB2 Universal Database and IBM System p5 550 servers running SUSE Linux. This powerful solution outperformed a combination of BEA and Oracle software running on Sun Solaris and at significantly lower total cost of ownership across the full configuration."

BEA executive Eric Stahl , the company's Senior Director of Investor Relations, begs to differ. He referred to IBM's results as "inefficient," and has developed charts that show WebLogic as the clear winner in this contest. In his blog, Stahl says "the new (SPECjAppServer2004) result from BEA sets a new scalability world record, beating the previous record which was already held by BEA," and he provides a chart in his blog to illustrate his point.

Neither Oracle nor Sun had a comment about the IBM release by the time this story was being posted. Oracle has also not published its results on this benchmark, although Sun has. The results table is available to anyone at the SPEC website.

Which leads to the price/performance question. Unlike the 2002 benchmark, the new one does not include a price/performance metric, so ladies and gentlement, please re-start your engines. IBM says it based its 64% margin of victory on the following comparison:

IBM/WAS/DB2
AppServer Hardware: $192,000
AppServer Software: $480,000
Database Hardware: $377,000
Database Software: $181,000
Total = $1,230,000
Total $/JOPS = $1,230,000/2921 JOPS = $421/JOP

SUN/BEA/Oracle
AppServer Hardware: $38,000
AppServer Software: $100,000
Database Hardware: $752,000
Database Software: $566,000
Total = $1,456,000
Total $/JOPS = $1,456,000/1781JOPS = $817/JOP

An IBM spokesperson stressed the importance of the $/JOP figure, which "gives customers an idea of the cost per transaction of the system setup and represents their TCO directly. Basically we achieved 64% better throughput at nearly 1/2 the cost of the Sun/BEA/Oracle benchmark solution."

Stahl, for his part, said IBM's computation makes it clear "that (BEA's WebLogic) is vastly cheaper than (IBM's) WebSphere. (BEA) only ate up $100k in app server software licenses vs. IBM needing $480K in application server software licenses. That's 4.8x the cost for only 64% more throughput. (Furthermore, BEA) only consumed $38,000 in hardware on the app server tier to their $192,000. That's more than 5x the cost for only 64% more throughput."

Stahl also said, "when you isolate the app server software and hardware cost, which is the only part that is relevant for us, you get the following:

BEA WebLogic: $100k + $38k = $138k @ 1781 JOPS = $77.48/JOPS
IBM WebSphere: $480k + $192k= $672k @ 2921 JOPS = $230/JOPS"

Stahl also said that the difference in IBM's calculations stems from the database cost, "and that's an IBM DB2 vs. Oracle issue..WebLogic Server runs on IBM hardware, SuSE Linux and DB2 as well as WebSphere does. We have thousands of customers using that configuration."

BEA has its own competitive battles with Oracle in this market as well, and the company is on record as saying that it thinks Oracle's database is overpriced.

IBM has convinced enough customers of its advantages to have steadily been gaining the upper hand in this market, gaining several points of market share from BEA (and others), even achieving what's been called a "slam dunk" by analyst Gartner Corp. Yet BEA, if the company's response to this latest news from IBM is any indication, shows no signs of backing off in developing and marketing its competing environment in a fast-growing market.

Both companies have been on the move recently, IBM announcing a new broad-based SOA strategy, and BEA acquiring transactional software company Plumtree, Eclipse-centric M7, and Java EE 5 persistence company Solarmetric.

Benchmark wars can appear to be risible at times, as the complexity of the benchmark process easily leads to companies emphasizing only those parts that cast their products in the best possible light. The result can seem to provide no illumination at all, when two companies can use the same figures to demonstrate opposing viewpoints.

Yet, certainly SPECmarks always have, and do, provide a valuable service in that they keep technology providers' noses to the grindstone to keep creating faster and more efficient hardware, software, and solutions for their customers.

Thus, it seems healthy for the industry to have heavyweights such as IBM and BEA trade verbal punches on a regular basis. Both companies (along with Sun, Oracle, Microsoft, and JBoss) will be represented during SYS-CON's AppServer Shootout Webcast from New York on November 18, and maybe their dialog over the relative strengths of their platforms can continue.

More Stories By Roger Strukhoff

Roger Strukhoff (@IoT2040) is Executive Director of the Tau Institute for Global ICT Research, with offices in Illinois and Manila. He is Conference Chair of @CloudExpo & @ThingsExpo, and Editor of SYS-CON Media's CloudComputing BigData & IoT Journals. He holds a BA from Knox College & conducted MBA studies at CSU-East Bay.

Comments (2)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


IoT & Smart Cities Stories
Moroccanoil®, the global leader in oil-infused beauty, is thrilled to announce the NEW Moroccanoil Color Depositing Masks, a collection of dual-benefit hair masks that deposit pure pigments while providing the treatment benefits of a deep conditioning mask. The collection consists of seven curated shades for commitment-free, beautifully-colored hair that looks and feels healthy.
The textured-hair category is inarguably the hottest in the haircare space today. This has been driven by the proliferation of founder brands started by curly and coily consumers and savvy consumers who increasingly want products specifically for their texture type. This trend is underscored by the latest insights from NaturallyCurly's 2018 TextureTrends report, released today. According to the 2018 TextureTrends Report, more than 80 percent of women with curly and coily hair say they purcha...
The textured-hair category is inarguably the hottest in the haircare space today. This has been driven by the proliferation of founder brands started by curly and coily consumers and savvy consumers who increasingly want products specifically for their texture type. This trend is underscored by the latest insights from NaturallyCurly's 2018 TextureTrends report, released today. According to the 2018 TextureTrends Report, more than 80 percent of women with curly and coily hair say they purcha...
We all love the many benefits of natural plant oils, used as a deap treatment before shampooing, at home or at the beach, but is there an all-in-one solution for everyday intensive nutrition and modern styling?I am passionate about the benefits of natural extracts with tried-and-tested results, which I have used to develop my own brand (lemon for its acid ph, wheat germ for its fortifying action…). I wanted a product which combined caring and styling effects, and which could be used after shampo...
The platform combines the strengths of Singtel's extensive, intelligent network capabilities with Microsoft's cloud expertise to create a unique solution that sets new standards for IoT applications," said Mr Diomedes Kastanis, Head of IoT at Singtel. "Our solution provides speed, transparency and flexibility, paving the way for a more pervasive use of IoT to accelerate enterprises' digitalisation efforts. AI-powered intelligent connectivity over Microsoft Azure will be the fastest connected pat...
There are many examples of disruption in consumer space – Uber disrupting the cab industry, Airbnb disrupting the hospitality industry and so on; but have you wondered who is disrupting support and operations? AISERA helps make businesses and customers successful by offering consumer-like user experience for support and operations. We have built the world’s first AI-driven IT / HR / Cloud / Customer Support and Operations solution.
Codete accelerates their clients growth through technological expertise and experience. Codite team works with organizations to meet the challenges that digitalization presents. Their clients include digital start-ups as well as established enterprises in the IT industry. To stay competitive in a highly innovative IT industry, strong R&D departments and bold spin-off initiatives is a must. Codete Data Science and Software Architects teams help corporate clients to stay up to date with the mod...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
Druva is the global leader in Cloud Data Protection and Management, delivering the industry's first data management-as-a-service solution that aggregates data from endpoints, servers and cloud applications and leverages the public cloud to offer a single pane of glass to enable data protection, governance and intelligence-dramatically increasing the availability and visibility of business critical information, while reducing the risk, cost and complexity of managing and protecting it. Druva's...
BMC has unmatched experience in IT management, supporting 92 of the Forbes Global 100, and earning recognition as an ITSM Gartner Magic Quadrant Leader for five years running. Our solutions offer speed, agility, and efficiency to tackle business challenges in the areas of service management, automation, operations, and the mainframe.