Sign in / up

back to article Oracle accused of stifling HP TPC benchmark

Oracle has been accused of stifling the publication of an HP/Oracle DBMS benchmark that indicates its own SPARC SuperCluster world-record benchmark system cost almost 60 per cent more per transaction than a similar test on an HP Proliant system. The record TPC-C benchmark result is held by a $30.53m, 108 processor, SPARC …

COMMENTS

House rules Send corrections
This topic is closed for new posts.
  1. Steve Davies 3 Silver badge
    Pint

    Children, children

    We don't care who started it, just go and sit on the naughty step until you can behave ok?

    Oh you don't want to behave?

    Then you will lose even more of what little creditabilty you have left.

    Pah.

    Off down the boozer. One that won't let kiddies in ok...

    1 0
    1. Anonymous Coward
      Reply Icon
      FAIL

      DL890 is a POJ....piece of junk

      we tested it against the ibm x3850 and IBM had much better performance and I/O throughput.

      Considering IBM is on its fifth generation of a glue chip for Intel it makes sense. Hp is just no getting back into the 8 socket space and the Itanium (loser) team created the 890.

      What is interesting is the poor design of the DL890 if you dont localize the application to the I/O card attached to the QPI link of the CPU. The I/O is a very very long way away if you dont plan the app to fun on the cpu connected directly to the i/o of the CPU.

      We tested both the HP system sucked much worse than the SAP 2 tier SD benchmark shows as they both have published results.

      Sandy Wiloughby

      0 2
  2. Anonymous Coward
    Thumb Down

    You what ....??

    SPARC - 30,249,688 TPM @ $1.01

    Proliant - 3,388,535 TPM @ $0.63

    Nearly 90% reduction in performance for a 40% reduction in cost. How is that more cost effective?

    3 3
    1. Anonymous Coward
      Reply Icon
      Stop

      Do I really need 30M TPM ?

      Benchmarks... Do I really need 30M Transactions per minute ? at 1.01 per

      Or would a crazy 3M TPM at .6$ be more appropriately sized for my non Fortune 50 enterprise ?

      5 3
    2. Chris Miller
      Reply Icon

      The dollar costs are per TPM

      So the SPARC system costs ~$20 million, while the Proliant is 'only' ~$2 million. Whether the Proliant could be scaled up 9x without substantially increasing the cost per TPM is an open question - my guess is it would still cost <$1.

      1 2
    3. Steven Knox
      Reply Icon
      Boffin

      Simple Math

      As mentioned, that's cost per TPM, not total cost:

      SPARC - 30,249,688 TPM @ $1.01/TPM = $30,552,184.88 total cost

      Proliant - 3,388,535 TPM @ $0.63/TPM = $2,134,777.05 total cost

      Total performance reduction: 88.80%

      Total cost reduction: 93.01%

      1 0
    4. Markho
      Reply Icon
      WTF?

      Oracle is choosing their own facts

      Folks - take a closer look.

      HP seems to just be setting a marketing benchmark for their DL980 (big box) using flash memory. This effort involved Oracle engineers and was successful - it seemed to take 5 months and during that time the Oracle/HP friendship dissolved due to senior executive misdeeds - not the engineers fault. Their benchmark was dated November and I hear is audited and valid.

      Then on December 2nd Oracle (Larry) put on a big show to show off the very impressive SuperCluster demo/benchmark. $30 million of equipment, 15 racks and $22 Million in Flash to make it work. Very impressive effort to get the big 30 million transaction number (but at a TPM of $1.01)

      The point seems to be that Oracle blocked HP from publishing their DL980 number (not aimed to be the biggest - just marketing support for the DL980) and went on to do their presentation reaching all the way back to 2007 for an HP TPC-C number

      -----all the while knowing a brand new number was sitting on an executives desk (for some reason HP needed an Oracle signature??? - why did their engineers spend so much time working on it in the first place???)

      It isn't a threat to the big Transaction number - just would have been inconvenient to have the TPM number being $0.63 when Oracle spent all that money for $1.01.

      Is everyone really comfortable with Oracle having that much latitude with the truth? (I know IBM is because the heat is no longer on them ;-) )

      2 0
  3. SplitBrain
    Megaphone

    Silly really....

    All the G7's are blinding server's, as are the IBM X5's, key point here, it's intel pushing forward and innovating with their CPU's, HP and IBM are just packaging it.

    The current 7500's cannot scale past 8 sockets in a glueless config, so if your workload will run on 8 sockets that fine, you get extremely good value for money, if you need more than that then you have no choice but to go for big Iron, P*95 or superdome, just so happens the Sparc Super duper cluster thingy me bobby happens to be the quickest at the moment. Either that or workload permitting you can buy a load of x86 tin and use RAC to save some dough.

    Chalk and cheese....

    0 0
    1. Matt Bryant Silver badge
      Reply Icon
      Stop

      RE: Silly really....

      ".....just so happens the Sparc Super duper cluster thingy me bobby happens to be the quickest at the moment....." I think you'll find what made it "the quickest" was the unrealistic database design used, plus the even more unrealistic and over-sized, flash-based SAN in the background doing all the real work. In real World tests, using real enterprise datasets in real environments, SPARC in any form trails in well behind Itanium or Power.

      1 4
  4. Stephen Channell
    Grenade

    Time to stop rigging tcp scores

    The Oracle cluster system only goes so fast because the Tuxedo client library is doing data-dependant routing and the Tuxedo server is doing async OCI transactions to the database.. tpc performance seems to be a function of the Brocade SAN switches. Next year Brocade will have faster switches, and Oracle will add more nodes and more tpc-c. In the real-world clustered DB-servers take a big performance hit when real transactions span multiple nodes.. serialising everything through the lock managers.

    IBM is little better using client-side Microsoft Windows as the TP monitor for its AIX/DB2 scores.. like asking you customers to handle cash transfers between accounts.. a dream for Fraud in the real world.

    Once you exclude the fiddled tpc-c scores.. the winner is.. err.. a four-year old HP box.

    1 0
  5. Ian Michael Gumby

    Now if HP wanted to have some fun...

    They should contact IBM and benchmark Informix on HP hardware...

    0 0
    1. Matt Bryant Silver badge
      Reply Icon
      Happy

      RE: Now if HP wanted to have some fun...

      No, for me the ultimate gag would be if they stole the record using MariaDB on RHEL!

      0 4
  6. Lars Silver badge
    Pint

    Washing powder

    We all know the importance and reliance of washing powder "bench marks" so perhaps there is a problem with us rather with IT companies producing them for "us" for what ever, until we learn.

    Learn to question the results, and find independent "bench markers".

    0 0
  7. Joe Bailey

    Oracle misleading by omission?

    All the comments on the technical and economic implications of the benchmarks are interesting, but isn't the real issue that Oracle gets to pick and choose which reports are published? If Mr. Ellison gets to veto publication of any benchmark that shows Oracle hardware in an unfavorable light, what credibility do his claims of leadership have? Hardly "free and fair elections," so to speak.

    And why is Violin Memory listed among the keywords for this article?

    2 0
  8. Dazed and Confused

    TPM cluster scores are meanless

    The TPC-C benchmark scales perfectly across clusters. A score of 30M tpm mean that poor old Oracle couldn't find enough money to build a cluster than would score 60M tpm or 600M tpm. M$ always used to be the top of the table since their marketting budget would fund any one wanting to take the TPC-C crown.

    The other issue with TPC-C is that the score must be accompanied by the price. Since Oracle is now a HW vendor and refuses to let anyone else quote a decent price value for the license for any competing benchmarketing result there is precisely zero value in the score.

    The last sensible TPC-C cluster score was when an HP rp8400 16 core box out performed a 16 core cluster.

    You'll notice that the HP's newer SuperDome2, which came out after the Oracle acquisition of Sun does not have a quoted TPM score, I wonder why?

    TPC-C score also pretty much scale with RAM sizes too, the ancient (in system architecture terms) SuperDome [1] manages 4M with 2TB of RAM, the new box will hold 4TB of memory so before any other improvement is thought about you'd except a score of ~8M tpm. But suddenly there is no pressure to post a result.

    Maybe IBM wil get around to posting a newer single image score and Oracle won't be able to beat it with their own tin, at which point they might finally agree to post an HP score. In the mean time they'll continue to behave little little kids.

    Please find a new benchmark. This one is broken.

    0 0
  9. Kebabbert

    Please epxlain

    why HP can not buy Oracle DB or some other DB and publish their own benchmark? Is it because Oracle license does not permit it?

    3 2
    1. Anonymous Coward
      Reply Icon
      Anonymous Coward

      Oracle SW license doesn't allow benchmarking

      Without a signature.

      That is the terms of the HP site license - so HP is held hostage due to the executive fallout.

      1 0
    2. Matthew Morris
      Reply Icon

      Some info and Some questions

      Oracle's license agreement prevents anyone (users, corporations, independent auditing groups) from publishing any performance related data benchmarks etc. without the written permission of Oracle Corporation.

      What would does Oracle gain from not signing off on the TPC audit report? Here is a system that shows Oracle database running very fast and at reasonable price?

      I can only think of Two reasons, a petty spat with HP over Mark Hurd and/or You cannot expect to close a 10 year $23 Billion Dollar pipeline for Exadata Database Machine if there are solutions that run Oracle Faster and CHEAPER than Exadata.

      My guess is that it is both.......

      2 0
    3. Steve Davies 3 Silver badge
      Reply Icon
      Alien

      re: Please Explain

      simple really.

      If HP were to do exactly what you said then the following day Oracle would EOL every version of every product in their pricebook that runs on HP.

      What about HP Proliant systems you ask? Surely they couldn't do that as they are Intel CPU based.

      Well suddenly all Proliant customers would find their support costs raised by a huge amount. Say 200%. Alongside it would be an 'Offer they coudn't refuse' to mode to Oracle RAC on SPARC.

      The only loser in this spat would be HP. That is also why IBM has not gotten involved. Far too many of their key accounts use Oracle. As much as they'd like them to move to DB2 IBM these days are (In my experience) far more pragmatic. They wouldn't want to lose all those nice Z/OS & AIX customers ro SPARC now would they?

      Their bottom line would take a pounding...

      0 0
    4. Kebabbert
      Reply Icon

      Question

      Ok, now I understand. Thanx for the explanation.

      A question: why are people always voting down my posts? Even when I have a normal question? Is people trying to give me bad stats or what?

      1 1
  10. Anonymous Coward
    FAIL

    Much ado about nothing...

    All HW and SW Vendors make their own policies regarding whether or not users have the rights to publish with/without approvals.

    This is not a practice exclusive to Oracle. Someone either missed or purposely overlooked this point.

    It is called marketing. Get over it.

    BTW, it looks to me like some parties here must be in blatant violation of legal agreements for this information to have surfaced-- this presumes that the benchmark in question even exists and was run, as is claimed. Somewhat calls into question the credibility of the sources, no?

    0 2
  11. Anonymous Coward
    Anonymous Coward

    Why won't Oracle run a TPC benchmark against Exadata 2?

    Says on the product pages they will never run a TPC benchmark.

    Odd?

    2 0
  12. Matthew Morris
    Pint

    Why won't Oracle run a TPC benchmark against Exadata 2?

    Most TPC runs last 120Minutes. Since the audited run starts from a clean platform, and you cannot pre-stage data in the Smart Flash Cache prior to the run sooooo. You would see how fast 8 dual socket (6 Core) DB servers with 96GB of RAM each would run Oracle and the back-end is all JBOD (Grid Disks).

    What is more interesting, since they added Flash Cache to support mixed mode workloads. They should still be able to go and run a TPC-H test. Though there seems to be no interest in that.

    Now Oracle should have kept the same montra as that of the Former Sun. Instead of proving how poorly CMT and SPARC processors run compared to the others bits of Silicon out there. They should stop running benchmarks.

    The 30Million number is based on 27 SPARC T3 servers delivering 1.1Million per server. An HP DL580 running SQL Server peaked at 1.8 Million TPC-C.

    3 0
    1. Kebabbert
      Reply Icon

      Scaling

      "....The 30Million number is based on 27 SPARC T3 servers delivering 1.1Million per server. An HP DL580 running SQL Server peaked at 1.8 Million TPC-C...."

      The difficulty is scaling. When you try to scale up, you can not need the same technique. You must use other more complex solutions. That is difficult. Compare writing single threaded software to multi threaded software. Multi threaded software is much more difficult to create. Single threaded software may be faster on one cpu, but it does not scale up to many cpus and cores.

      0 1
      1. Anonymous Coward
        Reply Icon
        Alert

        Scaling

        That is exactly the point of the TPC-C test. it runs the same code base across the different platforms.

        RAC allows for scaling provided that the DB and application can work within the confines of shared everything.

        0 0
    2. Jesper Frimann
      Reply Icon
      Linux

      $/tpmc

      It would be hugely expensive in $/tpmc. It's much better to run a system that's almost like Exadata, but not quite where you can use all the pricing tricks you have up your sleeves.

      The typical CTO is not clever enough to figure out that it's not the same system.

      // jesper

      0 0
      1. Anonymous Coward
        Reply Icon
        Coat

        $/tpmc

        I would be careful, CTOs are getting smarting.

        When I look at the information on TPC - It would seem that Flash is making the difference. Almost all of the leaders are running flash as the primary storage. HP must have used Flash for this test as well.

        1 0
This topic is closed for new posts.

Other stories you might like