IBM smacks rivals with 5.0GHz Power6 beast

Yeah, but...

How does this compare to the new Macs from Apple? Shoddy reporting as usual from Mr. Vance, he never looks at how these so called 'enterprise' systems stack up to the best computers on the planet....what is he afraid of? Is he thinking his overlords at Sun, IBM, and HP will crush him like a bug? Or that they will withhold his monthly stipend?

Is it a power beast as well?

So IBM has to resort to extreme water cooling in order to reduce this thing's power requirements on the data centers they are deployed in.

A) Does this help make for a more "beastly" price compared to a Sun based soution?

B) Even if this cooling solution reduces the power draw by 40% percent over previous Power solutions, is it still more power hungry than a Sun solution?

I'm not a Sun fanboy, just curious. Also very suspicious about what IBM did not reveal as opposed to what they did reveal.

I was in love

Before y'all mentionned Vista.

I might still get one of these shiny racks. That's on my todo list. Just before "buy a Vista-certified box for personal use" (they come at about the same price anyway).

Water cooling P6 adds extra costs and complexity.

Water cooling for CPU's is an admission of a design failure. The extra pipework and electrical power to run the water cooling just adds costs and complexity. Extra electicity is required to run all the water cooling, so there are not savings, this needs to be taken into account and cannot be a hidden infrastructure cost. Plus to add an extra water infrastructure to a computer center with all that electrical wiring is dangerous. Computers and water do not mix well together. Using water cooling for computers is a technological backward step.

All described here: http://blogs.sun.com/ValdisFilks/entry/water_and_electricity_do_not

Water cooled.....

You could also use it as a giant coffee machine.

"Fire up the database Geoff, I need a double skinny latte" (whatever that is).

Water Cooling

Good heavens - water cooling in data centres is dangerous? Well we managed back in the 70s and 80s without too many people being electrocuted on mainframes back in the days of ECL chips. In fact it can make a great deal of sense to use liquid cooling as it is a far more efficient way of carrying high concentrations of heat than is using air (a tad quieter too - the air cooling systems oin some disk arrays can be so loud as to require the use of ear defenders). Note that virtually all car engines are liquid cooled, not air-cooled. Gas cooled nuclear reactors turned out to be a dead end and liquid cooling has won out.

Using water cooling has one considerable advantage - it's possible to export the heat directly outside the building and not have to do through a process of cooling air, pumping the air through the building and then heating it up.

Of course it's better not to generate the heat in the first place, but for some sorts of workloads which cannot be readily split or which depend on single-thread processing speed (read huge, high-thoughput database servers as the prime example) then there often isn't a lot of choice.

I suspect that most IT people don't come across the sort of monster workloads that suit these beasts. There isn't a need for a vast number of them, but for certain applications they most certainly are needed.

Water Cooling

I disagree with Valdis.

Air cooling is inefficient and results in significant power wastage. If the heat can be removed with water to some central point where it can be lost, there will be significant energy savings relative to running CRAC units to keep an arctic datacentre. Water cooling also enables the datacentre power density to be much higher, instead of the rule-of-thumb 4KW per rack which is as much as conventional raised floor cooling can handle without hot spots.

The inhibitor on water cooling is that nobody wants to break ranks first. Kudos to IBM for going first on this. While I remain to be convinced whether cranking Power 6 up to 5GHz will offer any real customer benefits, it's entertaining to watch.

Greener than given credit for?

Surely with so much hot water being circulated, it could be "lost" to the building during winter, thus saving money on the heating bill...

Re: Greener than given credit for?

This could equally be done with an aircooled datacentre. Cambridge University's computer lab can have its waste air pushed through the building's heating system in winter, for example.

Water cooling

A couple points, raised in these comments, which I will try to answer.

1) Yes we used water last century in datacenters. But that is because we did not have a choice. When aircooled systems became available we had a choice and kicked the water cooled computers out very quickly, to be replaced with air cooled computers.

2) Water cooling on occassion failed, leaked, had to be switched off, water cooled servers needed to be switched off. Things got really ugly, I worked in v.large datacenters I saw these problems. Someone mentions that we used water without too many people being electocuted, to me one such water cooling disaster is one too many. Avoidance is a better policy.

3) Air cooled servers could coutinue to run, while water cooled servers needed to be switched off, when the safety guys decided the water leaks were too much of a risk in a high voltage datacenter then we had to switch off everything.

4) Yes water may be a better coolant than air, it adds complexity though. Magnox is a better coolant (shorter half life) in Nuclear reactors than water in PWR. But magnox much more complex to maintain than water. In the same way that water is more complex to manage in a datacenter. Using oil is just another level of complexity.

5) Are we proposing that we justify to buy a computer to help heat the building. When we decommission a water cooled server, do we need to buy extra boilers to heat the building. This is a plumbing nightmare, lets keep things simple. Again, water cooling adding complexity.

We are back to the starting point, do not make hot chips that require water cooling. Maybe I am too risk averse. But I worked with big computers (mainframes), then Amdahl and Hitachi came along and made air cooled machines. Which sold like hot cakes as they did not require all the extra baggage of water pipes, cooling, power, monitoring, maintenance. If you want to scare the hell out of a datacenter manager, tell him he has a water leak in the datacenter.

Prevention is better than cure, do not use water cooling.

I am not that old, but strive for simpler computer architectures avoiding the mistakes of the past. Have we learnt nothing from our computer heritage.

Water cooling

I'm with the pro water cooling crowd.

Water cooling might be a bind if you are thinking in terms of a small server cupboard cooled by through the wall air-con units, but once you have a whole building full of the beasts then your air-con system has to be driven by a massive chilled water plant anyway.

In this case getting the chilled water closer to the source of the heat has to reduce the overall power use.

The only time I remember a big puddle of water under the machine room tiles it was not a failure of the chilled water system but the dehumidifier drain pipe got blocked and it was the air-con system that caused the flood.

Sooooooooooo

We struggled with the Stone Age,

Battered our way through the Bronze Age

Ground through the Iron Age,

Had a bit of fun with the Industrial Revolution

Entered the Space Age

Landed in the Garb Age

And are now having a second go with a renaissance of the Steam Age

@Valdis Filks

Hitachi and Amdahl pushed air-cooled solutions because they were competing with IBM on price, whereas IBM was compteting on having better performance. IBM knew that Emitter-Coupled Logic (ECL) was a faster clocking solution than CMOS or NMOS at the time, and that despite the heat they would have a performance advantage on the very heaviest workloads.

Eventually, CMOS and NMOS gained in clock speeds, and even IBM made the transition - but acknowledged that the individual processors were slower than their predecessors. Today's energy-efficient CMOS processors require massive paralleization to compete with either more complex CMOS designs (i.e., the original scorching Pentium IV, et al) or ETL successors.

And what has the parallization of processors brought? An acknowledgement that for many sorts of problems, we can't effectively write parallel code. That's why Intel and others are crash-researching parallel coding techniques. That is why IBM continues to push the speed of individual processors, even to the point of resucitating water-cooling in the data center.

Now, that's not to say that MANY problems don't decompose well into parallel solutions: parallel databases have revolutionised data warehousing, clusters are wonders of mainstream application hosting, etc. But for several, important, and PROFITABLE types of workloads (esp. some scientific work and mathematical work, as well as some database solutions) individual processor performance are the limiting factor.

Lastly, I would be curious to see how the water cooling on these corresponds to the watercooling in old mainframes. Are they running the pipes within the case to radiator fans (as in a watercooled PC), or running them external to the case via datacenter cooling lines as in the old mainframes? In short, what is the real level of complexity?

Parallelism and water cooling.

The water cooling is a risk assesment, reduce the components involved and simplicity always is a better option.

For example you can have, in your infrastructure:

Design A = Power (electricity), Network (electricity), SAN (optics mainly), cooling (air)

Design B= Power (electricity), Network (electricity), SAN (optics) and cooling (air + water)

Design B has more components plus a catastrophic mixture of water and electricity. This is the problem, even if you use it to heat your building, your infrastructure just got very complex and dangerous, by adding water. Your safety regulations and such like has just increased costs.

Parallelism, this is also a virtualisation play.

With a low power air cooled server which has 8 cores you can virtualise 8 single threaded apps. You use the free software and the system to create 8 domains. As some of these cores can have more than one thread e.g. 2. You could even consolidate 16 servers or to one of these highly multithreaded servers which are 2U in size and go by the name of coolthreads. No water cooling required.

However, lets take any mainstrean already existing database e.g. MySQL, Oracle, Postgress and DB2.

These have many parallel processes (often more than 8) e.g. db writer, lock manager, transaction manager and so on. So put these on a multi core/thread server and you have a good match. No coding/changes/migration required. A very large majority of servers are running OLTP workloads that have a database engine as described above so a very large segment of the computer/server market is ideal for parallel computing.

Long term agree, need to write better parallel apps. But a lot of apps (OLTP) out there are parallel in nature already.

Can't quite agree on the water cooling, but I am open to suggestions.

Flawed thinking

Re: "Water cooling for CPU's is an admission of a design failure. The extra pipework and electrical power to run the water cooling just adds costs and complexity. Extra electicity is required to run all the water cooling, so there are not savings, this needs to be taken into account and cannot be a hidden infrastructure cost. Plus to add an extra water infrastructure to a computer center with all that electrical wiring is dangerous. Computers and water do not mix well together. Using water cooling for computers is a technological backward step."

Totally flawed thinking. Your suggesting that it's not worthwhile using water cooling as it requires electricity which would negate the benefit caused by the cooling process itself.

Total rubbish. The idea behind water cooling is to cool the hot processor, to allow that processor to run at a high clock frequency. Who cares if the cooling system requires some electricity to run it, the goal is not to save electricity, the goal is not to prevent the heat being generated, the goal is it to allow the processor to run at 5GHz to take that heat away from the chip and if water cooling is the way to do it then that's the way to do it, if forced air cooling is not sufficient.

What do you suggest? Not use water cooling, if so, what do you then do? The chip will burn out and be completely useless.

Think about automobiles, in most the engines are water cooled, and yes, it requires electricity to pump the water round.

By your argument, nearly every automobile that has ever been made is a design failure which is clearly nonsense.

Years ago, mainframe computers used ECL (silicon based chips) which drew considerable more current than current CMOS/MOS technologies, they were water cooled. If properly designed then it's not a problem.

Water cooling adds complexity.

Agree, water is used in chillers in the datacenter, most of the computer rooms that I have been in, the air con units/chillers are in the periphery/edges of the rooms. What water cooling to servers/computers does is put pipes and plumbing all over/under the floor mixing it with the network, SAN and electrical power. Do we need an extra substance/piping under the floor. Does it create more problems than it solves. I have seen many companies go through projects to simplify and reduce their underfloor/overhead wiring. Some datacenters even have alarmed floor tiles. Why do you want to mix water pipes with this.

The point of this is to reduce complexity, less is more, simplicity is better than complexity.

All computer manufacturers should help give the computer industry a good reputation for technology leadership by reducing complexity. This is the point about water cooling, it does not make things simpler for customers. Water cooling makes computing more complex.

Do you want to say to your customer, here is a new server and do you know a good plumber, on 24hr call out, 7 days a week.

NB I do not write anonymous but am open and stand behind my beliefs. I was born in a country and live in a country where free speech is respected. Do anonymous writers/comments have something to hide ?

Air cooling aint that great...

back in the IBM water cooled mainframe days it was the AIR cooling devices that were failing, not the WATER cooling. Dead fans caused more outages than anything else. So now there's redundant fans all over the place...not exactly simple...or quiet...or efficient. How many blade racks have caused hot spots in computer rooms due to expelling too much hot air?

I can't help but think

In these days of high energy costs, and growing environmental concerns, that water will win. You can dump heat from a water system by pumping it through a pipe in the ground. You could easily heat exchange it into an existing HW system. With a bit of forethought even the worst plumbing failures could be manageable - with reduced capacity.

More watercooling opinions

The critique of watercooling as I see it has a main flaw. While much of the reasoning in the examples is ok and appropriate the generalisation does not follow. So while it may very well be the case that there are questionable implementations of watercooling this is not a natural result of the principle of using watercooling. As a complementary proposition it is perfectly feasable to imagine flawed aircooling solutions. As I have come across quite a few I do not even have to use much imagination for this. And it is true that we could find many flawed watercooling systems that also fail. It is also true that we could find systems where watercooling was put as an afterthought because the original objectives of the design were not met. However this does not mean that it could not be possible to design intentionally a solution with watercooling (e.g. not a result of a 'design flaw'). In any specific practical implementation such a solution does not by default have to be more prone to failure than a specific aircooling solution. Compare to a car engine, while it could be argued that watercooled car engines in general add complexity (and would in principle than be more suspectible to problems) this does not necessarily mean that air cooled engines are more reliable in practice. We should also remember that if complexity is something we do not like than maybe we should stay away from all things IT - what about the result of moores law?...

One thread, one processor

Overclocked gaming PCs use water cooling, but not in the same fashion as the mainframes of old. So I think that the risk of leaks can be avoided. Since the real need for high speeds is for problems that are serial, I would think that a limited number of hot, high-speed chips would be needed, and the parallelizable part could be taken care of by slower multi-core chips that produce more throughput per watt. It would be nice to get a desktop machine with just one of these high-speed processors, but it would have to run Linux, not Vista, not being x86.

RE: Valdis, water vs air cooling

If the machine did not use water cooling, then it would use heat pipes with gargantuan fins to disipate the heat into the air. The fins for just one CPU would be enormous. The fins for *32* CPUs would mean that the case would either have to be huge, or else take another rack space behind it for the CPU fins.

I have worked in datacenters where I had to wear ear muffs so I wouldn't be deafened by the fan noise. The datacenter at Microsoft which has two HP Superdome servers has cold air running through it like a wind tunnel. Nobody stayed in that room longer than they absolutely had to.

Now consider water cooling. Water is inexpensive, and easy to handle. Large HF transmitters are water cooled, with the tubes in direct contact with pure circulating water, and nobody gets electrocuted and the equipment runs fine. Fluorinert is horribly expensive, and water is, well, cheap as water. Yes, plumbing is a problem if your plumbers are incompetent.

From what it looks like, your position is that anything other than air cooling is a failure. Sorry, 'tain't so.

Liquid cooling?

Makes sense for situations where you've got to conduct a lot of heat away. You trade off the extra infrastructure for efficiency, quietness, even heat distribution (important) and even the potential to recycle some of that energy. Most car -- and motorcycle -- engines are liquid cooled (it may be a surprise to know that many bikes are liquid cooled, the fins on the engine are just for show).

Air cooling in data centers doesn't make sense. You make a lot of noise (and spend a lot of energy) transferring the heat to the air only to have to remove it using liquid cooling through air-conditioning. Its much better to cut out the middleman.

Water works well. I've been on a project where we used de-ionized water because we had to plumb it to a tube anode running at 7.5Kv -- taking the ions out renders the water non-conductive. For data center use, though, the water's more likely to be like the stuff in your car engine.