NetApp accused of short-stroking its new hardness

Either way...

No one would buy netapp for performance. It's just not there. They're getting better. But still suck for performance.

And all the benchmarks are on systems setup to get the best #'s for that test. All the vendors do it.

Sounds like...

...and EMC sales pitch.

Storagezilla has its own interpretation of scale-out

Add Storagezilla to the mix of those storage vendors who have their own interpretation of scale-out.

Alex McDonald of NetApp here.

Chris, my apologies; I promised you some reasoned arguments and background information as to why EMC/Isilon appear to be misunderstanding the specSFS benchmarks. Since you've published, I'm replying here.

Twomey of EMC makes one valid point; "Scale-out means different things to Isilon, [IBM] SONAS, [HP] IBRIX and NetApp." But this isn't about definitions or about what we each mean by scale-out or scale-up or scale-anything; it's about scale -- full stop -- and a benchmark which is tightly defined (and where we spanked EMC). The rest of his arguments are, as usual, diversionary nonsense. What's eating Twomey is the fact that NetApp's submission was smaller, cheaper and faster.

But I am surprised at Peglar, the America's CTO (Chief Technology Officer) of Isilon, because he betrays a serious misunderstanding of the benchmark, and I'm surprised that he isn't better informed. Here's what he should know.

The specSFS benchmark creates 120MB of dataset for every requested NFS operation. You can't control how much space the benchmark is going to use -- in fact, the usual complaint is how big the SFS dataset size is. We (NetApp) chose a volume size of 12TB for each volume giving 288TB. The main number to look at for the benchmark is the file set size created which was 176176GB (176TB) for the 24 node test. We could have created much bigger volumes and could have exported the capacity of the entire system at 777TB. Which would have not made a difference to the results; since the fileset size created would *still* have been 176TB.

Isilon exported all the usable capacity. 864TB. The benchmark dataset size for them was 128889GB (129TB).

So, on inspection, it took Isilon 3,360 10K rpm disk drives (plus 42TB of flash SSDs) to service 129TB of data. NetApp took 1,728 15k rpm disk drives (plus 12TB of flash cache) to service 176TB of data.

Now who's short stroking?

There are two arguments un-informed arguments we hear about benchmarks all the time, and I thought Peglar would have understood them and why they aren't relevant.

Argument 1: If one doesn't touch every byte of the exported capacity then the system is being gamed, so as to short stroke the disks and gain an unfair advantage.

Response 2: There will never be any real world workload that touches *every single byte* of all available capacity. That is not the way systems have, or will ever be used. Benchmarks model a realistic workload and measure systems under that load, not bizarre edge cases.

Argument 2: Creating LUNs that are smaller than the maximum capacity is creating short stroking and an unfair advantage.

Response 2: Modern filesystems no longer couple the data layout with the exported capacity. Thus, there is no performance advantage that is related to LUN size or the exported capacity. As long as the same amount of data is accessed across systems then the equal performance comparison is valid; or, as in the NetApp submission, where a *lot* more data is being accessed, the benchmark demonstrates it's a much better performer. If you are seeing a difference in performance that is coupled to exported capacity, you might want to consider a NetApp system that does not have such an antiquated data layout mechanism.

Summary: The total exported capacity is the combined capacity of the volumes created. It does not have any bearing on the performance obtained.

The argument Peglar makes would seem to indicate that Isilon may have one of those old, steam-driven data layouts. But, of course, an Isilon system doesn't, so why he's making the points he does is beyond me. There are only a couple of reasons that EMC/Isilon could present an invalid premise for an argument; (1) they don't understand the subject material, and lack experience in debating these issues, or (2) they fully understand the subject material and believe that the person they are trying to convince does not.

I'll let you guess as to which I think is the case.

I called Jeremy Clarkson

We could do with some help from the Top Gear team on this. At least we get an injection of real humor!

Of course, what we have is hype thinly disguised as science, just as in the Bugatti-Ferrari war. I can't help but wonder if we could have really aced this test with white box nodes filled with SSD, and done it cheaper. Software might be a bit challenging, but Symantec might oblige, with or without Huawei!

I can see the contest now, Jeremy driving Netapp and boasting a lot, James running the slow, but sure, EMC and apologising all the time for the delays, and Richard running the white box, cackling every time he beats them on a run.

Delicious!