Isilon gets thin with phat storage app

thin provisioning is pretty neat

My company bought a 3PAR E200 a few months ago and have been using thin provisioning exclusively. It's pretty cool stuff. Though one thing to keep in mind is the provisioning is one-way(at least with 3PAR, I'd assume the others are the same). e.g. if you create a 1TB volume, and write 300GB to it, it consumes 300GB(+ raid overhead) on the array, if you then erase that 300GB, it still consumes 300GB on the array. Since the array has no way to safely determine if the space as been freed so it cannot reclaim it.

I expect to see in the coming years more intelligence between the array and the file system to allow safe, automatic "un-provisioning" of space when significant amounts of space is deleted.

Our array is fairly small, just 2 shelves with ~8.6TB usable. We have about 24TB of space that is thinly provisioned(29TB if you count snapshots, which 3PARs management tools count as virtual space when showing space usage), with roughly 6.1TB of physical space being used.

3PAR also allows you to set a warning threshold as well as a block I/O threshold, so you can guarantee that a particular volume won't use more than X amount of space if you want. Not sure if the other vendors offer this. An example of this in action is I set an oracle standby database logfile volume to warn at ~50% space utilization and block I/O at ~85%. The file system was using a small fraction of that(under 10%). But due to a long running snapshot, it ate up a lot of space over time, and one night the array hit the limit and it blocked new writes to the volume. The linux host was spewing scsi errors and oracle was bitching for hours until the morning when I saw the errors and removed the restriction. Oracle continued on as if nothing happened(no restart or anything).

3PAR support contacted me as a result of the SCSI errors and decoded the error saying it was reporting an out of space condition(something I wasn't aware was able to be communicated over the scsi protocol, I thought it was purely a file system thing). -- (with 3PAR in thin provisioned volumes snapshot space can be unlimited when it is tied to the parent volume)

Unless you have gobs of physical space it's important to stay on top of space utilization when using thin provisioning until you get a good handle on how it behaves with your applications, because the growth may not be what you expect, since array provisioning and file system utilization may be significantly different. I've rebuilt file systems that were consuming 500GB on the array but using less than 80GB on the file system itself because of lots of adds and deletes(on a 1TB volume).

At one point during talks with 3PAR they told us they were going to have us sign a document that said we understand the thin provisioning technology and agree that if we mis-use it(over allocate without proper planning) and it causes problems(run out of physical space) then it's our fault since we got training on it. Well they only ever told us that once, and we never saw the document. Not sure if they just forgot or maybe changed their minds. But in any case I've been watching it closely as we get closer to our physical capacity limit.

And we're still very much in the learning phase. We have 19 servers connected to the E200, it's been a great array for consolidating our storage needs. Really fast, real easy to use.