what about Isilon ?

You may not have fully understood the architecture behind why it nets out to 80TB. They are effectively using full mirroring so rebuild time is much faster when a disk fails than RAID 5.

It also gives you a slick way to upgrade disk sizes since you can replace a module with higher capacity disks and, since the mirrors are not on the same module, it just rebuilds them like a module had failed.

Upgrades on our Shark are much more complicated to do when we want to go to higher capacity disks.

Putting on my CIO hat, I can say that time and reliability are worth a lot more than the cost of additional commodity disk drives.

Isilon? What about them? He listed them as a potential. The issue with them is they're a point solution. If they're looking to buy a 1-trick pony they should snag lefthand before HP buys them (and destroys them). If they buy anyone to replace their LSI-based DS4000 series they will buy Pillar or Xiotech. Maybe Compellent (but there's so much to fix in that product that they would be nuts to buy them).

Chris,
Are you so share of what you write? I wish I could go into details, but I can't. As a customer, I'm very familiar with the XIV platform. As a customer, I'm also very familiar with the DMX-4/CX-5 platforms. The technical details that Barry provided are not accurate. In fact, it's VERY unlikely that a DMX-4 or CX-4 would be configured in the examples he's provided. Also, his usable numbers are not accurate at all. The XIV has a true usable capacity of 50 or 80TB, depending on the generation. While a DMX-4 or CX-4 is does not have the capacity that Barry describes. It's more like 72TB usable for a DMX-4 configured with 180 1TB drives. This changes the power costs substantially. Likely more on par. Also, a DMX-4 950 configured in that way would require an additional cabinet. That would require more power and twice the footprint. Not exactly an apples to apples comparison.

"Here's the major problem with XIV: You have to buy 180 1TB drives in the new IBM XIV utility storage system to get a mere 80TB of usable capacity. It's got some really fat software (make that a LOT of fat software) hogging a ton of space in that system, and there's something very constipated about that. "

That sounds kind of like RAID 1 utilization percentages. A bit worse, perhaps, but not drastically so when you consider that lots of arrays have additional overhead factors beyond RAID.

"The fact is, buying 180 1TB XIV drives to get 80TB of usable capacity makes no fiscal sense, because you can get about the same usable capacity with 33 percent fewer (100 to 120 1TB) drives in any conventional RAID 5 or 6 system. We can't even imagine what the extra cost might be in the XIV system to power and cool all those spinning disks for all that wasted storage."

Really? If it is in fact true that with this technology you can get the performance of much smaller fiber channel disk, as XIV claims, wouldn't it be cheaper to use 1TB SATA in this kind of config, than 146GB FC drives or even 300GB FC drives in a RAID 5 or 6 config? And let's not forget that there are a lot of FC drives out there in RAID 1 configurations, especially for high I/O write intensive workload. And as far as the power and cooling, isn't it true that the number of drives is more of a factor than the size of the drives. It takes 5 300GB FC drives in a RAID 5 (4+1) config to get about 1.1TB useable or 6 in a similar RAID 6 config, as opposed to 2 1TB SATA drives in a RAID 1 config to get almost as much useable storage. Which config uses more power and cooling? What really makes no fiscal sense is to reach a conclusion through a knee jerk reaction, rather than though careful analysis.

Thank you Stewey for your insights. Yes, I realize that there is redundancy built into the XIV, the DMX, and every other storage system worth its salt. But man, using 100 TB (!) for redundancy and the software that goes into running the system ... that seems to be way over the top. Maybe I'm off here, and I am not a storage architect, but more established systems get better marks in this area from evaluators.

I agree that due to configurations, it can be difficult to compare systems and the features they provide. Fair point. But I try to keep to the basics when comparing, and those are capacity and I/O performance. You basically get what you pay for in added features anyway.

Thanks for writing. You have more insight -- please send!

/cp

Joe:

Thanks for writing. I'm intrigued by what you wrote about Compellent: Tell me some of the things that need to be fixed in their systems.

Thanks in advance.

/cp

Barry:
Thanks for your insights. You obviously know a few things about storage system admin/config. That's not my expertise, certainly -- I am mainly reporting what some storage industry experts are concerned about, and they are concerned. Granted, some of them -- such as Barry Burke -- are on the other side in the sales/marketing wars. But that doesn't preclude them from having a legit point, which is that XIV isn't all that advanced. Is that your take on this also, or not?

Thanks for writing

/cp

This is nonsense - just bloviation to fill up some column inches. Customers are thrilled with XIV and will continue to be as IBM rolls out its offerings. Every customer who has tried it has decided to keep it. This product is in a class by itself. It is not in the least comparable to Symmetrix - the entire architecture is different - this is a new paradigm. You are grasping at straws here with the simple-minded focus on TB. The marketplace will decide. XIV is one of the best acquisitions IBM has ever made (not all have been successful).

Let me be clear that comments here are my own and have nothing to do with IBM's position on any of this!!

Thanks Mike for writing. I appreciate where you're coming from, but where you're coming from cannot be denied -- and that is Armonk, N.Y. (or at least IBM's headquarters is there). I thank you for being upfront with your affiliation. You might truly believe in your company's products (I certainly believe in eWEEK's!), but there are a number of people I respect in this business who are not as convinced as you about XIV.

Let the marketplace decide. No more true words were ever spoken in business.

I invite you to try and convince me why XIV will succeed over others. I'm listening ...

/cp

Joe,

I wanna hear about compellent's weaknesses myself. Can you elaborate?

If anyone else can elaborate on compellent's weaknesses...I would be much obliged.

The only thing that I found is that they think that can support 1000 drives with 2 RAID controllers and 7GB cache. And don't get me started about moving random blocks from FC to SATA... that's just silly. Anything else?

Thanks!

Chris,

You are missing the point of XIV. The attributes of a very fast, highly scalable, highly available storage system. That it uses a larger set of less expensive "mass market" drives to achieve the performance and availability is why it is the next generation of storage for IBM.

Stewey points out that it is difficult to compare storage from those like the EMC systems mentioned. This is true with a lot of present generation systems, and really why technologies like XIV, Compellent, and Exanet(in the NAS space) are hard to compare as they are in the next generation. Handling blocks of information make far more efficient use of space. This becomes reality when people move to the next generation and reduce actual storage needs to 10-20% of the previous generation because storage is pooled using virtual blocks. This also translates into reduction of replicated blocks if you are doing replication and snapshot sizes are also lower as they are just modified blocks and pointers...

Chris, you should also take a closer look at why XIV can rebuild drives so quickly. Try rebuilding an array of 5x1TB in RAID 5 on Dell/Equallogic, it takes days. On XIV, it uses the write speed of the entire set of drives other than the failed unit to "rebuild" the redundancy of the data. During this window, you are exposed to a double fault, anyone would want the protection back asap to minimize the opportunity for a double fault "making you weekend". By keeping the rebuild window short, one can use drives SATA drives without the concerns that push the normal paradigm to use RAID6(can you say a lot of overhead).

Robert has it right, putting on the CIO hat, it is an easy choice.

Chris, as you stated you try to compare capacity and I/O performance. Capacity we addressed above, it is not about the drives, it is about how much can the applications use, performing at the optimal IO rate given the need to for availability.
What dictates sheer IO performance in any of these boxes is cache. If you don't believe it, just turn it off. This is why the DMX has so much. IO requests perform better if disk is not part of the IO requested(it just physics). To this end, there is a lot of cache in XIV, which makes it and systems like XIV handle random IO workloads well.

Joe, I use Compellent. My opinion is after using multi-tier storage, thin provisioning at multiple sites, and replicating the data between them is that the system works without any problems. Performance is outstanding. It is by far the easiest to manage. Putting on the CIO hat, it's handling of disaster scenarios with VM's is the best I have seen and makes testing a breeze. I am now deploying a newer environment and doing boot from SAN to simplify the DR issues even further.

FYI: Magoo: I use the auto migration block feature on Compellent and it works well. It moves the data(2MB blocks) not used frequently to denser(cheaper) drives. This keeps your high-speed drives serving data frequently used, and you buy less of the expensive, power hungry high speed disks, a nice green thing. You can also control the block ILM on a per volume basis. This allows IT to assure performance for volumes that need it.

As far as supporting a lot of drives on a couple of controllers, it would depend on the workload. On this system, if you were doing an archive, you could consider a two tier storage with a lot of lightly used drives on a couple of controllers. You would configure it quite differently if you were setting up a high performance, low latency database like Oracle Enterprise....

Don

Stewey - I'm not sure where you've gotten your data, but the usable capacity points AND the power costs I used in my post were in fact based upon real configurations, and even included the additional cabinets where required. And even if your "72TB usable" for a DMX4 with 180 drives WAS accurate, it comes nowhere close to compensating for the fact that the XIV requires significantly more power for the exact same number of drives - even WITH the additional cabinet on the DMX4 - $14,614 / 72TBu = $202.97/TBu, verses the XIV's $250.56TBu.

But you're right about one thing - the right comparison isn't against the DMX-4, but against a CX4, where the advantage is far, far greater.

And one thing I've learned is never to speak in absolutes. I can assure you that Mike Ferrel's opinion above is unrelated to reality: "every customer" hasn't bought into the XIV kool-aid - I know of several accounts who have returned their XIV loaners with no intent to purchase or evaluate again.

Nobody bats 1000

And to Don P's points, rest assured that not all arrays suffer from "days" to rebuild a RAID 5 5+1 set like the Dell/Equallogic example he gave. Advanced arrays today will rebuild a failed drive at the maximum rate a drive can accept sequential writes - it is not uncommon for a 1TB drive to require little more than 5 hours to rebuild.

Further, probability mathematics show that you're less likely to have a second drive failure in that same 5+1 RAID 5 set during the 5 hour rebuild than you are to have a dependent drive failure in the XIV RAID-X set of dependent drives - even if the XIV rebuild is less than 30 minutes total (unlikely unless the drives are each less than half full, mind you). A 30 minute rebuild isn't fast enough to offset the size of the dependent drive set (which IBM made BIGGER with v2, not smaller).

And as to the performance of wide striping across so many drives - well, you can get the same (or better) benefits from widely-striped thin provisioning as well. And with 180 drives in a CLARiiON, you will likely not only get better performance, you'll get more usable capacity.

Finally, I've yet to see any explanation of how a 7200 rpm SATA drive can match the IOPS response times of a 10K or 15K drive. No matter how wide you stripe, each new I/O has to come off of disk, and it will do so slower off of slow ATA than fast FC. Sure, you might be able to aggregate lots of drive for a high total MB/s, but how well will the XIV approach fare when tasked with OLTP random I/O workloads on disks that are more than 60% full?

What's weird is how last year all the buzz was about "underutilized storage" and "green IT" - systems running at 60% or less utilization were all the concern, and thin provisioning was the solution demanded by the market to solve both issues.

And yet we're discussing a new product offering whose BEST utilization can be no more than 80/180=44%, and one that requires significantly more power per usable terabyte than virtually any other SATA-based solution you can buy today.

With our without my obvious vendor bias, there's simply something wrong with that equation, IMHO.

Thanks Don and Barry for some very informative comments here, with or without "vendor bias" for or against IBM or EMC. We'll be keeping a close check on these products in the marketplace to see what users think.

BTW, are there any actual users of XIV out there -- beta testers or purchasers -- who would care to talk about the pros/cons of the system here? It would be great to hear from you.

/cp

I keep reading about how 7200 rpm SATA drives can't possibly match the IOPS performance of faster disks... It's curious how Anarchist insists on broadening the scope of IBM's intended audience for this system in order to compare it to the DMX. It isn't INTENDED to be used for OLTP workloads. IBM has made it pretty darn clear that it is primarily intended for unstructured data...digital media, web providers, imaging, etc. It isn't intended to replace the DS8000. What a marketing droid!

Speaking of performance though - at least we'll likely have some standard, independently verified idea of how it performs in the near future. Too bad we can't compare that to any EMC system...wouldn't it be funny if this "slow" commodity-based SATA system out-performed your fancy new CX4? That would be a riot! Of course, you could easily put your money where your mouth is if you'd simply do what all other major storage vendors have been doing - get on the SPC train, baby! (No, I don't need to hear any whining about how benchmarks "don't matter." THEY DO. Period. End of discussion.)

And seriously dude...if a CIO is naive enough to make a purchasing decision on one cost metric alone, he deserves what he gets for listening to your FUD. It's hilarious to me that EMC is NOTORIOUS in the industry for all their hidden costs...and here you are making judgements about the one or two components of TCO that give you your FUD fix for the next 20 minutes or so...

Classic.

SRJ, thanks for your input. Wow. You don't feel too strongly about all this, do you? Say, could you elaborate a bit on the "hidden costs" EMC is supposedly hitting people with? Can you offer some examples you've experienced or heard of?

thanks,

/cp

Chris - heh...I guess having my RSS newsreader filled with his drivel for the past couple of weeks finally got to me. =) Quite honestly, I have absolutely zero allegiance to the XIV solution...I responded on principle. Truth matters...and listening to FUD gets REALLY tiresome at his pace.

I mean, seriously...have some modicum of respect for the rest of the industry. I don't need him to sugar-coat anything...I understand and enjoy a healthy dose of competitiveness...just stop with the *constant* twisting of information for marketing purposes. It insults the intelligence of everyone who reads it and makes him look like a 5th grade bully trying to impress those he's already bullied into submission.

Probably the most agitating thing about it is that he writes authoritatively (he's obviously a smart guy) while he's bashing the performance of other products, yet doesn't have the guts to back up his claims with cold, hard (independently tested and verified, like everyone else) performance metrics. I don't mean to be so harsh...I'll stop ranting now. Sometimes enough is enough, you know? Less is more....more is less... (hint hint, BarryB!)

Back to your actual question - if I were to answer it, I could be accused of spreading the same kind of FUD that I'm holding him accountable for. Then again, if I don't answer it I might be (incorrectly) accused of not backing up my claims with cold, hard facts. I've backed myself into a corner now, haven't I? =) The only responsible answer is a kind of non-answer... (less is more, remember Barry?) Here goes:

I grant that all vendors have their own questionable (at best) sales tactics. You asked about EMC's because I made the observation that they're notorious for it. I stand by that claim, without hiding the fact that other vendors do it too, albeit not to the extent where they're notorious for it. So, with my only burden being to defend my claim of EMC's notoriety for its sales tactics, I can simply point you to Google...or to Joe Tucci's speech about his sales force at the 2001 Storage Decisions conference in Chicago.

Interesting quote from that speech: "If there's one thing I can't stand, it's arrogance among my sales force."

That was in 2001. Time for a status check on the corporate bloggers, eh Joe! ;)

-SRJ

Chris -

One question about the story: What "fat (really fat) software" is in the XIV system?

Sometimes you choose to mirror data for performance reasons...you need a lot of spindles to get better performance than RAID 5 or 6. Mirroring SATA drives in an attempt to get better performance out of them isn't economically crazy.

With that out of the way, your question logically becomes: "What about that other 20TB? Where does that go? Software?" The answer is no...much of it is lost to factory formatting (a 1TB drive doesn't really give you 1TB, remember) and capacity for the equivalent of hot-spares in a traditional system. As the other guys mentioned, if you configured any other system for 80TB of mirrored 1TB SATA drives, you'd end up with about 180 drives. IBM's own DS4800 would require slightly more than 180 drives, actually. Stewey's estimate of 72TB for a DMX config with 180 drives is accurate as well when you account for spares, etc...

There doesn't seem to be anything "really fat" about the software when you realize that comparable configs for other systems would actually need slightly more drives for the same capacity/performance mix. It was probably unfair to characterize it as such...