Product Review: OWC ThunderBay 4 (SSD vs. Disk)

Posted on by Larry

[Please read my disclosure statement on product reviews.]

[ Updated with comments from Timothy Standing, the programmer of SoftRAID, July 21, 2014. I am grateful for his comments. ]

After I wrote my initial review of the OWC ThunderBay 4 Thunderbolt 2 RAID, the folks at OWC sent me some SSD drives to showcase the speeds this unit is capable of.

So, because I’d already written my initial review (I like the unit), I decided to focus exclusively on performance in this review and look at how the different ways we configure the RAID can make a difference on performance.

In this article, I’ll look at both SSD and standard hard disks – and, along the way, I discovered some big surprises.

Website: www.macsales.com

Unit tested: OWC ThunderBay IV (4-bay JBOD)

MSRP: $949 (12 TB spinning media) — $1,299 (1 TB SSD)

TECHNICAL NOTES

I’m doing all my testing on a late 2013 iMac. Speeds will improve somewhat if you are using a new MacPro and will be reduced somewhat on older systems. However, the speed relationships as you compare different RAID levels and settings will remain the same.

UPDATE – July 21, 2014

I just learned that my new iMac still only has Thunderbolt 1 ports on it. Currently, only the Retina MacBook Pro and new Mac Pro support Thunderbolt 2.

NOTE: My initial review of the OWC ThunderBay unit was done on a 2012 iMac. You can read my first review here to compare the performance between these two iMacs.

The drives are all 3 TB Toshiba drives, provided with the unit by OWC. (I will refer to these as “spinning media,” in contrast to the SSD (Solid State Drives) that I’ll be testing later in this review.

Speeds are measured using Blackmagic Design’s Disk Speed Test. All drives are empty.

NOTE: As drives fill with data, they slow down. As drives get within 20% of full, they slow down significantly. These results should be considered “best case.”

I’m using OS X 10.9.4, the current version of Disk Utility and the B15 release of SoftRAID.

NOTE: There is no indication on the OWC website, but the RAID unit offered for sale is the JBOD version. If you need to create a RAID 5, which requires SoftRAID, please talk to a sales rep at OWC before purchasing the unit to make sure you are purchasing the right version.

UPDATE – TIM STANDING

You are right that the files at the end of a disk read and write much more slowly than at the beginning. We recommend that video editors use only the first 30-50% of each disk for there performance critical work. SoftRAID has the ability to resize a volume, even one with files on it. This allows users to take their existing RAID 5 volume and shrink it down so that they are no longer using the slower part of each disk.

Larry asks: Can we partition the array into a “fast” partition for active editing and a “slow” partition for general storage?

Tim replies: Yes, when you create a volume which uses less than the entire disk, SoftRAID uses the fastest part first. So you can just take a set of disks, make a first volume with 30 – 50% of the Max size (the fast volume) and then go and create a volume with the rest. In my testing, if a volume at the beginning of the disk gives you 500 MB/sec a volume at the end will give you around 330 MB/sec. The real slowdown happens in the last 30%. The first 30% is pretty much the same speed.

ADDITIONAL TECHNICAL BACKGROUND

It isn’t intuitive, but hard disks and RAIDs can be optimized for different purposes. For instance, video requires very large files, audio requires medium-sized files, while databases send small bursts of data into the master database.

A technical setting called “stripe size” configures the hard disk, or RAID, for a specific purpose. While there are a number of options, here are the three recommended choices from SoftRAID.

Most of the time, the default stripe size of 128 KB will be fine – especially if you are working with a variety of files. For reasons I’ll illustrate shortly, I’m testing these RAIDs using the default stripe size of 128 KB.

The SoftRAID program, which is available at extra cost as part of the OWC ThunderBay RAID, provides single-menu access to optimizing the drive for specific purposes.

NOTE: While RAIDs are the best choice for video production, many times, configuring the individual drives in a RAID as separate drives(called a “JBOD”) is often better for audio production; especially when you are creating music using sampling. While JBODs don’t have any data redundancy, they will have much faster performance when you need to access lots of small files all at once; which is what is required when playing back a multitrack music project.

UPDATE – TIM STANDING

You write that audio files require lots of simultaneous i/o to many small files. If [an editor] can accept slower write performance, then a RAID 5 volume would be great for audio files. RAID 5 volumes have awesome read performance even for simultaneous i/o to lots of small files (the read performance for a 4 disk RAID 5 volume will be very close to that of a 3 disk stripe (RAID 0) volume). This is true even if the volume is degraded (i.e. one of the disks is missing).

While writing to many small files simultaneously to a RAID 5 volume is much slower than that of large files, it is often faster than writing to a single disk. When duplicating a folder of 100,000 files, each 128 KB in size, a RAID 5 volume is actually 1.5 times as fast as a non-RAID volume.

Audio editors [should also consider] a RAID 1+0 volume with a larger stripe unit size. If you are trying to read many small files at once, a RAID volume is much better than a non-RAID volume as long as the read size is smaller than the stripe unit size. So if you’re streaming 8 audio files and reading 64 kb blocks from each file, then a stripe unit size larger than 64 kb would be best.

If the program is smart enough to buffer larger chunks of audio, then a RAID 5 volume would give you the best performance as it would burst the data into the program’s cache almost twice as fast as a RAID 1+0 volume.

With RAID 4 and RAID 5 volumes, the stripe unit size is set at 16 kb. It was an architectural decision I made. I think it is the right one.

DEFINITIONS

Data Redundancy. The ability to recover your data in the event a drive crashes or dies. While all drives can (and should) be backed up, data redundancy allows you to recover data from a hardware failure.

BIG NOTE: Data redundancy does not allow you to recover a file that you deleted. Backups or Time Machine provide that capability.

JBOD. “Just a Bunch of Drives.” This configures all the drives in a RAID as separate drives, each of which appears on the desktop. JBODs do not have data redundancy, but do provide the fastest i/o when accessing multiple files.

RAID 0. A RAID configured for fastest performance, but without any data redundancy.

RAID 1. A RAID configured for maximum data redundancy, but very slow performance.

RAID 5. A RAID configured for fast performance with data redundancy such that if one drive dies, all your data is safe.

RAID 6. A RAID configured for fast performance, though not as fast as RAID 5, with data redundancy such that if two drives die at the same time, all your data is safe.

My general belief is that RAID 5 is the best balance between data redundancy and performance. Today, I get to test to see if that theory is correct.

NOTE: Here’s an article that explains RAID levels in more detail.

JBOD – A COLLECTION OF SINGLE DRIVES

There are two ways to format these OWC units – using Apple’s Disk Utility or the OWC SoftRAID application. In this article, I tested both.

Configuring a RAID as a JBOD turns all drives – in this case four – into individual hard disks. The benefit to buying a RAID when all you want is a JBOD is that all the drives are all in single unit, which decreases clutter on your desk and only requires one power and Thunderbolt connection.

However, with a JBOD, each drive is independent, which means that all drives perform at about the same speed; and no drive has any inherent data redundancy.

When I format the drives using Apple Disk Utility (in the Utilities folder), these are the results.

When I format the drives using SoftRAID, these are the results – about 2% slower. While not a big deal, this indicates that if you want to use the ThunderBay as a JBOD, there’s no big performance advantage to spending the extra money for SoftRAID.

RAID 0 – FLAT OUT SPEED

However, most of us buy RAIDS to get faster performance than we can get from a single drive. RAID 0 combines the speed of all the drives to yield MUCH faster performance. In fact, this is the fastest speed we can get; but, there is no data redundancy. Lose one drive in the RAID, and you’ve lost everything.

NOTE: As always, data loss can be prevented with good backups stored on a separate device.

Here are the results when formatting the unit as a RAID 0 using Apple Disk Utility. Whoa! This is, essentially, the speed of one hard disk multiplied by the number of drives in the unit.

Let’s reformat the RAID using SoftRAID and measure the performance. For this test, I set the stripe size to 1 MB. Hmm…. not as good as when using the Apple Disk Utility. In fact, 7.5% slower Writes and 2.5% slower Reads.

NOTE: Deleting or reformatting a RAID volume destroys all the data stored on that RAID. So only do this when there is no data on the RAID.

Here I reformatted the unit as RAID 0 with a stripe size of 128 KB. Changing the stripe size improved performance by 8%.  Setting the stripe size to 128 KB provides performance roughly the same as Apple’s Disk Utility. I would have expected better performance with a larger stripe size; interesting.

RAID 1

A RAID 1 is a fully-mirrored drive, where each drive is a mirror of the other. RAID 1 is the best option for DIT work on set, where speed is less important that an absolute guarantee that you won’t lose data if a drive dies.

Here are the results of a RAID 1 created by the Apple Disk Utility. These are in-line with the results I expect. A RAID 1 should have the speed and storage of a single drive, even though the data is being written, for security, to multiple drives at the same time.

Here are the results of a RAID 1 created by SoftRAID. Again, this is in-line with what I would expect, though the write speeds using SoftRAID are not as fast as Apple’s Disk Utility.

RAID 5

Apple’s Disk Utility only creates RAIDs 0 and 1. To create a RAID 5, we need to use SoftRAID. (There are a variety of other RAID formats, but, for video editing, the only other RAID format I would consider is RAID 6, which SoftRAID does not support.)

When we add data redundancy by configuring the system to a RAID 5, we gain security, but we lose the speed of one hard disk along with the storage capacity of one hard disk. This is due to using the equivalent of one drive to store the parity data, which is used to recover our data in the event of a disk crash. For this reason, if speed is your goal, RAID 0 is the best choice. For general editing, though, I recommend a RAID 5.

Here are the performance results of a RAID 5 setting. Two things I find interesting:

  1. Write speeds are slower than Read speeds. This is not what I would expect.
  2. Write speeds are about 10% slower than I would expect in a RAID 5 with drives of this speed and quantity.

INITIAL THOUGHTS

If video editing is your goal, these speeds are more than sufficient to support up to 25 streams of ProRes 422 video. This is PLENTY fast for virtually all editing, except stereoscopic 3D, uncompressed 4K, or high-bit-depth RAW formats.

Second, performance varies depending upon how we configure the RAID. So, let’s test some SSD drives.

SOLID-STATE DRIVES

SSD (Solid-state Drives) drives (I know, the second “drives” is redundant) generally provide big speed improvements, but with a huge trade-off in storage capacity and price.

When using traditional hard disks, this four-drive unit can hold up to 20 TB of data. When using SSD drives, it only holds 4 TB. Also, I would not recommend SSD drives for a RAID 1 configuration. Since a RAID 1 is not designed for speed, spinning media would be a better – and much more economical – choice.

OWC provided me with four Mercury Extreme Pro 6G drive (240 GB each) SSDs for my tests. Installing the drives so they can take the place of a traditional hard disk involves mounting them in a NewerTech AdaptaDrive.

The process is not hard, though it is persnickety. We need to place four screws to hold the SSD drive into the SSD drive carrier, then install the carrier into the drive sled for the RAID. Eight screws per drive. The process isn’t hard and took me about half an hour. (Except for the third drive, which I installed upside-down. Fortunately, the RAID won’t let you install a drive unless it is position correctly. I should also mention that I’m grateful OWC sends extra screws with the unit…)

NOTE: It is fashionable, in these minimalist days, to feel that we should only install a screw or two to save time. Folks, this is YOUR data we are talking about. Use all the screws.

SECOND NOTE: Pay attention to how the carriers are mounted in the sleds. If the sled does not slide all the way into the RAID unit, it is because you mounted the carrier upside-down in the sled.

UPDATE

A reader pointed out that additional sleds are available at the OWC website for about $20.

Once installed, the SSD drives need to be formatted. I used Apple’s Disk Utility for this and formatted them as HFS+; however, I discovered that I should have used SoftRAID. It wasn’t hard to change the formatting, but it wasted a chunk of time.

ERROR MESSAGE

After installing the SSDs, I opened SoftRAID, which asked if I wanted to update the Mac OS to support trimming to non-Apple devices. Since trimming does not affect performance, but IS a good idea for overall SSD longevity, I said NO.

At which point, the SoftRAID system crashed with a meaningless error message.

So, if you are using this unit with SSDs, be SURE to say YES when it asks if you want to support Trimming for non-Apple SSD drives.

UPDATE – TIM STANDING

Larry, thanks for the detailed description. This allowed me to find and fix this bug.

Creating the SSD RAID volume takes longer than creating a spinning media volume. After creating the RAID, it took about five minutes for SoftRAID to create the file system so the RAID would appear on the desktop. There is a thermometer which indicates progress, which you can see here.

NOTE: The default stripe size of 64 KB is probably not the best choice for video. I changed it to 128 KB; which also matched the stripe size of the spinning media tests above.

Now that the SSD drives are installed, formatted and the RAID is mounted, let’s take a look at the speeds we can achieve.

UPDATE – TIM STANDING

The reason for this delay is that the Mac OS X pauses all i/o to the volume until TRIM is complete on the SSDs. You can also see this in action if you create a 100GB file, drag it to the Trash, erase the Trash and then start copying a new file to the volume. It will take 10 – 20 seconds before the copy starts. You are waiting on the TRIM command. Either way, you get the benefit of freshly erased flash storage for your SSD to use. This is mainly going to affect performance when your SSD gets closer to being full.

SSD – RAID 0

The first thing I notice is that the SSDs are dead quiet. The fan noise is barely perceptible. However, the four drives in total provide slightly less than 1 TB in storage. Clearly, there’s a big trade-off between storage and performance; not to mention price.

Using SoftRAID, I created a RAID 0 (though I could have used Disk Utility as well).

However, on my late-2013 iMac, the SSD speeds are not what I was expecting. The Write speeds were 3% slower than using a RAID 0 with spinning media. Read speeds, though, were 11.5% faster. Still, on a RAID 0, I was expecting speeds closer to 1 GB/second.

NOTE: Video editing performance is more dependent upon Read speed than Write speed.

Configuring this as a RAID 5, we see write speeds 5% slower than when using spinning media, while Read speeds are about 12% faster.

Hmmm… the speed increases are not at the level I was expecting. (The OWC website claims speeds in the mid-800 MB/second range.)

This tells me that if you are planning to this unit while with an iMac or MacBook Pro, you would be far wiser to buy spinning media, not SSDs.

UPDATE – TIM STANDING

The results from the SSDs are not what I expect. I have seen performance of up to 1.4 GB/sec read – 1.2 GB/sec writes on a MacBook Pro using 2 Thunderbolt 2 ports to 6 SSDs. I have seen performance of up to 3.3 GB/sec reads – 1.4 GB/sec writes on a Mac Pro using 6 SSDs. Your results show that there is obviously some performance tuning I have to do.

I won’t release a driver until we have many billions of i/os of testing (we have been testing this extensively since last September). We wore out 6 disks during our testing of RAID 4, RAID 5 and RAID 1+0 volumes (and had a substantially higher electric bill). Our standard test is 3 Macs, each running 4 or 8 disks for 48 – 72 hours. This is usually over 500,000,000 i/os per Mac and about 120 TB or transfers. Our test is a multithreaded read/write verify type of test. Towards the end, it was taking 2 – 3 days to reproduce the bugs we were chasing. Oh yes, and we did 2,000 cable pulls on RAID volumes before we officially passed the driver on a RAID level.

Since I have so many months of testing the driver, I don’t want to make any changes to it so close to our release date. I plan to go back into the driver code in a month and figure out why some of the performance numbers are not what I expect. I suspect that I am running out of buffers and just need to increase the number of these in order to see the faster performance on SSDs.

UPDATE – July 22, 2014

Larry O’Connor, the CEO of OWC, also provided a comment to this review. [When connecting the ThunderBay to a new Mac Pro,] “our ThunderBay 4 hits over 1300 MB/s with 4 SSDs inside our Thunderbolt 2 unit formatted as RAID 0 – this maxes the bandwidth Apple allocated [ to Thunderbolt] pretty much 🙂

EXTRA CREDIT

SoftRAID provides several advantages over using Apple Disk Utility:

However, as a software RAID, I suspect that SoftRAID won’t achieve the same speeds as a hardware RAID. On the other hand, when comparing the speeds of the SoftRAID in the OWC unit to the software RAID in a Drobo, the OWC FAR out-performs the Drobo.

Because Thunderbolt allows up to six devices to be connected at the same time on the same bus, it is possible to connect multiple ThunderBay RAIDS and create a RAID 0 or RAID 5. While possible, I’m reluctant to recommend this approach. For me, there is too much opportunity to things to become disconnected or power to be off on one unit, and not the other.

If I need more speed or storage, I would recommend buying a RAID that contains more drives.

SUMMARY

There were several surprises for me in this test:

Here are my recommendations:

Given all of these observations, I think OWC has a solid performer that is small, quiet and, once set-up, more than meets the needs of most video editors working in any video format smaller than 4K, or with uncompressed RAW media.


Bookmark the permalink.

13 Responses to Product Review: OWC ThunderBay 4 (SSD vs. Disk)

  1. sethmellman says:

    Larry-

    Thank you for all of this great info on the OWC unit. I am interested in setting up a similar Raid 5 system and this was exactly the type of info I needed. As a novice in this area, you mentioned “hardware” raids vs. software raids. Can you shed some light on hardware raids and anything we might consider in this area? Thank you.

    Seth

    • LarryJ says:

      Seth:

      A hardware RAID is a RAID controller embedded in a chip, rather than emulated in software. Traditionally, hardware RAID controllers are faster, but much less flexible, than a software RAID controller.

      The best way to find out what type of controller is being used is to visit the vendor’s website.

      Larry

  2. Robin Harris says:

    Larry, a great review of the issues and, like you, I found the disk vs SSD results surprising. In RAID 0, the performance should be 4x that of a single SSD. For the Mercury Extreme Pro 6G, OWC says over 500MB/s per drive, sustained writes – the kind you have in video editing – or ≈2GB/sec.

    Clearly there is a bottleneck.

    It is probably the Thunderbolt 1 connection on the iMac, which is roughly limited to 1GB/sec. Bottom line: you should see better performance with the SSDs on a Mac Pro with Thunderbolt 2 because it can devote twice the bandwidth to the RAID.

    On the issue of RAID 0 vs RAID 5 on low-cost arrays: I prefer RAID 0. Based on the best available research evidence, all hard drives have an MTBF of ≈300,000 hours or an annual failure rate of 3-4%. This means that if you use RAID 0 you have less than a 1 in 5 chance of a drive failure in any given year with a 4 drive array. However, low-cost arrays like the ThunderBay only have one power supply and, in my experience, a higher failure rate than any one drive, or even any four drives.

    Therefore, in my experience – which includes working in large storage companies, so I have a large sample – a small array is as likely to have a power supply failure as a disk failure. And when that happens you’ll have to stop work until you get your backup loaded on a new array. So the extra safety of RAID 5 is more apparent than real, and it’s slower than RAID 0.

    With ANY RAID professionals should backup DAILY. Depending your budget this could be to a large USB 3.0 disk, a USB 3.0 or eSATA disk array, or a 2nd ThunderBay 4. If the primary fails, you’ll lose at most a day’s work and be back running in minutes, not hours or days. You should NEVER rely on a single device to protect vital data. That’s 30+ years of storage experience speaking!

    Regards,

    Robin

    • Larry Jordan says:

      Robin:

      I completely respect your opinion – thanks for contributing to this.

      Regarding the difference in speeds for SSDs, I’ve been talking with the SoftRAID programmer who has some ideas on what is causing it. However, the iMac I’m testing on using Thunderbolt 2, so it isn’t a Thunderbolt 1 bottleneck issue.

      Your comments regarding RAID 0 are VERY!! interesting. Hmmm… I need to ponder them.

      I also agree totally about backups – the problem is that buying a second RAID for backup is not affordable for many editors. (Granted, backups are cheaper than rebuilding lost data, but a credit limit is still a credit limit…)

      Thanks,

      Larry

  3. Robin Harris says:

    I checked the iMac tech specs on Apple.com before I wrote because I thought as you do, but they do not say Thunderbolt 2, which is what the Mac Pro and MacBook Pro specs say. Have to take them at their word.

    So yes, I’m afraid Thunderbolt 1 is the bottleneck.

    Affordability is a problem. People just need to be aware – as I’ve found many aren’t – that just because it says RAID it doesn’t mean your data is safe. It can still fail – usually at the worst possible time – and hose your data.

    Keep up the good work!

    Cheers,

    Robin

  4. Hi Larry,

    Longtime fan … thanks for the very timely article.

    I too am surprised at the SSD speeds. Robin’s take on this is comforting as it would be a slam-dunk answer … but I’ll leave you two to figure that out. Let us know.

    My take is that, as much as I love the folks at OWC, the Mercury Extreme Pro 6G isn’t the fastest SSD out there. That arguably belongs to the new Samsung 850 Pro 1TB at 550 MB/s. That small amount of extra speed, along with a Hardware RAID 0 might get you closer to that 1 GB/s threshold.

    I certainly hope so. I’m looking at the Promise M4 with 4 of the Samsung 850 1TB drives. Anything less than 1.2 – 1.3 GB/s will be disappointing. I will be buying the ThunderBay iV empty and stuffing HGST 4TB Enterprise drives into it as near-line and back-up. Deep back-up is going to bare drives via a Rocketstor 5212 from High Point.

    Heard anything about the Promise M4 yet?

    Thanks!

  5. Mark Dobson says:

    Hi Larry,

    Thanks for both of these OWC Thunderbay 4 reviews. Really useful as I was about to press the button on another 12TB Raid 5 system ( without mentioning names its only recently been released and looks a tiny bit like a black plastic New Mac Pro )

    I’ve just had an 8TB Thunderbolt drive die on me ( again no names mentioned but you’ll know which one!) So with the refund money I’m going to invest in this OWC system, so a big thanks for the heads up.

    In the last 10 years I’ve now had 2 hard drives go belly up on me so, yes, a Raid 5 makes a lot of sense, not because of loosing data as I’ve slowly taught myself to backup & backup, but in the sense that one can continue working rather than go through a time-consuming process of gathering disparate media together before recommencing.

    I’m going for the base enclosure and installing the drives myself, partly for cost and also because I couldn’t find a UK supplier who had any stock. OWC have an idiot proof video on their site to help me through this process.

    I was particularly impressed with the comments from the OWC developers. Be nice to see Apple chipping in like that with their new products.

    One question I have is regarding whether its best to leave these units on all the time or to turn them off at the end of an editing session. Also once the disks have spun down is it ok to move the unit as sometime when I’m off filming I like to lock crucial bits of equipment in a secure cupboard.

    Thanks again for these clear and well thought out articles and also for the excellent Compressor talk you gave at Earls Court during your TVBay tour of the UK. We did have a brief chat about using h264 mp4s to replace WMV file delivery.

    Mark

  6. Mark says:

    Hi Larry.

    How can I tell whether the two Thunderbolt 1 ports on my 2011 iMac are on the same bus or are each on their own bus? Here’s what “System information” tells me:

    iMac:

    Vendor Name: Apple, Inc.
    Device Name: iMac
    UID: 0x0001000A16B32D80
    Firmware Version: 22.1
    Port:
    Status: No devices connected
    Link Status: 7
    Port Micro Firmware Version: 2.0.7
    Port:
    Status: No devices connected
    Link Status: 7
    Port Micro Firmware Version: 2.0.7

    Many thanks,

    Mark

    • Larry Jordan says:

      Mark:

      They are on the same bus. Apple didn’t release multiple Thunderbolt busses until the new Mac Pro.

      Larry

      • Mark says:

        Good to know! It helps me with some purchase decisions: No use trying to reduce bottlenecking by keeping devices on separate TB ports if both TB ports are on the same bus!

        Thank you so much for sharing your expertise, Larry. I really appreciate it.

        All my best,

        Mark

  7. […] I took an OWC ThunderBay RAID and divided it into two partitions using the SoftRAID utility. (Again, remember that each drive […]

  8. Bean says:

    Software RAID, no thanks, seems like an ok product for the consumer, but not a pro-sumer.

Leave a Reply to LarryJ Cancel reply

Your email address will not be published. Required fields are marked *

Larry Recommends:

FCPX Complete

NEW & Updated!

Edit smarter with Larry’s latest training, all available in our store.

Access over 1,900 on-demand video editing courses. Become a member of our Video Training Library today!

JOIN NOW

Subscribe to Larry's FREE weekly newsletter and save 10%
on your first purchase.