Improve RAID Speed with Partitions

[ Updated July 28, 2014, to better explain how partitions improve speed.]

During the last couple of months, I’ve been testing a lot of different storage from companies like Promise Technology, G-Technology and OWC, among others. Single drives, multiple drives and RAIDs.

I’ve known for a long time that hard drive speed varies as the drive fills up. Drives are at their fastest when they are empty. As they get closer and closer to full, they slow down. What I recently realized was that RAIDs behave the same way: empty RAIDs are really fast, while full RAIDs are much slower.

This is because hard drives, which are the essential storage component of a RAID, store data to the inner (faster) tracks first, then, as the drives fill up, data is stored to the outer tracks, which are slower.

THE “SECRET SAUCE”

We can take advantage of this fact to improve the overall speed of our RAIDs by partitioning them. (We can partition individual hard drives, too, but the performance benefits are much less significant. I no longer recommend partitioning single hard drives.)

What a partition does is divide the physical storage capacity of a RAID into separate logical “chunks.” Think of this as increasing the number of drawers in a filing cabinet. The overall cabinet does not increase in size, but by increasing the number of drawers it makes it easier to access files stored in the drawers.

WHAT ARE WE ACTUALLY IMPROVING?

Whether a RAID has one or more partitions, the fastest bandwidth it will ever have is when it is empty. When a RAID has only one partition, as the drives fill up, the RAID slows down. What partitioning does is minimize the amount of slow-down when using the first (fastest) partition.

The reason this improves performance in a RAID is because the first partition always includes the fastest portions of the RAID. While additional partitions only include slower portions of the drive. While the actual method you use to create partitions varies by RAID and manufacturer, the results are easy to illustrate.

NOTE: We can create multiple partitions on the same RAID. But, in general, you only need two: one for fastest performance and one for everything else.

AN EXAMPLE

Recently, I took an OWC ThunderBay RAID and divided it into two partitions using the SoftRAID utility. (Again, remember that each drive vendor will have its own partitioning utility. Contact the support department of your RAID’s manufacturer for instructions.)

In this example, the RAID had 12 TB of total storage. I put 8 TB in the first (fastest) partition and 4 TB in the second, slower, partition. (There isn’t any big advantage to creating more than two partitions.) Then, I measured the speed of the RAID.

NOTE: I would not make the first partition smaller than 50% of the total RAID capacity, nor larger than 75%.

Here are the results of the first (fastest) partition: close to 500 MB/second for both Read and Write.

And here are the results of the second (slower) partition. There is more than 110 MB/second in speed difference between the two partitions. THAT is a lot!

BIG NOTE: You can only partition a RAID or a drive when it is empty, because partitioning erases all data currently stored on the system. Consequently, this technique is best used when configuring a new system for editing, or after copying all existing data off the RAID and temporarily storing it somewhere else.

SUMMARY

Dividing the RAID into two partitions, guarantees that the performance of the first (fastest) partition doesn’t drop as the RAID fills up, because while the partition is getting full, the drives themselves, which determine the actual speed of the system, are not.

Not all the files stored on a RAID need high-performance access. By creating two partitions, I can put files that need fast access on the first partition and files that are not speed-dependent (think compressed media files, word processing, spreadsheets, PDFs and so on) on the second partition.

I’ve currently implemented this technique on the two RAIDs in my edit suite. I’ll let you know how it works, but, so far, this works really, really well.