NOTE: This is Part 1 of a 2-part series looking at how larger frame sizes and faster frame rates affect our storage. You’ll find Part 2, on frame rates, here.
It seems like just last week that we were discussing whether it made sense for productions to step up from HD to 4K frame sizes. It probably won’t surprise you that I was skeptical, because, frankly, in non-theatrical environments we can’t see the resolution of 4K when sitting at normal viewing distances from a monitor. (But, when does reality interfere with marketing?)
Still, 4K offered benefits in post, so that became the default frame size for many productions. However, technology continually moves on and now we are hearing the drumbeats for 8K media. Sheesh!
Editing 4K takes a lot of computer horsepower. Editing 8K requires 4 TIMES more than 4K! Which is why Apple is promoting the new Mac Pro for use with 8K workflows. (Though, truthfully, I disagree with their promotion of real-time 8K multi-cam editing – that is MUCH better served by using proxies and supports far more cameras. Smile, though a PROXY for 8K media is, ah, 4K media…!)
(Image courtesy: Apple. Inc.)
“Mac Pro is designed for pros who need to build high-bandwidth capabilities into their systems…. For many pros, a high-performance graphics architecture is critical to their workflows. Especially for tasks like animating 3D film assets, compositing 8K scenes, and building lifelike gaming environments…. Introducing the Mac Pro Expansion Module, or MPX Module…. [and] Apple Afterburner. Blaze through 8K video. Created to transform the workflow for film and video professionals, Afterburner allows you to go straight from camera to timeline and work natively with 4K and even 8K files from the start. No more time-consuming transcoding, storage overhead, or errors during output. Proxy workflows, RIP.” (Apple MacPro website)
I don’t minimize the need for a powerful CPU or the potential of the new Mac Pro when editing frame sizes this huge. However, important as the computer is in editing media, the speed and size of your storage is even MORE critical.
Let’s start by looking at storage requirements for different frame sizes of media.
For all these examples, I’m using ProRes 422 and 4444 because Apple has done a great job documenting the technical requirements of these codecs. Other codecs will have different numbers, but the size and bandwidth relationships will be very similar.
NOTE: All my numbers came from Apple’s excellent ProRes White Paper. Feel free to refer to the tables at the end of this document to develop your own analysis.
More specifically, the three frame sizes in my examples are:
I standardized on 30 frames per second just to pick something in the middle. 24 or 25 fps files will be 10-15% smaller, while 60 fps files will be close to double the size of 30 fps.
All of us are comfortable managing HD frame sizes. ProRes 422 requires 66 GB per hour when shooting 1080p/30 media. However, each time the frame size doubles, the storage requirements increase by 4X!
So, an hour of 8K media takes 1.2 TB for ProRes 422 and 2.5 TB for ProRes 4444! These amounts require totally rethinking the capacity of our storage – and remember, this does not include typical work or cache files, many of which will also be 8K.
Next, let’s take a look as how fast (called “bandwidth”) our storage needs to transfer data to and from the computer.
When compared to 8K, the data speeds of HD are almost trivial…. Remember when – and it wasn’t that long ago – we were connecting devices using FireWire 800 and thought 80 MB/sec was super-fast? Right. Not anymore.
Again, as frame size increases storage speeds surge. Keep in mind that normal editing will require data rates equal to 2X the data rate of a single stream of media. 8K moves us into the world of gigabytes of data a second!
NOTE: If you double the frame rate, say from 30 to 60, all these numbers essentially double as well. Shooting 3D doubles them again!
Now that we have a sense of the storage requirements of our media, let’s take a look at the speed of different storage systems. And, let’s assume for this example, that all devices are connected via Thunderbolt 3. Why? Because Thunderbolt 1 is too slow to support 8K media and Thunderbolt 2 limits bandwidth to about 1,250 MB/second, enough for some 8K media, but I would not recommend it for editing any frame sizes larger than 4K.
In this chart, I added three lines:
NOTE: For planning purposes in editing media, assume data rates equal to 2X the single stream data rate.
The internal SSDs provided with both the iMac Pro and the new Mac Pro are more than fast enough to support even 8K workflows. But… they are not big enough. Even configured with a 4 TB internal SSD, this provides storage for less than 90 minutes of 8K media. 8K media requires massive – and super fast – external storage.
In other words, the technology to support 8K frame rates is here. But it isn’t cheap. It’s also RAID based. NVMe SSDs are the fastest, but they are far more expensive than spinning hard disks and they don’t hold nearly enough. To get the capacity you need, you’ll need to spend more for an SSD solution than spinning media.
A NOTE ON PRORES RAW
(Chart courtesy: Apple. Inc.)
More typical examples of high-end shooting would involve Raw files. However, these are much harder to quantify because every camera sensor is different and ProRes RAW uses variable compression. As you can see from the chart above, ProRes RAW file sizes range from ProRes 422 to ProRes 4444. For this reason, I used ProRes 4444 in my examples because it represented the high-end of the ProRes RAW range.
Keep in mind that other camera formats generate even larger files than ProRes 4444, requiring ever bigger and faster storage. The key point here is that larger frame sizes can quickly exceed the capacity or speed of your storage to support.
A NOTE ON THUNDERBOLT
Working with large frame size media not only requires high-capacity and high-speed storage, but it also requires that storage to be connected via a high-speed protocol. For Macs, that means Thunderbolt. However, not all Thunderbolt connections will successfully support 8K media:
NOTE: The maximum speed of your storage is based upon both the speed of the storage and the speed of the protocol, whichever is slowest.
The rest of the Thunderbolt protocol is reserved for driving monitor displays.
SUMMARY
There are benefits to working with really large media frame sizes. But, the hidden cost is that you also need REALLY big and REALLY fast storage. For those of us working with the internal SSDs of our Mac, or 4-drive RAIDs, shifting to 6K or 8K frame sizes will involve buying all new, much bigger and MUCH more expensive storage to handle it.
These facts are good to know before you start a project.
7 Responses to The Brave New World of 8K Media
Looking forward to that first client that wants to shoot 8K for their Instagram feed.
currently doing 12k and 14k pr4444 VR editing… it’s NUUUUUUUUUTS!
Fabien:
Wow…! Just, um, wow…!
Larry
Great article, but your graphic with 4 Drive SSD should be Raid 0 and 1350 Mb/sec
Blair:
Maybe. If you want the best speed, RAID 0 is the best choice. However, RAID 4, which is better for SSDs than RAID 5, provides data redundancy in the event of an SSD failure. Assuming 4 SSDs in the device, one reserved for redundancy and the other three sending data at 400 MB/s (which is a plug number as different devices provide variations in speed), yields about 1200 MB/second.
Thunderbolt 1 can’t deliver that performance, but Thunderbolt 2 and 3 can.
Again, this chart is designed to illustrate relationships, rather than provide an absolute measure of performance.
Larry
Visited a film set last year that was being shot on Arri Alexa. At 50G per minute, they were generating 8-10TB per shooting day. And they had a lot of shooting days. It takes a long time just to *copy* that much data, let alone do something useful with it.
And I thought my dvx200’s 1G/min was a big deal…
Eric:
Yeah. Video redefines the term: “Large File.”
Larry