[ This article was first published in the February, 2011, issue of
Larry’s Final Cut Pro Newsletter. Click here to subscribe. ]
Sebastian writes:
I’m working with ProRes output. The transcode itself is fast. It’s the stitching together of the individual ProRes files at the end of this process (when compressor says “merging”) that to me would seem I/O dependent. The number crunching is complete at this point, what the system is now doing is copying all the QuickTimes into one BIG QuickTime. If it’s doing this on one disk, for 110 GB worth of HD ProRes, I suspect the speed of this part of the procedure is I/O dependent… What do you think?
Larry adds: Sebastian then went on, in a separate email, to say…
[To speed things up,] turn on QuickCluster in system preferences (under Qmaster), set instances to 6 (if you have 8 cores, to leave two free – make sure you have a GB per core you intend to use) and click ‘start sharing’. You maybe familiar with this much. When you submit in compressor, don’t submit to ‘this computer’, submit to the QuickCluster you just switched on. This will use 6 cores of the machine to transcode, instead of 2 – i.e., 6 smaller QT’s can generate simultaneously. Now look in the destination folder where you’ve set the quicktime to output to – as the job compresses, you should see multiple smaller QuickTimes appear. Your transcode will be much faster than submitting to ‘this computer’, even counting the time it takes to stitch these files together at the end.
Larry replies: Sebastian, I’m not a fan of QuickClusters because, personally, I’ve found them problematic and unreliable. However, if you can get them to work, then by all means use them.
[NOTE: This statement is true for Final Cut Pro 7 and earlier. It is too early to know how well QuickClusters work for in Compressor 4, for which they were completely rewritten.]
UNRELATED SIDEBAR
Sebastian was working with ProRes files, which are I-frame based. In his situation, I suspect he is correct that the gating factor on speed is I/O based. However, when compressing files for the web using H.264, clusters have an even bigger problem.
H.264 is a Long-GOP codec that was not designed to work with multiple cores or cluster processing. H.264 was designed to deal with a single compressed stream of video coming from a single processor. Compiling files from multiple processors is very complex math, because, unlike MPEG-2, the GOP length of H.264 can vary wildly within a single clip. This makes the final compile very difficult and time-consuming.
Cluster compression is an excellent idea if you are compressing lots of different files into H.264, but performance falls off rapidly when you are trying to piece together multiple elements of a single, large file.
Since most of the H.264 compression projects that I do are single large files, I have never been satisfied with the performance of cluster compression on my Mac.
