Building FFMPEG with support for DeckLink capture and h264_qsv encoding

NOTE: this document covers Intel’s Media Server Studio 2016. If you want to use Media Server Studio 2017 with a Skylake processor, see this article.

ffmpeg has come a long way since the pre-1.0 days. With its elaborate system of routing filter outputs, its ability to capture video from video cards, and support for GPU-based encoding, it has become quite the powerhouse in the video world.

This article outlines how we built ffmpeg to capture video from a Blackmagic Design DeckLink mini and encode it using Intel’s QuickSync technology (h264_qsv).

Install CentOS 7.1 with Media Server Studio

Intel has a bit of a branding problem with their GPU-based encoding. I have seen it referred to as “Media Server Studio”, “MSS”, “MFX”, “QuickSync”. I never know what to call it when I’m talking about it. I guess I’ll call it “QuickSync”.

It’s also a bit tricky to install support for QuickSync under Linux. As of this writing, you can’t use QuickSync with a Skylake processor on a Linux system. Broadwell is your best bet. Skylake support is promised for Q3 2016.

First, stand up a CentOS 7.1 system with a Broadwell processor and one or more Blackmagic Design DeckLink cards. Install Media Server Studio by following the steps in this document. For convenience, I present them here.

Install from the CentOS 7.1 DVD. Under Software Selection, select Development and Creative Workstation. It is critical that you choose this configuration, as there are steps later that depend on specific packages being installed.

Once you boot up the system, you might want to change the default runlevel so that it doesn’t use the GUI. This is not strictly necessary, but I don’t like to have a graphical UI running on my video encoders. As root:

Before proceeding, double-check that the OS can see your GPU.

You should see something like this:

If you don’t see the controller, you may need to tweak some BIOS settings to get the OS to recognize the graphics hardware.

Add the user(s) who will use QuickSync to the video group. As root:

Unpack the Media Server Studio tarball:

You’ll need to modify the install_sdk_UMD_CentOS.sh script. By default, it will use yum to install kernel-devel, which will install a newer version of kernel-devel and kernel-headers. Replace this line:

with:

Download the version of kernel devel that matches the default CentOS 7.1 kernel:

As root, install the RPM:

As root, run your modified install script:

After installing, reboot the system as instructed. When the system comes back up, run this as root:

As a regular user, build the kernel:

Note: once you have built the kernel RPM, you can install it on other systems without building it from source. This will save you a lot of time, and you won’t have to install all of those build tools onto the other systems.

As root, install the newly-built kernel:

Note: you will probably get some warnings like:

I believe these are harmless and can be ignored.

Make sure that you don’t have nomodeset in your kernel boot parameters. This is not explained in the Intel documentation, but it is absolutely critical. I had to edit /etc/default/grub to remove nomodeset. Then I ran

If you are using a UEFI boot, the process may be different for you.

Reboot your system, making sure that you are booting with the new MSS kernel. Once your system is back up and running, issue this command:

You should see this:

Assuming you’re running the right kernel, use Intel’s System Analyzer Utility to make sure that the system is ready for QuickSync.

Install Blackmagic’s Desktop Video

Install EPEL (required to install dkms for Desktop Video, as well as yasm, imlib2, libass, libdc1394, openal-soft, schroedinger, soxr, and zvbi that we will use in our ffmpeg build). As root:

Download Desktop Video (Blackmagic’s name for their DeckLink drivers) from the Support section of Blackmagic Design’s web site. The installer builds a driver custom to your specific kernel using dkms. You’ll need to make sure you have the right build tools installed. As root:

Note: if you’ve followed this guide from the beginning and built Media Server Studio kernel RPMs, you’ll already have all of these installed except dkms. But if you’re later installing prebuilt RPMs on a system without going through the entire build process on that machine, you might not have these packages.

Before you install the desktopvideo package, you need to make sure the version of the installed kernel headers package matches exactly the version of the running kernel. If you followed the instructions closely and modified install_sdk_UMD_CentOS.sh before running it, you won’t have a problem.

But if you didn’t modify the script, you will have a mismatch, because the install_sdk_UMD_CentOS.sh script ran a yum command that pulled the latest version of the kernel-headers package from the centos repo. This version does not match the running kernel.

To check this, run the following command:

In the list of installed packages, look at the version numbers for the kernel package and the kernel-headers package. They should match.

If they don’t match, you can fix it by running this command as root:

Now you are ready to install the desktopvideo package. As root:

Reboot your system. When it comes back up, make sure that the OS sees your DeckLink card(s):

You should see something like:

Note: this system has two DeckLink Mini Recorder cards installed. Your log entries will probably be different. The key is you want to see the devices getting detected.

Update the firmware on your card(s) using /usr/bin/BlackmagicFirmwareUpdater. As root, run:

If the output indicates that the card needs a firmware update, run this:

If you had to update the firmware on your card(s), reboot one more time.

Build ffmpeg

You will need to install the Desktop Video SDK (this is separate from the driver). Download it from the Support section of Blackmagic Design’s web site. As root, do the following:

Install some more build tools. As root:

Prepare the MFX (Intel Media Server Studio libraries) for linking. This is documented in steps 1 and 2 in the section “Installing FFmpeg with Intel® Media Quick Sync Video (Intel® QSV) hardware acceleration support” in the document
Intel QuickSync Video and FFmpeg. I have included them here for convenience.

As root:

As root, create the file /usr/lib64/pkgconfig/libmfx.pc with the contents:

As root, run ldconfig:

In order to build the ffmpeg RPM, you’ll need rpmbuild. As root:

Install gnutls-devel nettle-devel, which are required for the build:

Now our build environment is nearly complete. We need to acquire some of the multimedia libraries that we want to include in our ffmpeg build. Many of them are easily obtained from CentOS itself or EPEL. Use yum to install devel versions of these libraries (which will also install the base RPMs of each as well, since the devel RPMs depend on the base RPMs). As root:

There are a few RPMs that are not readily available. These you will need to build from source.

First install some build prerequisites:

Download the following source RPMs and install them (version numbers as of this writing are in parentheses):

• faac (1.28-7)
• libmp4v2-compat (1.5.0.1-17)
• lame (3.99.5-2)
• rtmpdump (2.4-0)
• xvidcore (1.3.4-1)

If you’re really paranoid, you can download the source tarballs from the various developer groups and replace the tarballs that are installed by the src RPMs. I was that paranoid, but when I downloaded the source files, there was no difference.

Use rpmbuild to build the RPMs. Note that faac depends on libmp4v2-compat, so you have to build libmp4v2-compat and install it before you can build faac.

Now download the x264 spec file and the x264 source. Package up the source properly and put the tarball and the spec file under ~/rpmbuild:

Install the base and devel versions of the resulting RPMs, which will be in ~/rpmbuild/RPMS/x86_64. As root:

Now download the ffmpeg source RPM install it, and build from spec:

Note: this source RPM applies a patch to ffmpeg to support decoding CEA-608 captions from VANC data. It adds a couple of extra command-line options to control which VANC line to look in and where to write the captions (both real-time captions and SRT caption files). It is not beautiful code and it isn’t implemented in an ffmpeg-like way, so there’s no way the team would accept my changes. But it gets the job done for us until there is official support for such captions. If you don’t want that patch applied, remove the “Patch0:” line and the “%patch0” line from the spec file before building.

Finally, install ffmpeg from ~/rpmbuild/RPMS/x86_64. As root:

Testing

First, make sure that you can use the h264_qsv encoder from within ffmpeg. Grab a sample video file and name it test.mp4. Then run this command:

Second, make sure you can capture from the DeckLink card and feed it to h264_qsv. Connect a source to your DeckLink card. Then get its name with this command:

Your output will look something like this, depending on what device(s) you have installed:

You will use the name of the device in your encoding commands, along with an input format specifier. To get the available format values for your card, run this command:

You will get a list of formats like this:

To capture on the 1st card with an input format of 11, you would specify the input as “DeckLink Mini Recorder (1)@11”.

Now capture some video and encode it.

If all goes well, you will be encoding your video on the GPU, allowing for massive throughput. We have been able to capture 1080i video from 3 separate cards and generate seven h264 files/streams from each input, all while only using about 35% of the CPU. Encoding performance for VOD clips is also very impressive, with 5x – 6x real-time transcoding. Through some clever use of filters, you can generate a lot of outputs from your inputs without overloading your system.