Decoding videos in Firefox with VA-API
Full Potential
Today's graphics cards not only specialize in quickly drawing graphics on the screen, but they can also speed up video playback, reducing the CPU's load. While Firefox supports this optimization, you must manually enable hardware acceleration to quickly decode video on the browser.
For awhile, graphics cards have contained dedicated components for decoding and encoding video formats such as MPEG2, H.264, and H.265. In chip developer jargon, these embedded units are called application-specific integrated circuits (ASICs) [1].
CPUs without ASICs have a hard time decoding videos. However, graphics cards with their specialized circuits can decode video even on low-powered systems like the Raspberry Pi without jerking and with lower power consumption, thus shifting some of the load off the main processor. Lower power consumption means longer battery life and less heat generated, which in turn makes the system quieter.
Interface
On Linux, programs access the graphics chip's acceleration functions via the Video Acceleration API (VA-API) [2]. The vainfo
tool (included in the vainfo or libva-utils packages) shows you the algorithms supported by the graphics card. The VLD
suffix indicates the decoding capability, while EncSlice
denotes the encoding capability.
For example, the integrated Intel 5500 HD graphics chip in the Intel CPU i7 5600U processor (based on the Broadwell architecture) supports decoding and encoding of MPEG2 and H.264, but not VP9 (Listing 1). Intel's eighth generation Core microprocessor ("Coffee Lake") – for example, an i7 9700 with UHD Graphics 630 – can already decode and encode VP9 (Listing 2). An older AMD graphics card only speeds up decoding of H.264, but this GPU does not yet work with VP8 and VP9 (Listing 3).
Listing 1
Intel 5550 HD Graphics Chip
§§number $ vainfo [...] vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 20.2.0 () vainfo: Supported profile and entrypoints [...] VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Simple : VAEntrypointEncSlice [...] VAProfileH264Main : VAEntrypointVLD VAProfileH264Main : VAEntrypointEncSlice VAProfileH264Main : VAEntrypointFEI [...] VAProfileVP8Version0_3 : VAEntrypointVLD
Listing 2
Intel Coffee Lake Graphics Card
§§number $ vainfo libva info: VA-API version 1.3.0 [...] vainfo: Driver version: Intel i965 driver for Intel(R) Coffee Lake - 2.3.0.pre1 (2.3.0.pre1) [...] VAProfileVP9Profile0 : VAEntrypointVLD VAProfileVP9Profile0 : VAEntrypointEncSlice VAProfileVP9Profile2 : VAEntrypointVLD
Listing 3
AMD Graphics Card
$ vainfo [...] vainfo: Driver version: Mesa Gallium driver 20.0.1 for AMD OLAND (DRM 3.39.0, 5.8.0.mx64.339-13250-g66bc99e8bbd14, LLVM 9.0.1) VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Main : VAEntrypointVLD VAProfileVC1Simple : VAEntrypointVLD VAProfileVC1Main : VAEntrypointVLD VAProfileVC1Advanced : VAEntrypointVLD VAProfileH264ConstrainedBaseline: VAEntrypointVLD VAProfileH264Main : VAEntrypointVLD VAProfileH264High : VAEntrypointVLD VAProfileNone : VAEntrypointVideoProc
Because not all required software components are under a free license, you have to enable VA-API support on Ubuntu or Debian (for example) for fifth generation Intel chips (Broadwell) by installing intel-media-va-driver-non-free from the package sources.
Then you use the mpv media player to check if the graphics card is used during decoding. To do this, call the player with the --hwdec=vaapi
option. If the program reports
VO: [...] vaapi
when it starts playing back a video, the interface is working (Listing 4). Involving the dedicated decoder chip reduces the entire system's energy consumption. For example, PowerTOP now reports only 6.32W (Figure 1) instead of 7.87W (Figure 2).
Listing 4
Checking Graphics Card Use
§§number $ mpv --hwdec=vaapi /<path>/<to>/video.mkv [...] VO: [gpu] 1920x1080 vaapi[nv12] VO: [gpu] 1920x1080 yuv420p [...]
Web Browser
Today many Internet sites – not least YouTube – integrate videos, resulting in web browsers often functioning as video playback programs. Users can benefit in particular if the browser can access the chips on the graphics card during decoding. With the WebRender [3] compositor, Mozilla laid the foundation for using the graphics card to display web pages. WebRender lets you use hardware acceleration in addition to the GL Compositor. However, even the current Firefox 80 only enables the function for Windows 10 [4] by default, because Linux developers complain that there are too many problems with the graphics card drivers.
As is so often the case, a Red Hat developer has taken the Linux desktop another step forward with Linux graphics card drivers. Martin Stransky programmed the changes for Firefox; from Fedora 31 on, they are implemented in Firefox 77.0. In his blog post [5], Stransky reports that thanks to Wayland, many errors in the Linux graphics card drivers have now been discovered and fixed, as Wayland makes intensive use of the acceleration functions. For hardware-accelerated decoding and encoding, the developer resorts to the proven FFmpeg [6], for which Firefox writes the material directly to the graphics card memory via the dma-buf subsystem, thus removing the need to copy the data. By default, the functions are disabled.
In a Wayland desktop environment, calling
MOZ_ENABLE_WAYLAND=1 MOZ_WEBRENDER=1 firefox
from the terminal activates WebRender, as well as the VA-API interface. After appropriate configuration, Listing 5 shows the browser call with VA-API in an X11 environment, which Firefox has supported since version 80. The MOZ_LOG
variable controls the output messages.
Listing 5
Browser Call with VA-API
§§number $ MOZ_X11_EGL=1 MOZ_WEBRENDER=1 MOZ_LOG="PlatformDecoderModule:5" firefox [...] [Child 19296: MediaPDecoder #3]: D/PlatformDecoderModule Choosing FFmpeg pixel format for VA-API video decoding. [Child 19296: MediaPDecoder #3]: D/PlatformDecoderModule Requesting pixel format VAAPI_VLD [h264 @ 0x7f01f5720000] Format vaapi_vld chosen by get_format(). [h264 @ 0x7f01f5720000] Format vaapi_vld requires hwaccel initialization. [h264 @ 0x7f01f5720000] Considering format 0x3231564e -> nv12. [h264 @ 0x7f01f5720000] Picked nv12 (0x3231564e) as best match for yuv420p. [...]
Next, open the configuration editor with the pseudo-URL about:config and enable VA-API support by double-clicking on media.ffmpeg.vaapi.enabled to switch from false to true. To find out if WebRender works, check the URL about:support in the Graphics section. This is where Firefox should output information about WebRender.
Currently, YouTube delivers VP9-encoded movies by default due to the smaller size for the same quality. For hardware-accelerated VP8/VP9 playback in Firefox, do not load the ffvpx library provided by Firefox, because it cannot use the VA-API interface. Setting the media.ffvpx.enabled key to false in about:config tells Firefox to use the system FFmpeg libraries [7]. Some distributions already configure Firefox accordingly.
If the graphics card does not support H.264, the Firefox h264ify [8] or enhanced-h264ify [9] extensions block formats encoded with VP8 and VP9. If you find messages in the logs such as
D/PlatformDecoderModule DMA-Buf/VA-API can't be used, WebRender/DMA-Buf is disabled
please check the settings again.
Troubleshooting
If problems occur when playing videos, you should first check whether the errors still occur in the nightly build of the browser with the latest changes. If the bugs are confirmed, you can contribute to the project by reporting the problem to Mozilla's bug tracker [10], appending the output of MOZ_LOG="PlatformDecoderModule:5"
.
Download the current nightly build [11] for Linux to your computer as shown in Listing 6 and unpack the archive; the last command starts the browser. Make sure that you close the regular Firefox instance completely before doing this. After completing testing, you can delete the firefox/
directory created by unpacking.
Listing 6
Downloading the Firefox Nightly Build
§§number ### Download 64-bit version: $ wget 'https://download.mozilla.org/?product=firefox-nightly-latest-ssl&os=linux64&lang=en-US' -O firefox-nightly64.tar.bz2 ### Download 32-bit version: $ wget 'https://download.mozilla.org/?product=firefox-nightly-latest-ssl&os=linux&lang=en-US' -O firefox-nightly32.tar.bz2 ### Unzip and call: $ tar xf firefox-nightly*.tar.bz2 $ firefox/firefox
Buy this article as PDF
(incl. VAT)