Help, I'm dropping a lot of frames when watching Twitch

Hello!

I’m struggling to understand why I am quickly dropping frames when watching Twitch. It happens both on Firefox or Chrome. See the 66 lost frames (i.e. “Images perdues”) in the screenshot, this is just after a few seconds of watching a Twitch stream:

image

I did activate hardware acceleration following these howtos:

Everything seem to be configured properly. I run Fedora 37 (GNOME/Wayland) with an AMD RX 480 GPU and an AMD FX-8320 8-Core CPU. My 8 cores are all at ~10% usage (Chrome or Firefox), which seems suspicious if the video was indeed decoded with the GPU… I have a 500/500 Mbps internet connexion, through Wi-Fi so I get at least 100 Mbps and up to 300 Mbps (which is largely sufficient to handle the 6 Mbps Twitch streams).

Also, I tried the same experiment on my Dell XPS laptop (arguably less powerful), also through WiFi, and I get 0 dropped frames even after a few minutes of watching a stream.

Here’s the output of vainfo:

$ vainfo
Trying display: wayland
libva info: VA-API version 1.16.0
libva info: Trying to open /usr/lib64/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_16
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.16 (libva 2.16.0)
vainfo: Driver version: Mesa Gallium driver 22.3.1 for AMD Radeon RX 480 Graphics (polaris10, LLVM 15.0.6, DRM 3.48, 6.0.15-300.fc37.x86_64)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSlice
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSlice
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileNone                   :	VAEntrypointVideoProc

I think this issue is independent from the browser or the website, but the live nature of Twitch streams makes it more apparent. Most notably I get a lot of audio/video desync. If I keep the stream open for a few hours I can get several seconds between video and audio. And when the resolution is set to “auto”, it switches a lot between the various resolutions.

Am I missing something? Do you have the same issue?

Cheers!

I’m not sure if this page is up-to-date (page history show last revision = 2022-05-26), but it looks like the Radeon RX series cards are not yet fully-supported by the radeonsi driver.

https://www.x.org/wiki/RadeonFeature/

You can see the additional columns for the newer cards here:

There is an interesting note here:

Disregard the above wiki statement. Fedora Linux always ships with the latest AMD drivers.

Scratch that. I misread your output. On closer inspection, it looks like your card is actually part of the “Arctic Islands” series. It should be fully-supported.

is there a way to force the amdgpu driver instead of radeonsi? (both are in the Linux kernel, and from the Wikipedia page it seems that the RX 480 should be compatible with amdgpu, but it is still using radeonsi by default apparently).

I’m not sure that I’m reading it right, but from this chart:

https://www.x.org/wiki/RadeonFeature/

It looks like the A.Islands cards use the amdgpu driver for 2D rendering and the radeonsi driver for 3D rendering (notice the Kernel/3D driver row).

This page appears to indicate that RPM Fusion’s ffmpeg package needs to be installed:

https://fedoraproject.org/wiki/Firefox_Hardware_acceleration#Configure_VA-API_Video_decoding_on_AMD

rpm -qi ffmpeg should work to verify what packager’s ffmpeg package you have installed.

Yeah… Between amdgpu and radeonsi and the various layers in the graphics stack, I’m a bit lost.

ffmpeg is installed too (though I don’t know if Firefox is actually using the ffmpeg installed on the system… it feels strange to depend on a binary and not a library, and that it’s not event recommended in the RPM package)

It’s easy to get lost when trying to configure hardware acceleration. I think it is largely because Linux is always trying to reverse-engineer things. The big companies with the :heavy_dollar_sign: :heavy_dollar_sign: don’t have to do that. Instead, they pay the hardware manufacturers to make the device work the way they want.

The only things I can think of at this point is to make sure that you have RPM Fusion’s ffmpeg (as opposed to Fedora’s ffmpeg) installed (i.e. the @System version in the below output should not show ffmpeg-free).

[/home/gregory]$ dnf --repo=updates,rpmfusion-free-updates provides /usr/bin/ffmpeg
Last metadata expiration check: 0:06:10 ago on Fri 30 Dec 2022 07:38:45 PM CST.
ffmpeg-5.0.2-1.fc36.x86_64 : Digital VCR and streaming server
Repo        : @System
Matched from:
Filename    : /usr/bin/ffmpeg

ffmpeg-5.0.2-1.fc36.x86_64 : Digital VCR and streaming server
Repo        : rpmfusion-free-updates
Matched from:
Filename    : /usr/bin/ffmpeg

ffmpeg-free-5.0.2-1.fc36.x86_64 : A complete solution to record, convert and stream audio and
                                : video
Repo        : updates
Matched from:
Filename    : /usr/bin/ffmpeg

And I think there were some reports that the Flatpak versions of various packages did not work because they could not access the required system libraries due to their sandboxing (i.e. you might want to make sure the below command doesn’t list Firefox on your system).

[/home/gregory]$ flatpak list
Name                                           Application ID                      Version         Branch         Installation
GNOME Application Platform version 42          org.gnome.Platform                                  42             system
Arc-Dark Gtk theme  

But this one should (i.e. it is the RPM version).

[/home/gregory]$ rpm -q firefox
firefox-107.0.1-1.fc36.x86_64

That’s all I can come up with. Sorry. :person_shrugging:

Thanks!

I have ffmpeg installed from RPM fusion, so everything seems fine from here too.

I think we (as in the Linux community) are missing tools to troubleshoot video hardware acceleration. To know if hardware acceleration is enabled and working with a tool actually testing the various video formats. That would reduce the amount of questions a user would ask when trying to figure that out.

I don’t know if it is even technically possible, but a top but for gpu would be awesome! This way we could see if, indeed, Firefox is trying to decode videos using hardware acceleration.

I’ve discovered System Monitoring Center which can give the GPU used in real-time. It helps to know if your GPU is being used when decoding video within Firefox!

It’s not yet perfect (like… it could be just some window manager’s compositing operations that would use the GPU and not the video decoding itself, but it seems to get more active when playing a video on Twitch)

There are also several tools listed here:

https://wiki.archlinux.org/title/benchmarking#Graphics

glxgears is the only one that I’m really familiar with though.

1 Like

So I made a bit more tests and I think the source of the issue is not whether hardware acceleration is enabled or not, but more that the entire rendering chain is rather unstable.

For instance, this is after around 30 minutes of watching Twitch: I lost 4267 frames, that is rather huge!

image

But when I look at the GPU tab in System Monitoring Center, I can clearly see that the GPU is used at around 10% and drops to 0% when I close Chrome:

So in the end, I’d say that I do have hardware acceleration enabled on my machine, but something is happening somewhere in the rendering stack that makes it not as efficient as on other OSes, even if it is accelerated.

FYI, I just noticed this comment on the kernel test-day reports.

amdgpu driver is currently broken since kernel 6.0.14

I wonder if that is the bug you’ve been hitting? If so, it might be fixed in the next kernel release.

The issue in the kernel test seems more serious, the user seems to fallback entirely to software rendering, and the screen resolution is smaller than it should be.

On my side, everything looks fine, it’s just with video playback that I encounter some issues, especially with Twitch where its live nature makes it more prone to lag issues.

When it comes to a top for the GPU, there is radeontop. (Not that I can explain what the numbers mean, but maybe you understand more.)

1 Like

Simple, Twitch uses VP9 codec, and your hardware doesn’t support VP9 hardware accelerated decoding. You can use an extension such as H264ify to solve this problem.

1 Like

I have an AMD RX 480, I’m pretty sure this card support VP9.

Is there a way to:

  1. Know if my card has VP9 hardware acceleration?
  2. Know if the Twitch stream is using VP9? (from what I see, it’s using AVC, which is H.264 AFAIK – see screenshot above)

I think vainfo should show you what codecs your video card supports.