Can't get AMD dGPU + iGPU to work together

I’m on Fedora 35 Silverblue. I have a Ryzen 5600G which includes a iGPU and I also have a discrete Vega64. I have two monitors connected to the dGPU and a third on the iGPU. Both GPUs are enabled in BIOS – Windows has no trouble seeing and using all three monitors.

But Fedora only sees the two monitors on my discrete GPU. It looks like something goes wrong during driver initialization. Here is what I see in kernel logs:

[    4.566223] [drm] amdgpu kernel modesetting enabled.
[    4.566404] amdgpu: Virtual CRAT table created for CPU
[    4.566410] amdgpu: Topology: Add CPU node
[    4.566467] checking generic (d0000000 300000) vs hw (d0000000 10000000)
[    4.566469] checking generic (d0000000 300000) vs hw (d0000000 10000000)
[    4.566470] fb0: switching to amdgpu from EFI VGA
[    4.566550] amdgpu 0000:12:00.0: vgaarb: deactivate vga console
[    4.566595] amdgpu 0000:12:00.0: enabling device (0006 -> 0007)
[    4.566639] [drm] initializing kernel modesetting (VEGA10 0x1002:0x687F 0x1DA2:0xE37F 0xC1).
[    4.566642] amdgpu 0000:12:00.0: amdgpu: Trusted Memory Zone (TMZ) feature not supported
[    4.566653] [drm] register mmio base: 0xFCC00000
[    4.566654] [drm] register mmio size: 524288
[    4.566662] [drm] add ip block number 0 <soc15_common>
[    4.566663] [drm] add ip block number 1 <gmc_v9_0>
[    4.566664] [drm] add ip block number 2 <vega10_ih>
[    4.566665] [drm] add ip block number 3 <psp>
[    4.566666] [drm] add ip block number 4 <powerplay>
[    4.566667] [drm] add ip block number 5 <dm>
[    4.566668] [drm] add ip block number 6 <gfx_v9_0>
[    4.566668] [drm] add ip block number 7 <sdma_v4_0>
[    4.566669] [drm] add ip block number 8 <uvd_v7_0>
[    4.566670] [drm] add ip block number 9 <vce_v4_0>
[    4.566805] amdgpu 0000:12:00.0: amdgpu: Fetched VBIOS from VFCT
[    4.566807] amdgpu: ATOM BIOS: 113-D0500110-O01
[    4.567747] [drm] UVD(0) is enabled in VM mode
[    4.567748] [drm] UVD(0) ENC is enabled in VM mode
[    4.567748] [drm] VCE enabled in VM mode
[    4.567780] amdgpu 0000:12:00.0: amdgpu: MEM ECC is not presented.
[    4.567781] amdgpu 0000:12:00.0: amdgpu: SRAM ECC is not presented.
[    4.567783] [drm] vm size is 262144 GB, 4 levels, block size is 9-bit, fragment size is 9-bit
[    4.567788] amdgpu 0000:12:00.0: amdgpu: VRAM: 8176M 0x000000F400000000 - 0x000000F5FEFFFFFF (8176M used)
[    4.567789] amdgpu 0000:12:00.0: amdgpu: GART: 512M 0x0000000000000000 - 0x000000001FFFFFFF
[    4.567790] amdgpu 0000:12:00.0: amdgpu: AGP: 267419648M 0x000000F800000000 - 0x0000FFFFFFFFFFFF
[    4.567795] [drm] Detected VRAM RAM=8176M, BAR=256M
[    4.567796] [drm] RAM width 2048bits HBM
[    4.567817] [drm] amdgpu: 8176M of VRAM memory ready
[    4.567818] [drm] amdgpu: 8176M of GTT memory ready.
[    4.567821] [drm] GART: num cpu pages 131072, num gpu pages 131072
[    4.567971] [drm] PCIE GART of 512M enabled.
[    4.567972] [drm] PTB located at 0x000000F400900000
[    4.570099] amdgpu 0000:12:00.0: amdgpu: PSP runtime database doesn't exist
[    4.570102] amdgpu: hwmgr_sw_init smu backed is vega10_smu
[    4.590219] [drm] Found UVD firmware Version: 66.43 Family ID: 17
[    4.590229] [drm] PSP loading UVD firmware
[    4.595474] [drm] Found VCE firmware Version: 57.6 Binary ID: 4
[    4.595478] [drm] PSP loading VCE firmware
[    4.783797] [drm] reserve 0x400000 from 0xf5fec00000 for PSP TMR
[    4.830338] [drm] Display Core initialized with v3.2.160!
[    4.997052] [drm] kiq ring mec 2 pipe 1 q 0
[    5.018588] [drm] UVD and UVD ENC initialized successfully.
[    5.219149] [drm] VCE initialized successfully.
[    5.220023] kfd kfd: amdgpu: Allocated 3969056 bytes on gart
[    5.247348] memmap_init_zone_device initialised 2097152 pages in 11ms
[    5.247354] amdgpu: HMM registered 8176MB device memory
[    5.247376] amdgpu: SRAT table not found
[    5.247377] amdgpu: Virtual CRAT table created for GPU
[    5.247462] amdgpu: Topology: Add dGPU node [0x687f:0x1002]
[    5.247465] kfd kfd: amdgpu: added device 1002:687f
[    5.247481] amdgpu 0000:12:00.0: amdgpu: SE 4, SH per SE 1, CU per SH 16, active_cu_number 64
[    5.250808] [drm] fb mappable at 0xD0CEE000
[    5.250810] [drm] vram apper at 0xD0000000
[    5.250811] [drm] size 33177600
[    5.250811] [drm] fb depth is 24
[    5.250812] [drm]    pitch is 15360
[    5.251216] fbcon: amdgpudrmfb (fb0) is primary device
[    5.251217] fbcon: Deferring console take-over
[    5.251219] amdgpu 0000:12:00.0: [drm] fb0: amdgpudrmfb frame buffer device
[    5.263949] amdgpu 0000:12:00.0: amdgpu: ring gfx uses VM inv eng 0 on hub 0
[    5.263951] amdgpu 0000:12:00.0: amdgpu: ring comp_1.0.0 uses VM inv eng 1 on hub 0
[    5.263952] amdgpu 0000:12:00.0: amdgpu: ring comp_1.1.0 uses VM inv eng 4 on hub 0
[    5.263953] amdgpu 0000:12:00.0: amdgpu: ring comp_1.2.0 uses VM inv eng 5 on hub 0
[    5.263954] amdgpu 0000:12:00.0: amdgpu: ring comp_1.3.0 uses VM inv eng 6 on hub 0
[    5.263955] amdgpu 0000:12:00.0: amdgpu: ring comp_1.0.1 uses VM inv eng 7 on hub 0
[    5.263956] amdgpu 0000:12:00.0: amdgpu: ring comp_1.1.1 uses VM inv eng 8 on hub 0
[    5.263956] amdgpu 0000:12:00.0: amdgpu: ring comp_1.2.1 uses VM inv eng 9 on hub 0
[    5.263957] amdgpu 0000:12:00.0: amdgpu: ring comp_1.3.1 uses VM inv eng 10 on hub 0
[    5.263958] amdgpu 0000:12:00.0: amdgpu: ring kiq_2.1.0 uses VM inv eng 11 on hub 0
[    5.263959] amdgpu 0000:12:00.0: amdgpu: ring sdma0 uses VM inv eng 0 on hub 1
[    5.263960] amdgpu 0000:12:00.0: amdgpu: ring page0 uses VM inv eng 1 on hub 1
[    5.263960] amdgpu 0000:12:00.0: amdgpu: ring sdma1 uses VM inv eng 4 on hub 1
[    5.263961] amdgpu 0000:12:00.0: amdgpu: ring page1 uses VM inv eng 5 on hub 1
[    5.263962] amdgpu 0000:12:00.0: amdgpu: ring uvd_0 uses VM inv eng 6 on hub 1
[    5.263963] amdgpu 0000:12:00.0: amdgpu: ring uvd_enc_0.0 uses VM inv eng 7 on hub 1
[    5.263963] amdgpu 0000:12:00.0: amdgpu: ring uvd_enc_0.1 uses VM inv eng 8 on hub 1
[    5.263964] amdgpu 0000:12:00.0: amdgpu: ring vce0 uses VM inv eng 9 on hub 1
[    5.263965] amdgpu 0000:12:00.0: amdgpu: ring vce1 uses VM inv eng 10 on hub 1
[    5.263966] amdgpu 0000:12:00.0: amdgpu: ring vce2 uses VM inv eng 11 on hub 1
[    5.265370] [drm] Initialized amdgpu 3.44.0 20150101 for 0000:12:00.0 on minor 0
[    5.265526] amdgpu 0000:30:00.0: enabling device (0006 -> 0007)
[    5.265596] [drm] initializing kernel modesetting (RENOIR 0x1002:0x1638 0x1002:0x1636 0xC9).
[    5.265599] amdgpu 0000:30:00.0: amdgpu: Trusted Memory Zone (TMZ) feature enabled
[    5.265612] [drm] register mmio base: 0xFC900000
[    5.265613] [drm] register mmio size: 524288
[    5.266705] [drm] add ip block number 0 <soc15_common>
[    5.266706] [drm] add ip block number 1 <gmc_v9_0>
[    5.266707] [drm] add ip block number 2 <vega10_ih>
[    5.266707] [drm] add ip block number 3 <psp>
[    5.266708] [drm] add ip block number 4 <smu>
[    5.266708] [drm] add ip block number 5 <dm>
[    5.266709] [drm] add ip block number 6 <gfx_v9_0>
[    5.266709] [drm] add ip block number 7 <sdma_v4_0>
[    5.266710] [drm] add ip block number 8 <vcn_v2_0>
[    5.266710] [drm] add ip block number 9 <jpeg_v2_0>
[    5.266719] amdgpu 0000:30:00.0: amdgpu: Fetched VBIOS from VFCT
[    5.266720] amdgpu: ATOM BIOS: 13-CEZANNE-019
[    5.267266] [drm] VCN decode is enabled in VM mode
[    5.267267] [drm] VCN encode is enabled in VM mode
[    5.267267] [drm] JPEG decode is enabled in VM mode
[    5.267268] amdgpu 0000:30:00.0: amdgpu: PCIE atomic ops is not supported
[    5.267301] [drm] vm size is 262144 GB, 4 levels, block size is 9-bit, fragment size is 9-bit
[    5.267305] amdgpu 0000:30:00.0: amdgpu: VRAM: 32M 0x000000F400000000 - 0x000000F401FFFFFF (32M used)
[    5.267306] amdgpu 0000:30:00.0: amdgpu: GART: 1024M 0x0000000000000000 - 0x000000003FFFFFFF
[    5.267307] amdgpu 0000:30:00.0: amdgpu: AGP: 267419648M 0x000000F800000000 - 0x0000FFFFFFFFFFFF
[    5.267312] [drm] Detected VRAM RAM=32M, BAR=32M
[    5.267313] [drm] RAM width 128bits DDR4
[    5.267563] [drm] amdgpu: 32M of VRAM memory ready
[    5.267563] [drm] amdgpu: 3072M of GTT memory ready.
[    5.267566] [drm] GART: num cpu pages 262144, num gpu pages 262144
[    5.267681] [drm] PCIE GART of 1024M enabled.
[    5.267683] [drm] PTB located at 0x000000F400900000
[    5.270187] amdgpu 0000:30:00.0: amdgpu: PSP runtime database doesn't exist
[    5.273519] [drm] Loading DMUB firmware via PSP: version=0x0101001C
[    5.293090] [drm] Found VCN firmware Version ENC: 1.16 DEC: 5 VEP: 0 Revision: 3
[    5.293094] amdgpu 0000:30:00.0: amdgpu: Will use PSP to load VCN firmware
[    6.019335] [drm] reserve 0x400000 from 0xf401800000 for PSP TMR
[    6.097346] amdgpu 0000:30:00.0: amdgpu: RAS: optional ras ta ucode is not available
[    6.106689] amdgpu 0000:30:00.0: amdgpu: RAP: optional rap ta ucode is not available
[    6.106690] amdgpu 0000:30:00.0: amdgpu: SECUREDISPLAY: securedisplay ta ucode is not available
[    6.106691] amdgpu: smu firmware loading failed
[    6.106692] amdgpu 0000:30:00.0: amdgpu: amdgpu_device_ip_init failed
[    6.106693] amdgpu 0000:30:00.0: amdgpu: Fatal error during GPU init
[    6.106694] amdgpu 0000:30:00.0: amdgpu: amdgpu: finishing device.
[    6.112146] [drm] free PSP TMR buffer
[    6.138843] amdgpu: probe of 0000:30:00.0 failed with error -95
[    6.141461] ------------[ cut here ]------------
[    6.141462] Memory manager not clean during takedown.
[    6.141478] WARNING: CPU: 3 PID: 519 at drivers/gpu/drm/drm_mm.c:995 drm_mm_takedown+0x1f/0x30 [drm]
[    6.141495] Modules linked in: amdgpu(+) iommu_v2 gpu_sched i2c_algo_bit drm_kms_helper crct10dif_pclmul cec crc32_pclmul crc32c_intel drm_ttm_helper ttm ghash_clmulni_intel nvme mxm_wmi nvme_core drm ccp sp5100_tco video wmi ipmi_devintf ipmi_msghandler fuse
[    6.141504] CPU: 3 PID: 519 Comm: systemd-udevd Not tainted 5.16.12-200.fc35.x86_64 #1
[    6.141505] Hardware name: Micro-Star International Co., Ltd. MS-7B78/X470 GAMING PRO CARBON (MS-7B78), BIOS 2.H7 01/25/2022
[    6.141506] RIP: 0010:drm_mm_takedown+0x1f/0x30 [drm]
[    6.141517] Code: 48 e9 6a ff ff ff 0f 1f 44 00 00 0f 1f 44 00 00 48 8b 47 38 48 83 c7 38 48 39 f8 75 01 c3 48 c7 c7 c0 dc 31 c0 e8 24 1f 96 ef <0f> 0b c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 0f 1f 44 00 00
[    6.141517] RSP: 0018:ffffbb73c089bc08 EFLAGS: 00010296
[    6.141518] RAX: 0000000000000029 RBX: ffff925a64e00000 RCX: 0000000000000000
[    6.141519] RDX: 0000000000000002 RSI: ffffffffb063494d RDI: 00000000ffffffff
[    6.141520] RBP: ffff925a64e05d88 R08: 0000000000000000 R09: ffffbb73c089ba48
[    6.141520] R10: ffffbb73c089ba40 R11: ffffffffb0f462a8 R12: ffff925a64e05d88
[    6.141521] R13: ffff925a64e05d70 R14: ffff925a64e05d88 R15: ffff925a416f137c
[    6.141521] FS:  00007f6983333b40(0000) GS:ffff92694bac0000(0000) knlGS:0000000000000000
[    6.141522] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    6.141523] CR2: 000055d66920f000 CR3: 00000001088d8000 CR4: 0000000000750ee0
[    6.141524] PKRU: 55555554
[    6.141524] Call Trace:
[    6.141526]  <TASK>
[    6.141527]  amdgpu_vram_mgr_fini+0xe0/0x150 [amdgpu]
[    6.141656]  amdgpu_ttm_fini+0x94/0x100 [amdgpu]
[    6.141740]  gmc_v9_0_sw_fini+0x3a/0x40 [amdgpu]
[    6.141828]  amdgpu_device_fini_sw+0xbb/0x2a0 [amdgpu]
[    6.141925]  amdgpu_driver_release_kms+0x12/0x30 [amdgpu]
[    6.142009]  devm_drm_dev_init_release+0x3d/0x60 [drm]
[    6.142020]  devres_release_all+0xa8/0xe0
[    6.142026]  really_probe+0x100/0x3f0
[    6.142028]  __driver_probe_device+0xfe/0x180
[    6.142030]  driver_probe_device+0x1e/0x90
[    6.142031]  __driver_attach+0xc0/0x1c0
[    6.142032]  ? __device_attach_driver+0xe0/0xe0
[    6.142033]  ? __device_attach_driver+0xe0/0xe0
[    6.142034]  bus_for_each_dev+0x64/0x90
[    6.142037]  bus_add_driver+0x149/0x1e0
[    6.142038]  driver_register+0x8f/0xe0
[    6.142040]  ? 0xffffffffc0ca7000
[    6.142041]  do_one_initcall+0x44/0x200
[    6.142044]  ? kmem_cache_alloc_trace+0x163/0x2c0
[    6.142046]  do_init_module+0x4c/0x260
[    6.142050]  __do_sys_init_module+0x11d/0x180
[    6.142052]  do_syscall_64+0x3b/0x90
[    6.142055]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[    6.142058] RIP: 0033:0x7f6983ecd92e
[    6.142059] Code: 48 8b 0d fd 74 0e 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 49 89 ca b8 af 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d ca 74 0e 00 f7 d8 64 89 01 48
[    6.142060] RSP: 002b:00007ffdfd30c3e8 EFLAGS: 00000246 ORIG_RAX: 00000000000000af
[    6.142061] RAX: ffffffffffffffda RBX: 000055d66720d810 RCX: 00007f6983ecd92e
[    6.142062] RDX: 000055d6671f9bd0 RSI: 0000000000f299ee RDI: 000055d6682e6d80
[    6.142062] RBP: 000055d6682e6d80 R08: 000055d6671f2300 R09: 0000000000f28010
[    6.142062] R10: 0000000000000005 R11: 0000000000000246 R12: 000055d6671f9bd0
[    6.142063] R13: 000055d667202a30 R14: 0000000000000000 R15: 000055d667211a90
[    6.142064]  </TASK>
[    6.142064] ---[ end trace 6da5ef28779e69c7 ]---

There are a bunch of errors while loading the “Cezanne” device (which is the 5600G). Afterwards, it does show up in lspci -k but without any “Kernel driver in use” line, which I suspect means it’s not actually enabled:

12:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 XL/XT [Radeon RX Vega 56/64] (rev c1)
	Subsystem: Sapphire Technology Limited Device e37f
	Kernel driver in use: amdgpu
	Kernel modules: amdgpu
30:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cezanne (rev c9)
	Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Device 1636
	Kernel modules: amdgpu

Any ideas what’s wrong? Thank you!

After reading through the sources, I realized the “smu” is a component used for power management. Because of occasional hangs in GPU-intensive games I had disabled DPM (dynamic power management) for the driver by adding the kernel argument amdgpu.dpm=0. That made me suspicious.

It turns out that everything works fine with DPM is enabled, and the system initializes the iGPU well. It looks like a pretty clear bug, which I filed here.

1 Like