Screen does not light up after the suspend-resume

I saw this issue in several places, and I tried most (if not all) of the suggested solutions, but I still get a black screen when the laptop tries to resume from suspension.

In the past I had this issue and was able to solve with by reinstalling the NVIDIA drivers, or by re-enable the power services,

systemctl status nvidia-{resume,suspend,hibernate}

I also tried to switch to negativo17 nvidia repo/packages, and I still have the same issue.

from the journal log I have these messages:
(the only part which looks somewhat related is the stack trace @ 09:53:38)

May 01 03:23:54 localhost systemd[1]: Reached target Sleep.
May 01 03:23:54 localhost systemd[1]: Starting NVIDIA system suspend actions...
May 01 03:23:54 localhost suspend[123995]: nvidia-suspend.service
May 01 03:23:54 localhost logger[123995]: <13>May  1 03:23:54 suspend: nvidia-suspend.service
May 01 03:23:55 localhost systemd[1]: nvidia-suspend.service: Deactivated successfully.
May 01 03:23:55 localhost systemd[1]: Finished NVIDIA system suspend actions.
May 01 03:23:55 localhost audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=nvidia-suspend comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
May 01 03:23:55 localhost audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=nvidia-suspend comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
May 01 03:23:55 localhost systemd[1]: Starting System Suspend...
May 01 03:23:55 localhost systemd-sleep[124020]: Entering sleep state 'suspend'...
May 01 03:23:55 localhost kernel: PM: suspend entry (deep)
May 01 03:23:55 localhost kernel: Filesystems sync: 0.090 seconds
May 01 09:53:23 localhost kernel: Freezing user space processes ... (elapsed 0.001 seconds) done.
May 01 09:53:23 localhost kernel: OOM killer disabled.
May 01 09:53:23 localhost kernel: Freezing remaining freezable tasks ... (elapsed 0.000 seconds) done.
May 01 09:53:23 localhost kernel: printk: Suspending console(s) (use no_console_suspend to debug)
May 01 09:53:23 localhost kernel: sd 3:0:0:0: [sda] Synchronizing SCSI cache
May 01 09:53:23 localhost kernel: sd 3:0:0:0: [sda] Stopping disk
May 01 09:53:23 localhost kernel: e1000e: EEE TX LPI TIMER: 00000011
May 01 09:53:23 localhost kernel: PM: suspend devices took 0.722 seconds
May 01 09:53:23 localhost kernel: ACPI: EC: interrupt blocked
May 01 09:53:23 localhost kernel: ACPI: PM: Preparing to enter system sleep state S3
May 01 09:53:23 localhost kernel: ACPI: EC: event blocked
May 01 09:53:23 localhost kernel: ACPI: EC: EC stopped
May 01 09:53:23 localhost kernel: ACPI: PM: Saving platform NVS memory
May 01 09:53:23 localhost kernel: Disabling non-boot CPUs ...
May 01 09:53:23 localhost kernel: smpboot: CPU 1 is now offline
May 01 09:53:23 localhost kernel: smpboot: CPU 2 is now offline
May 01 09:53:23 localhost kernel: smpboot: CPU 3 is now offline
May 01 09:53:23 localhost kernel: smpboot: CPU 4 is now offline
May 01 09:53:23 localhost kernel: smpboot: CPU 5 is now offline
May 01 09:53:23 localhost kernel: smpboot: CPU 6 is now offline
May 01 09:53:23 localhost kernel: smpboot: CPU 7 is now offline
May 01 09:53:23 localhost kernel: ACPI: PM: Low-level resume complete
May 01 09:53:23 localhost kernel: ACPI: EC: EC started
May 01 09:53:23 localhost kernel: ACPI: PM: Restoring platform NVS memory
May 01 09:53:23 localhost kernel: Enabling non-boot CPUs ...
May 01 09:53:23 localhost kernel: x86: Booting SMP configuration:
May 01 09:53:23 localhost kernel: smpboot: Booting Node 0 Processor 1 APIC 0x2
May 01 09:53:23 localhost kernel: CPU1 is up
May 01 09:53:23 localhost kernel: smpboot: Booting Node 0 Processor 2 APIC 0x4
May 01 09:53:23 localhost kernel: CPU2 is up
May 01 09:53:23 localhost kernel: smpboot: Booting Node 0 Processor 3 APIC 0x6
May 01 09:53:23 localhost kernel: CPU3 is up
May 01 09:53:23 localhost kernel: smpboot: Booting Node 0 Processor 4 APIC 0x1
May 01 09:53:23 localhost kernel: CPU4 is up
May 01 09:53:23 localhost kernel: smpboot: Booting Node 0 Processor 5 APIC 0x3
May 01 09:53:23 localhost kernel: CPU5 is up
May 01 09:53:23 localhost kernel: smpboot: Booting Node 0 Processor 6 APIC 0x5
May 01 09:53:23 localhost kernel: CPU6 is up
May 01 09:53:23 localhost kernel: smpboot: Booting Node 0 Processor 7 APIC 0x7
May 01 09:53:23 localhost kernel: CPU7 is up
May 01 09:53:23 localhost kernel: ACPI: PM: Waking up from system sleep state S3
May 01 09:53:23 localhost kernel: ACPI: EC: interrupt unblocked
May 01 09:53:23 localhost kernel: sd 3:0:0:0: [sda] Starting disk
May 01 09:53:23 localhost kernel: ACPI: EC: event unblocked
May 01 09:53:23 localhost kernel: nvme nvme0: Shutdown timeout set to 8 seconds
May 01 09:53:23 localhost kernel: nvme nvme0: 8/0/0 default/read/poll queues
May 01 09:53:23 localhost kernel: usb 2-4: Disable of device-initiated U1 failed.
May 01 09:53:23 localhost kernel: usb 2-4: Disable of device-initiated U2 failed.
May 01 09:53:23 localhost kernel: usb 2-4: reset SuperSpeed USB device number 2 using xhci_hcd
May 01 09:53:23 localhost kernel: ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 320)
May 01 09:53:23 localhost kernel: ata4.00: ACPI cmd f5/00:00:00:00:00:a0(SECURITY FREEZE LOCK) filtered out
May 01 09:53:23 localhost kernel: ata4.00: ACPI cmd ef/10:03:00:00:00:a0(SET FEATURES) filtered out
May 01 09:53:23 localhost kernel: ata4.00: supports DRM functions and may not be fully accessible
May 01 09:53:23 localhost kernel: ata4.00: Security Log not supported
May 01 09:53:23 localhost kernel: ata4.00: ACPI cmd f5/00:00:00:00:00:a0(SECURITY FREEZE LOCK) filtered out
May 01 09:53:23 localhost kernel: ata4.00: ACPI cmd ef/10:03:00:00:00:a0(SET FEATURES) filtered out
May 01 09:53:23 localhost kernel: ata4.00: supports DRM functions and may not be fully accessible
May 01 09:53:23 localhost kernel: ata4.00: Security Log not supported
May 01 09:53:23 localhost kernel: ata4.00: configured for UDMA/133
May 01 09:53:23 localhost kernel: ahci 0000:00:17.0: port does not support device sleep
May 01 09:53:23 localhost kernel: usb 1-13: reset low-speed USB device number 7 using xhci_hcd
May 01 09:53:23 localhost kernel: usb 2-4.4: Disable of device-initiated U1 failed.
May 01 09:53:23 localhost kernel: usb 2-4.4: Disable of device-initiated U2 failed.
May 01 09:53:23 localhost kernel: usb 1-9: reset full-speed USB device number 5 using xhci_hcd
May 01 09:53:23 localhost kernel: usb 1-14: reset full-speed USB device number 9 using xhci_hcd
May 01 09:53:23 localhost kernel: usb 1-4: reset high-speed USB device number 2 using xhci_hcd
May 01 09:53:23 localhost kernel: usb 1-8: reset high-speed USB device number 3 using xhci_hcd
May 01 09:53:23 localhost kernel: usb 2-4.4: reset SuperSpeed USB device number 3 using xhci_hcd
May 01 09:53:23 localhost kernel: usb 1-4.1: reset full-speed USB device number 4 using xhci_hcd
May 01 09:53:23 localhost kernel: usb 1-4.4: reset high-speed USB device number 8 using xhci_hcd
May 01 09:53:23 localhost kernel: usb 1-4.2: reset full-speed USB device number 6 using xhci_hcd
May 01 09:53:23 localhost kernel: usb 2-4.4.1: reset SuperSpeed USB device number 4 using xhci_hcd
May 01 09:53:23 localhost kernel: usb 1-4.4.2: reset high-speed USB device number 12 using xhci_hcd
May 01 09:53:23 localhost kernel: r8152 1-4.4.2:1.0: skip request firmware
May 01 09:53:23 localhost kernel: r8152 1-4.4.2:1.0: load rtl8153a-3 v2 02/07/20 successfully
May 01 09:53:23 localhost kernel: usb 1-4.4.1: reset high-speed USB device number 11 using xhci_hcd
May 01 09:53:23 localhost kernel: usb 1-4.4.3: reset full-speed USB device number 13 using xhci_hcd
May 01 09:53:23 localhost kernel: usb 1-4.4.1.3: reset high-speed USB device number 15 using xhci_hcd
May 01 09:53:23 localhost kernel: PM: resume devices took 3.166 seconds
May 01 09:53:23 localhost kernel: Bluetooth: hci0: Bootloader revision 0.0 build 2 week 52 2014
May 01 09:53:23 localhost kernel: OOM killer enabled.
May 01 09:53:23 localhost kernel: Bluetooth: hci0: Device revision is 5
May 01 09:53:23 localhost kernel: Restarting tasks ... 
May 01 09:53:23 localhost kernel: Bluetooth: hci0: Secure boot is enabled
May 01 09:53:23 localhost kernel: usb 1-4.4.1.2: USB disconnect, device number 14
May 01 09:53:23 localhost kernel: done.
May 01 09:53:23 localhost kernel: Bluetooth: hci0: OTP lock is enabled
May 01 09:53:23 localhost kernel: Bluetooth: hci0: API lock is enabled
May 01 09:53:23 localhost kernel: Bluetooth: hci0: Debug lock is disabled
May 01 09:53:23 localhost kernel: Bluetooth: hci0: Minimum firmware build 1 week 10 2014
May 01 09:53:23 localhost kernel: Bluetooth: hci0: Found device firmware: intel/ibt-11-5.sfi
May 01 09:53:23 localhost kernel: thermal thermal_zone2: failed to read out thermal zone (-61)
May 01 09:53:23 localhost audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=sssd-kcm comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
May 01 09:53:23 localhost sssd_kcm[120883]: Shutting down (status = 0)
May 01 09:53:23 localhost rtkit-daemon[1794]: The canary thread is apparently starving. Taking action.
May 01 09:53:23 localhost systemd[1]: sssd-kcm.service: Deactivated successfully.
May 01 09:53:23 localhost rtkit-daemon[1794]: Demoting known real-time threads.
May 01 09:53:23 localhost rtkit-daemon[1794]: Successfully demoted thread 3635 of process 3621 (/usr/bin/wireplumber).
May 01 09:53:23 localhost rtkit-daemon[1794]: Successfully demoted thread 3621 of process 3621 (/usr/bin/wireplumber).
May 01 09:53:23 localhost rtkit-daemon[1794]: Successfully demoted thread 3639 of process 3622 (/usr/bin/pipewire-pulse).
May 01 09:53:23 localhost rtkit-daemon[1794]: Successfully demoted thread 3622 of process 3622 (/usr/bin/pipewire-pulse).
May 01 09:53:23 localhost rtkit-daemon[1794]: Successfully demoted thread 3636 of process 3620 (/usr/bin/pipewire).
May 01 09:53:23 localhost rtkit-daemon[1794]: Successfully demoted thread 3620 of process 3620 (/usr/bin/pipewire).
May 01 09:53:23 localhost rtkit-daemon[1794]: Demoted 6 threads.
May 01 09:53:23 localhost systemd[1]: Starting system activity accounting tool...
May 01 09:53:24 localhost systemd[1]: sysstat-collect.service: Deactivated successfully.
May 01 09:53:24 localhost audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=sysstat-collect comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
May 01 09:53:24 localhost audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=sysstat-collect comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
May 01 09:53:24 localhost systemd[1]: Finished system activity accounting tool.
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-14
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-9
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb2/2-4
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb2/2-4
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-14
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-9
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb2/2-4/2-4.4
May 01 09:53:24 localhost systemd[1]: Stopped target Bluetooth Support.
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb2/2-4/2-4.4
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb2/2-4/2-4.4/2-4.4.1
May 01 09:53:24 localhost systemd[1]: Reached target Bluetooth Support.
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4
May 01 09:53:24 localhost systemd[2834]: Reached target Bluetooth.
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4.4
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4.2
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb2/2-4/2-4.4/2-4.4.1
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4.2
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4.4
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4.4/1-4.4.3
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4.4/1-4.4.2
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4.4/1-4.4.2
May 01 09:53:24 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4.4/1-4.4.3
May 01 09:53:25 localhost NetworkManager[1919]: <info>  [1651388005.4692] device (enp0s20f0u4u4u2): carrier: link connected
May 01 09:53:25 localhost kernel: r8152 1-4.4.2:1.0 enp0s20f0u4u4u2: carrier on
May 01 09:53:27 localhost kernel: PM: suspend exit
May 01 09:53:27 localhost systemd-sleep[124020]: System returned from sleep state.
May 01 09:53:28 localhost audit[124473]: AVC avc:  denied  { write } for  pid=124473 comm="tpacpi-bat" name="call" dev="proc" ino=4026532672 scontext=system_u:system_r:tlp_t:s0 tcontext=system_u:object_r:proc_t:s0 tclass=file permissive=1
May 01 09:53:28 localhost syncthing[2878]: [LIPEH] INFO: Joined relay relay://79.136.5.160:22067
May 01 09:53:29 localhost kernel: usb 1-4.4.1.2: new full-speed USB device number 16 using xhci_hcd
May 01 09:53:29 localhost chronyd[1839]: Forward time jump detected!
May 01 09:53:29 localhost chronyd[1839]: Can't synchronise: no selectable sources
May 01 09:53:29 localhost kernel: usb 1-4.4.1.2: New USB device found, idVendor=4c4a, idProduct=4155, bcdDevice= 1.00
May 01 09:53:29 localhost kernel: usb 1-4.4.1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
May 01 09:53:29 localhost kernel: usb 1-4.4.1.2: Product: UACDemoV1.0
May 01 09:53:29 localhost kernel: usb 1-4.4.1.2: Manufacturer: Jieli Technology
May 01 09:53:29 localhost kernel: usb 1-4.4.1.2: SerialNumber: 1120021E02070907
May 01 09:53:29 localhost kernel: usb-storage 1-4.4.1.2:1.0: USB Mass Storage device detected
May 01 09:53:29 localhost kernel: scsi host4: usb-storage 1-4.4.1.2:1.0
May 01 09:53:29 localhost kernel: input: Jieli Technology UACDemoV1.0 as /devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4.4/1-4.4.1/1-4.4.1.2/1-4.4.1.2:1.4/0003:4C4A:4155.000A/input/input36
May 01 09:53:30 localhost kernel: Bluetooth: hci0: Waiting for firmware download to complete
May 01 09:53:30 localhost kernel: hid-generic 0003:4C4A:4155.000A: input,hidraw6: USB HID v1.00 Device [Jieli Technology UACDemoV1.0] on usb-0000:00:14.0-4.4.1.2/input4
May 01 09:53:30 localhost kernel: Bluetooth: hci0: Firmware loaded in 4601702 usecs
May 01 09:53:30 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4.4/1-4.4.1
May 01 09:53:30 localhost upowerd[1813]: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4.4/1-4.4.1
May 01 09:53:30 localhost systemd-homed[1809]: block device /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4.4/1-4.4.1/1-4.4.1.2/1-4.4.1.2:1.0/host4/target4:0:0/4:0:0:0/block/sdb has been removed.
May 01 09:53:30 localhost kernel: Bluetooth: hci0: Waiting for device to boot
May 01 09:53:30 localhost kernel: Bluetooth: hci0: Device booted in 93035 usecs
May 01 09:53:30 localhost kernel: Bluetooth: hci0: Found Intel DDC parameters: intel/ibt-11-5.ddc
May 01 09:53:30 localhost mtp-probe[124499]: checking bus 1, device 16: "/sys/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4.4/1-4.4.1/1-4.4.1.2"
May 01 09:53:30 localhost mtp-probe[124499]: bus: 1, device: 16 was not an MTP device
May 01 09:53:30 localhost kernel: Bluetooth: hci0: Applying Intel DDC parameters completed
May 01 09:53:30 localhost kernel: Bluetooth: hci0: Firmware revision 0.0 build 14 week 44 2021
May 01 09:53:30 localhost kernel: scsi 4:0:0:0: Direct-Access     BR25     UDISK            1.00 PQ: 0 ANSI: 2
May 01 09:53:31 localhost kernel: sd 4:0:0:0: Attached scsi generic sg1 type 0
May 01 09:53:31 localhost kernel: sd 4:0:0:0: [sdb] Media removed, stopped polling
May 01 09:53:31 localhost kernel: sd 4:0:0:0: [sdb] Attached SCSI removable disk
May 01 09:53:31 localhost systemd[2834]: Reached target Sound Card.
May 01 09:53:32 localhost kernel: ------------[ cut here ]------------
May 01 09:53:32 localhost kernel: WARNING: CPU: 3 PID: 124305 at /tmp/akmodsbuild.YAPyniWJ/BUILD/nvidia-kmod-510.60.02-x86_64/_kmod_build_5.17.4-200.fc35.x86_64/nvidia/nv.c:3935 nv_restore_user_channels+0xc9/0xe0 [nvidia]
May 01 09:53:32 localhost kernel: Modules linked in: snd_seq_dummy snd_hrtimer overlay xt_CHECKSUM xt_MASQUERADE xt_conntrack ipt_REJECT acpi_call(OE) bridge stp llc wireguard curve25519_x86_64 libcurve25519_generic ip6_udp_tunnel udp_tunnel nf_nat_tftp nf_conntrack_tftp nft_objref nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nf_log_syslog vboxnetadp(OE) vboxnetflt(OE) nft_log vboxdrv(OE) nft_ct nft_chain_nat ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_raw iptable_security ip_set nf_tables nfnetlink ip6table_filter ip6_tables iptable_filter qrtr sunrpc binfmt_misc vfat fat nvidia_drm(POE) nvidia_modeset(POE) uas usb_storage btusb btrtl btbcm btintel btmtk bluetooth uvcvideo videobuf2_vmalloc nvidia(POE) videobuf2_memops videobuf2_v4l2 videobuf2_common ecdh_generic rmi_smbus rmi_core snd_usb_audio
May 01 09:53:34 localhost kernel:  snd_usbmidi_lib snd_rawmidi intel_rapl_msr intel_rapl_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp iwlmvm iTCO_wdt mei_hdcp intel_pmc_bxt kvm_intel mei_pxp mac80211 ee1004 mei_wdt iTCO_vendor_support kvm snd_ctl_led libarc4 irqbypass snd_hda_codec_realtek rapl snd_hda_codec_generic iwlwifi intel_cstate intel_uncore snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi iwlmei snd_hda_codec snd_hda_core cfg80211 snd_hwdep snd_seq think_lmi intel_wmi_thunderbolt joydev thinkpad_acpi pcspkr i2c_i801 firmware_attributes_class snd_seq_device mei_me wmi_bmof ledtrig_audio i2c_smbus snd_pcm mei platform_profile intel_pch_thermal rfkill snd_timer drm_kms_helper snd cec soundcore tcp_bbr v4l2loopback(OE) videodev drm mc ip_tables dm_crypt r8152 mii hid_logitech_hidpp hid_logitech_dj rtsx_pci_sdmmc mmc_core crct10dif_pclmul crc32_pclmul crc32c_intel nvme e1000e nvme_core rtsx_pci ghash_clmulni_intel serio_raw wmi video ipmi_devintf ipmi_msghandler fuse
May 01 09:53:34 localhost kernel: CPU: 3 PID: 124305 Comm: nvidia-sleep.sh Tainted: P           OE     5.17.4-200.fc35.x86_64 #1
May 01 09:53:34 localhost kernel: Hardware name: LENOVO 20EN001EUS/20EN001EUS, BIOS N1EET94W (1.67 ) 12/10/2021
May 01 09:53:34 localhost kernel: RIP: 0010:nv_restore_user_channels+0xc9/0xe0 [nvidia]
May 01 09:53:34 localhost kernel: Code: a1 1e c8 be 01 00 00 00 4c 89 e7 e8 81 9f 00 00 4c 89 ff e8 b9 a0 1e c8 ba 02 00 00 00 4c 89 e6 48 89 ef e8 09 33 94 00 eb 94 <0f> 0b eb c6 41 bd 51 00 00 00 eb 9f 66 66 2e 0f 1f 84 00 00 00 00
May 01 09:53:34 localhost kernel: RSP: 0018:ffffa9d80383fe28 EFLAGS: 00010206
May 01 09:53:35 localhost kernel: RAX: 0000000000000003 RBX: 0000000000000002 RCX: 0000000000000000
May 01 09:53:35 localhost kernel: RDX: 0000000000000087 RSI: 0000000000000246 RDI: 00000000ffffffff
May 01 09:53:35 localhost kernel: RBP: ffff8aa255918000 R08: 0000000000000001 R09: 00000000802a0024
May 01 09:53:35 localhost kernel: R10: ffff8aa5037bd780 R11: 0000000000000000 R12: ffff8aa104f5d000
May 01 09:53:35 localhost kernel: hrtimer: interrupt took 4104579 ns
May 01 09:53:35 localhost kernel: R13: 0000000000000003 R14: ffff8aa104f5d510 R15: ffff8aa104f5d000
May 01 09:53:35 localhost kernel: FS:  00007f3f563e8740(0000) GS:ffff8ab063ec0000(0000) knlGS:0000000000000000
May 01 09:53:35 localhost kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
May 01 09:53:36 localhost kernel: CR2: 00007fb3d397a4c0 CR3: 000000061af44001 CR4: 00000000003706e0
May 01 09:53:36 localhost kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
May 01 09:53:36 localhost kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
May 01 09:53:36 localhost kernel: Call Trace:
May 01 09:53:36 localhost kernel:  <TASK>
May 01 09:53:36 localhost kernel:  nv_set_system_power_state+0x226/0x3d0 [nvidia]
May 01 09:53:36 localhost kernel:  nv_procfs_write_suspend+0xeb/0x140 [nvidia]
May 01 09:53:36 localhost kernel:  proc_reg_write+0x52/0xa0
May 01 09:53:37 localhost kernel:  ? preempt_count_add+0x49/0xa0
May 01 09:53:37 localhost kernel:  vfs_write+0xb9/0x2a0
May 01 09:53:37 localhost kernel:  ksys_write+0x4f/0xc0
May 01 09:53:37 localhost kernel:  do_syscall_64+0x38/0x90
May 01 09:53:37 localhost kernel:  entry_SYSCALL_64_after_hwframe+0x44/0xae
May 01 09:53:37 localhost kernel: RIP: 0033:0x7f3f5652f147
May 01 09:53:37 localhost kernel: Code: 0b 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24
May 01 09:53:37 localhost kernel: RSP: 002b:00007ffdb96ec798 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
May 01 09:53:37 localhost kernel: RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007f3f5652f147
May 01 09:53:37 localhost kernel: RDX: 0000000000000007 RSI: 0000555f7f021c90 RDI: 0000000000000001
May 01 09:53:37 localhost kernel: RBP: 0000555f7f021c90 R08: 0000000000000003 R09: 0000000000000077
May 01 09:53:38 localhost kernel: R10: 000000000000005d R11: 0000000000000246 R12: 0000000000000007
May 01 09:53:38 localhost kernel: R13: 00007f3f565e65a0 R14: 0000000000000007 R15: 00007f3f565e67a0
May 01 09:53:38 localhost kernel:  </TASK>
May 01 09:53:38 localhost kernel: ---[ end trace 0000000000000000 ]---
May 01 09:53:38 localhost kernel: ------------[ cut here ]------------
May 01 09:53:38 localhost kernel: WARNING: CPU: 1 PID: 124305 at /tmp/akmodsbuild.YAPyniWJ/BUILD/nvidia-kmod-510.60.02-x86_64/_kmod_build_5.17.4-200.fc35.x86_64/nvidia/nv.c:4152 nv_set_system_power_state+0x2c8/0x3d0 [nvidia]
May 01 09:53:39 localhost kernel: Modules linked in: snd_seq_dummy snd_hrtimer overlay xt_CHECKSUM xt_MASQUERADE xt_conntrack ipt_REJECT acpi_call(OE) bridge stp llc wireguard curve25519_x86_64 libcurve25519_generic ip6_udp_tunnel udp_tunnel nf_nat_tftp nf_conntrack_tftp nft_objref nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nf_log_syslog vboxnetadp(OE) vboxnetflt(OE) nft_log vboxdrv(OE) nft_ct nft_chain_nat ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_raw iptable_security ip_set nf_tables nfnetlink ip6table_filter ip6_tables iptable_filter qrtr sunrpc binfmt_misc vfat fat nvidia_drm(POE) nvidia_modeset(POE) uas usb_storage btusb btrtl btbcm btintel btmtk bluetooth uvcvideo videobuf2_vmalloc nvidia(POE) videobuf2_memops videobuf2_v4l2 videobuf2_common ecdh_generic rmi_smbus rmi_core snd_usb_audio
May 01 09:53:39 localhost kernel:  snd_usbmidi_lib snd_rawmidi intel_rapl_msr intel_rapl_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp iwlmvm iTCO_wdt mei_hdcp intel_pmc_bxt kvm_intel mei_pxp mac80211 ee1004 mei_wdt iTCO_vendor_support kvm snd_ctl_led libarc4 irqbypass snd_hda_codec_realtek rapl snd_hda_codec_generic iwlwifi intel_cstate intel_uncore snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi iwlmei snd_hda_codec snd_hda_core cfg80211 snd_hwdep snd_seq think_lmi intel_wmi_thunderbolt joydev thinkpad_acpi pcspkr i2c_i801 firmware_attributes_class snd_seq_device mei_me wmi_bmof ledtrig_audio i2c_smbus snd_pcm mei platform_profile intel_pch_thermal rfkill snd_timer drm_kms_helper snd cec soundcore tcp_bbr v4l2loopback(OE) videodev drm mc ip_tables dm_crypt r8152 mii hid_logitech_hidpp hid_logitech_dj rtsx_pci_sdmmc mmc_core crct10dif_pclmul crc32_pclmul crc32c_intel nvme e1000e nvme_core rtsx_pci ghash_clmulni_intel serio_raw wmi video ipmi_devintf ipmi_msghandler fuse
May 01 09:53:40 localhost kernel: CPU: 1 PID: 124305 Comm: nvidia-sleep.sh Tainted: P        W  OE     5.17.4-200.fc35.x86_64 #1
May 01 09:53:40 localhost kernel: Hardware name: LENOVO 20EN001EUS/20EN001EUS, BIOS N1EET94W (1.67 ) 12/10/2021
May 01 09:53:40 localhost kernel: RIP: 0010:nv_set_system_power_state+0x2c8/0x3d0 [nvidia]
May 01 09:53:41 localhost kernel: Code: ff ff ff 41 83 fc 02 74 ea 48 8b 85 70 02 00 00 ba 02 00 00 00 48 8b 70 78 48 8b 78 60 e8 90 d3 ff ff 85 c0 74 cd 0f 0b eb c9 <0f> 0b e9 5f ff ff ff 48 c7 c7 90 47 2d c3 e8 d5 71 1e c8 e8 90 23
May 01 09:53:41 localhost kernel: RSP: 0018:ffffa9d80383fe58 EFLAGS: 00010206
May 01 09:53:41 localhost kernel: RAX: 0000000000000003 RBX: 0000000000000002 RCX: 0000000080020001
May 01 09:53:41 localhost kernel: RDX: 0000000080020002 RSI: fffff90a49564600 RDI: 0000000040000000
May 01 09:53:41 localhost kernel: RBP: ffff8aa104f5d000 R08: ffff8aa255918000 R09: 0000000080020001
May 01 09:53:41 localhost kernel: R10: ffff8aa255918000 R11: 0000000000000001 R12: 0000000000000000
May 01 09:53:41 localhost kernel: R13: 0000555f7f021c90 R14: ffffa9d80383ff10 R15: 0000000000000007
May 01 09:53:42 localhost kernel: FS:  00007f3f563e8740(0000) GS:ffff8ab063e40000(0000) knlGS:0000000000000000
May 01 09:53:42 localhost kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
May 01 09:53:42 localhost kernel: CR2: 00007fbee0bec000 CR3: 000000061af44005 CR4: 00000000003706e0
May 01 09:53:42 localhost kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
May 01 09:53:42 localhost kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
May 01 09:53:42 localhost kernel: Call Trace:
May 01 09:53:42 localhost kernel:  <TASK>
May 01 09:53:43 localhost kernel:  nv_procfs_write_suspend+0xeb/0x140 [nvidia]
May 01 09:53:43 localhost kernel:  proc_reg_write+0x52/0xa0
May 01 09:53:43 localhost kernel:  ? preempt_count_add+0x49/0xa0
May 01 09:53:43 localhost kernel:  vfs_write+0xb9/0x2a0
May 01 09:53:43 localhost kernel:  ksys_write+0x4f/0xc0
May 01 09:53:43 localhost kernel:  do_syscall_64+0x38/0x90
May 01 09:53:43 localhost kernel:  entry_SYSCALL_64_after_hwframe+0x44/0xae
May 01 09:53:43 localhost kernel: RIP: 0033:0x7f3f5652f147
May 01 09:53:43 localhost kernel: Code: 0b 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24
May 01 09:53:43 localhost kernel: RSP: 002b:00007ffdb96ec798 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
May 01 09:53:44 localhost kernel: RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007f3f5652f147
May 01 09:53:44 localhost kernel: RDX: 0000000000000007 RSI: 0000555f7f021c90 RDI: 0000000000000001
May 01 09:53:44 localhost kernel: RBP: 0000555f7f021c90 R08: 0000000000000003 R09: 0000000000000077
May 01 09:53:44 localhost kernel: R10: 000000000000005d R11: 0000000000000246 R12: 0000000000000007
May 01 09:53:44 localhost kernel: R13: 00007f3f565e65a0 R14: 0000000000000007 R15: 00007f3f565e67a0
May 01 09:53:44 localhost kernel:  </TASK>
May 01 09:53:44 localhost kernel: ---[ end trace 0000000000000000 ]---
May 01 09:53:44 localhost mtp-probe[124532]: checking bus 1, device 16: "/sys/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4.4/1-4.4.1/1-4.4.1.2"
May 01 09:53:44 localhost mtp-probe[124532]: bus: 1, device: 16 was not an MTP device
May 01 09:53:47 localhost kernel: nvidia-modeset: WARNING: GPU:0: Lost display notification (0:0x00000000); continuing.
May 01 09:53:50 localhost kernel: nvidia-modeset: ERROR: GPU:0: Idling display engine timed out: 0x0000947d:0:0:407
May 01 09:54:09 localhost kernel: usb 1-4.4.1.2: reset full-speed USB device number 16 using xhci_hcd
May 01 09:54:23 localhost syncthing[2878]: [LIPEH] INFO: Connection to GXIHOP7-I4OIRTE-PGJOV36-ZEOAISB-7YGKJ5D-OGXGCX5-KGU4SLO-2625QQR at 10.10.10.1:22000-192.115.210.42:49570/tcp-server/TLS1.3-TLS_AES_128_GCM_SHA256 closed: read timeout

# uname -r 
Kernel: 5.17.4-200.fc35.x86_64

# cat /proc/cmdline
BOOT_IMAGE=(hd2,gpt2)/vmlinuz-5.17.4-200.fc35.x86_64 root=UUID=30d9ae96-0366-494a-aaae-ddeaadf87d06 ro rootflags=subvol=root noibrs noibpb nopti nospectre_v2 nospectre_v1 l1tf=off nospec_store_bypass_disable no_stf_barrier mds=off tsx=on tsx_async_abort=off mitigations=off rd.driver.blacklist=nouveau modprobe.blacklist=nouveau
# rpm -qa | ag nvidia
nvidia-kmod-common-510.60.02-1.fc35.noarch
akmod-nvidia-510.60.02-1.fc35.x86_64
nvidia-libXNVCtrl-510.60.02-1.fc35.x86_64
nvidia-driver-libs-510.60.02-1.fc35.x86_64
nvidia-driver-510.60.02-1.fc35.x86_64
nvidia-settings-510.60.02-1.fc35.x86_64
kmod-nvidia-5.17.4-200.fc35.x86_64-510.60.02-1.fc35.x86_64

If anyone can help with this matter, thx in advance

UPDATE #1

In the logs I see that the service nvidia-suspend.service is called and the script is executed,

ExecStart=/usr/bin/logger -t suspend -s "nvidia-suspend.service"
ExecStart=/usr/bin/nvidia-sleep.sh "suspend"
#j ournalctl -b -1 | tail -n1000 | grep -i suspend

May 01 20:10:18 localhost ModemManager[2096]: <info>  [sleep-monitor] system is about to suspend
May 01 20:10:18 localhost systemd[1]: Starting NVIDIA system suspend actions...
May 01 20:10:18 localhost suspend[15829]: nvidia-suspend.service
May 01 20:10:18 localhost logger[15829]: <13>May  1 20:10:18 suspend: nvidia-suspend.service
May 01 20:10:19 localhost systemd[1]: nvidia-suspend.service: Deactivated successfully.
May 01 20:10:19 localhost systemd[1]: Finished NVIDIA system suspend actions.
May 01 20:10:19 localhost audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=nvidia-suspend comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
May 01 20:10:19 localhost audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=nvidia-suspend comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
May 01 20:10:19 localhost systemd[1]: Starting System Suspend...
May 01 20:10:19 localhost systemd-sleep[15858]: Entering sleep state 'suspend'...
May 01 20:10:19 localhost kernel: PM: suspend entry (deep)

But on resume, I don’t see the resume (nvidia-resume.service) service called

# journalctl -b -1 | grep -i resume
<nothing>

UPDATE #2

When putting a watch on the files under /usr/lib/systemd/system-sleep I do see them called.

UPDATE #3

My machine spec,

System:
  Kernel: 5.17.4-200.fc35.x86_64 arch: x86_64 bits: 64 compiler: gcc
    v: 2.37-10.fc35 Console: pty pts/2 Distro: Fedora release 35 (Thirty Five)
Machine:
  Type: Laptop System: LENOVO product: 20EN001EUS v: ThinkPad P50
    serial: <filter>
  Mobo: LENOVO model: 20EN001EUS serial: <filter> UEFI: LENOVO
    v: N1EET94W (1.67 ) date: 12/10/2021
CPU:
  Info: quad core model: Intel Core i7-6820HQ bits: 64 type: MT MCP
    arch: Skylake-S rev: 3 cache: L1: 256 KiB L2: 1024 KiB L3: 8 MiB
  Speed (MHz): avg: 2638 high: 2999 min/max: 800/3600 cores: 1: 2999
    2: 2496 3: 2424 4: 2615 5: 2540 6: 2901 7: 2752 8: 2377 bogomips: 43198
  Flags: avx avx2 ht lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx
Graphics:
  Device-1: NVIDIA GM107GLM [Quadro M2000M] vendor: Lenovo driver: nvidia
    v: 510.60.02 bus-ID: 01:00.0
  Device-2: Chicony Integrated Camera type: USB driver: uvcvideo
    bus-ID: 1-8:2
  Display: server: X.Org v: 1.20.14 driver: X: loaded: nvidia
    gpu: nvidia,nvidia-nvswitch resolution: 1: 3840x2160~60Hz 2: 3440x1440~60Hz
  OpenGL: renderer: Quadro M2000M/PCIe/SSE2 v: 4.6.0 NVIDIA 510.60.02
    direct render: Yes
Swap:
  Alert: No swap data was found.

My laptop has some issues with this. Mostly a short power key push helps.

@jpbn what do you mean? did you put the device to sleep again and resume?

@rabiny Close the lid for sleep and open it after some time. The power button could help you.

@jpbn Tried it, nothing happened, the laptop won’t go to sleep again, and I’m forced to restart the machine.

I can see you are in a 5.17.4 kernel where the problem still present can you try with a older kernel 5.16.18 or older if you have try boot into that with grub and see if the problem still persists.
And it is always recommended to give cpu and os version with your bug report.
As you are in fedora35
I think what you are having is a on going kernel bug here is a reference
https://bugzilla.redhat.com/show_bug.cgi?id=2079706
You can give your output also with system info.

Hi @frankjunior , I just updated the top post with the system information, in my case the problem is the same as in the Bugzilla you mention, only that my CPU is Intel.

Should I add my details to the but report as well?

PS.
Downgraded to kernel-5.16.9-200.fc35 and had the same problem.
Same with kernel-5.15.18-200.fc35:face_exhaling:

1 Like

Please check if there is a new version … and did you have a look on the Lenovo’s page about this problem (search with your serial number)?

About the bios:

I keep my laptop up2date, including running fwupd frequently. And I just double-check to make sure, I’m running the latest firmware.

Actually, this model is quite old now and should have very mature support. I can’t remember having any problem with suspend/resume until this last year with Nvidia driver v470 splitting the power-management script into a separate package, which was solved easily, and now this time, where I really don’t know what cussing it.

I upgraded to Fedora 36, and I still have this problem.

Also, I have this problem with a Lenovo P1 Gen3 laptop, fresh installed && updated with KDE.

How do you suspend? Just closing the lid or did you check with the suspend option on the shutdown window?
Would like to know if this makes a difference.

Did you give a try with the live ISO of F36?

How about the nouveau drivers do you have the same issue?

How do you suspend? Just closing the lid or did you check with the suspend option on the shutdown window?

I’m running systemdctl suspend by a shortcut.

Would like to know if this makes a difference.

The lid trigger is disabled as I use a dock station, and I don’t want my laptop to auto suspend when I disconnect my laptop from the dock.

Did you give a try with the live ISO of F36?
How about the nouveau drivers do you have the same issue?

Not yet. But I’ll give it a try.

Which brand and Model?
How do you connect the ThinkPad to it?
What is on the docking station?
One or two monitors? How do you connect it/them?

Which brand and Model?

The model is 04W3955

How do you connect the ThinkPad to it?

And it is connected via the bottom, via Lenovo proprietary connection.

What is on the docking station?
One or two monitors? How do you connect it/them?

2 Monitors, one is a 4K display, and another is a regular display connected vertically.
both connected via DisplayPort

And there are other peripherals, like an external mic and headphones, connected via USB.

Are you up to date with the firmware driver?

Are you up to date with the firmware driver?

Yes.

This setup worked for me like this for years, and just recently broke for some reason. The first one was when NVIDIA released their v470 of their driver, which split the power control into a separate package, and was not installed as part of the upgrade.

And that is the 2nd time, which I can’t pinpoint the reason.

Update

TL;DR

The issue was resolved, but I have no idea how!

Details

I tried running Fedora Live CD to test if I will have the same issue, so I downloaded the KDE live CD, but this image failed to boot correctly for me. For some reason, the screen will load, and I get a shattered display.

I gave up on the LiveCD and tried to boot my machine, only to get a blank screen. Seems like there was another kernel update, which trigger another driver compilation which I don’t know what the result was.

So I booted into a single-mode and removed all NVIDIA drivers, cleared manually all leftovers in /etc/default/grub which referred to nouveau and modeset.

Rebooted the machine again, entered multi-user target, set up networking, and installed Nvidia drivers again. And left the postscript to finish the compilation of the module, and rebooted.

The system came up normally, and I was able to suspend and resume back the session.

So I don’t know what happened :thinking: