Looking through what I have just written, it may be pretty clear what the issue is…
➜ ~ sudo dnf install kernel
[sudo] password for nflx:
Last metadata expiration check: 1:03:00 ago on Sun 11 Sep 2022 05:46:15 PM CDT.
Package kernel-5.17.5-300.fc36.x86_64 is already installed.
Package kernel-5.19.7-200.fc36.x86_64 is already installed.
Package kernel-5.19.8-200.fc36.x86_64 is already installed.
Dependencies resolved.
Nothing to do.
Complete!
I’m going to try upgrading my system kernel to match the akmods target version. (Not sure how things got so mismatched in the first place, either.)
I had a special case with installing the bootloader because of a prior installation of Arch Linux, as well as a dual boot of Windows 11. I needed to manually uninstall a separate bootloader, rEFInd, and reinstall GRUB 2 as a result of having a very different EFI setup. As a result, I haven’t configured my EFI to automatically mount to my OS’s separate local version of /boot/efi, so the vmlinuz isn’t actually updated on my EFI partition…
I think auto mounting my EFI should fix things, so I’ll look into fixing the issue and report back.
You definitely have the proper kernel module for the latest kernel built as is shown by the kmod-nvidia package.
You should be able to do the following to verify things are correct.
Using rEFInd may cause an issue since fedora expects to have the uefi partition mounted at /boot/efi, and an ext4 partition mounted at /boot.
Using ls /boot should show several kernel files such as this
If those 3 details are not correct then fedora updates are not properly being done and that would explain why you are booting with the default kernel (5.17.5) from the initial install of fedora instead of the current kernel which seems to be installed as you showed with the output of the kmod-nvidia package.
What I would guess is wrong may be that rEFInd overwrote /boot/efi/EFI/fedora/grub.cfg while the fedora updates write to /boot/grub2/grub.cfg.
and it essentially involves deleting the corrupted grub.cfg files (at both locations) then doing dnf reinstall grub2-efi* grub2-common which will recover for you as long as the file systems are mounted properly.
One important detail I left out (before I realized this was a bootloader issue) is that Anaconda was having issues with installing the bootloader, and same with the Ubuntu Boot Repair ISO. I actually ended up removing rEFInd altogether from my EFI so it would fall back to GRUB 2, which I was able to reinstall manually from the live USB.
Note that the Ubuntu Boot Repair process seemed to just idle, so I don’t think it even touched the EFI at all (I’m pretty certain that it wasn’t able to locate it for some reason) - I might be wrong, though, because its boot repair doesn’t give you log information (at least, by default)…
Before I explain more, let me show you what my EFI looks like (scroll for full tree):
A lot of these files are remnants from my Arch installation.
Most of the installation here was manual. I ended up removing EFI/rEFInd/and adding EFI/fedora/ on my own. This only pertains to the EFI partition on my hard drive, and NOT the one that came with the ISO.
I changed the vmlinuz-linux and initramfs files to the updated kernel, but it didn’t affect anything, so I’m assuming Fedora doesn’t use these.
Note that this uses grub/ instead of grub2/.
I looked for Fedora’s GRUB config file that should be in the local/boot directory, but it doesn’t exist for some reason. I think that’s the main culprit. I tried reinstalling GRUB 2 as instructed from the wiki (removing the extra config files that I could find in /boot/grub2) but the grub-common script wasn’t called for some reason. (I ended up moving the corrupted .cfg file into a backup directory just to be safe).
So the likely result of this file being missing is that there’s nothing mounted to /boot like there should be. (The mount shown is one I mounted manually):
[root@fedora nflx]# mount | grep boot
/dev/nvme0n1p1 on /mnt/boot/efi type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro)
There’s something blocking grub-common from running its config generation, but I don’t know why. What should I do to regenerate my config files?
Please post the output of lsblk. It seems you do not have the proper content in /boot nor /boot/efi for a proper fedora boot.
This is certainly a boot loader issue, and I will come back and edit this with my suggestions after looking up the solution I found elsewhere.
EDIT:
The solution is posted here and can be summarized with
Boot to the live install media then do the following.
su
identify the installed media root file system then mount <your root file system> /mnt
If this is ext4 then you only need the file system path name.
If this is btrfs then you will need to format it as UUID=XXXXXX-XXXX-XXXX-XXXXXXXX -t btrfs -o subvol=root,compress=zstd:1 /mnt where the UUID is that of your btrfs root filesystem volume. That UUID can be found with one of the options of lsblk like lsblk -o PATH,FSTYPE,UUID and looking at the fstype and path to confirm it is the installed file system volume.
for part in sys proc run dev ; do mount /$part /mnt/$part ; done
chroot /mnt
mount -a
this should finish mounting the installed system as it normally is when running.
Now you can begin the recovery as if booted from the installed system.
sudo rm -r /boot/efi/EFI/fedora/* /boot/grub2/grub.cfg
This will remove all interfering files and allow the next steps to run properly.
cat /sys/firmware/efi
If this returns cat: /sys/firmware/efi: Is a directory then you have booted from uefi and the remainder should work properly. If not then you have legacy booted and the recovery is different.
dnf reinstall shim* grub2-efi* grub2-common
This will reinstall the named packages and properly build the content of /boot/grub2, /boot/loader, and /boot/efi/EFI/fedora.
dnf reinstall kernel*
This should reinstall the latest kernel and rebuild all the modules for it. If this errors then use dnf upgrade --refresh The latest kernel for fc36 is currently 5.19.8.
At this point you should be ready for a proper reboot, so do that and let us know the result.
Although this looked hopeful, unfortunately it didn’t work:
➜ ~ uname -r
5.17.5-300.fc36.x86_64
➜ ~ modinfo -F version nvidia
modinfo: ERROR: Module nvidia not found.
The installation steps worked and everything, though.
I’m not certain, but my theory is that Fedora isn’t using the system EFI partition at all, and it installed GRUB 2 over its own locally provided version on my ext4 partition. So my machine isn’t even working with it, because /boot isn’t mounted from /dev/nvme0n1p1. Hopefully this makes sense.
I’m just learning. This could be wrong for all I know! But I’m pretty sure this is the case, because I seem to recall /boot being auto mounted when I was using Arch Linux.
Here’s my lsblk:
➜ ~ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 1 118G 0 disk # Live USB
└─sda1 8:1 1 118G 0 part
zram0 252:0 0 8G 0 disk [SWAP]
nvme0n1 259:0 0 953.9G 0 disk
├─nvme0n1p1 259:1 0 260M 0 part # EFI
├─nvme0n1p2 259:2 0 16M 0 part # Microsoft Reserved
├─nvme0n1p3 259:3 0 441G 0 part # Windows 11
└─nvme0n1p4 259:4 0 512.6G 0 part / # Fedora Linux
Also, my bootloader file structures look radically different…
➜ ~ ls /mnt/boot/efi # /dev/nvme0n1p1 - manually mounted EFI partition
'$RECYCLE.BIN' BOOT EFI grub initramfs-linux-fallback.img initramfs-linux.img mach_kernel 'System Volume Information' vmlinuz-linux
➜ ~ ls /boot/ # Local unmounted version of /boot
config-5.17.5-300.fc36.x86_64 grub2 loader System.map-5.19.7-200.fc36.x86_64 vmlinuz-5.19.8-200.fc36.x86_64
config-5.19.7-200.fc36.x86_64 initramfs-0-rescue-83c455977aa84dc19afd5e653811b190.img symvers-5.17.5-300.fc36.x86_64.gz System.map-5.19.8-200.fc36.x86_64
config-5.19.8-200.fc36.x86_64 initramfs-5.17.5-300.fc36.x86_64.img symvers-5.19.7-200.fc36.x86_64.gz vmlinuz-0-rescue-83c455977aa84dc19afd5e653811b190
efi initramfs-5.19.7-200.fc36.x86_64.img symvers-5.19.8-200.fc36.x86_64.gz vmlinuz-5.17.5-300.fc36.x86_64
extlinux initramfs-5.19.8-200.fc36.x86_64.img System.map-5.17.5-300.fc36.x86_64 vmlinuz-5.19.7-200.fc36.x86_64
➜ ~ sudo ls /boot/efi
EFI mach_kernel System
I think that’s why a lot of the maintenance I’ve been doing off my EFI partition /dev/nvme0n1p1 has been manual; for some reason, it’s just never been able to mount over /boot on my system.
I’m able to boot into my main system and everything, it’s just weird that it’s not able to auto mount. That’s why I suggested some way to fix it with efibootmgr.
This has been a wild ride so far though, haha. :)
Arch Linux does things much differently, I suppose!
Ok, it seems you may have the system installed with / as an ext4 filesystem, in which case it would not automatically create a separate /boot partition, but we need to confirm that.
When booted to the normal file system please give us the output of lsblk -o PATH,FSTYPE,UUID,MOUNTPOINT , ls /boot, ls /boot/grub2, ls /boot/loader/entries and ls /boot/efi/EFI/fedora. Also the output of cat /sys/firmware/efi
If doing that from the live boot media with the / partition mounted at /mnt then add /mnt to the front of each of those commands.
It seems that your output of ls /boot showed that as unmounted and does not contain either the vmlinuz images nor the System-map files.
The properly filled and used for booting version of /boot should contain files similar to this.
I just noted what you said.
Your efi partition should NOT be mounted over /boot, but rather mounted at /boot/efi.
Please give us the output of cat /etc/fstab as well.
Yep, this is correct - I do remember setting / as my root.
I posted everything below - you can scroll through the text below in both directions!
➜ ~ lsblk -o PATH,FSTYPE,UUID,MOUNTPOINT
PATH FSTYPE UUID MOUNTPOINT
/dev/zram0 [SWAP]
/dev/nvme0n1
/dev/nvme0n1p1 vfat XXXX-A5DF /mnt/boot/efi
/dev/nvme0n1p2
/dev/nvme0n1p3 ntfs XXXXXXXXXXBDCC9A
/dev/nvme0n1p4 ext4 xxxxxxxx-xxxx-xxxx-99bd-a745922edadf /
# /boot is the locally installed boot directory, not the mounted one.
# Scroll this text area to the right to see more files,
# including the /boot/vmlinuz-* files.
[root@fedora nflx]# ls /boot
config-5.17.5-300.fc36.x86_64 grub2 loader System.map-5.19.7-200.fc36.x86_64 vmlinuz-5.19.8-200.fc36.x86_64
config-5.19.7-200.fc36.x86_64 initramfs-0-rescue-83c455977aa84dc19afd5e653811b190.img symvers-5.17.5-300.fc36.x86_64.gz System.map-5.19.8-200.fc36.x86_64
config-5.19.8-200.fc36.x86_64 initramfs-5.17.5-300.fc36.x86_64.img symvers-5.19.7-200.fc36.x86_64.gz vmlinuz-0-rescue-83c455977aa84dc19afd5e653811b190
efi initramfs-5.19.7-200.fc36.x86_64.img symvers-5.19.8-200.fc36.x86_64.gz vmlinuz-5.17.5-300.fc36.x86_64
extlinux initramfs-5.19.8-200.fc36.x86_64.img System.map-5.17.5-300.fc36.x86_64 vmlinuz-5.19.7-200.fc36.x86_64
[root@fedora nflx]# ls /boot/grub2/
fonts grubenv
[root@fedora nflx]# ls /boot/loader/entries
83c455977aa84dc19afd5e653811b190-0-rescue.conf 83c455977aa84dc19afd5e653811b190-5.19.7-200.fc36.x86_64.conf
83c455977aa84dc19afd5e653811b190-5.17.5-300.fc36.x86_64.conf 83c455977aa84dc19afd5e653811b190-5.19.8-200.fc36.x86_64.conf
[root@fedora nflx]# ls /boot/efi/EFI/fedora
BOOTIA32.CSV BOOTX64.CSV gcdia32.efi gcdx64.efi grubia32.efi grubx64.efi mmia32.efi mmx64.efi shim.efi shimia32.efi shimx64.efi
[root@fedora nflx]# cat /sys/firmware/efi
cat: /sys/firmware/efi: Is a directory
[root@fedora nflx]# cat /etc/fstab
#
# /etc/fstab
# Created by anaconda on Thu Sep 8 09:40:40 2022
#
# Accessible filesystems, by reference, are maintained under '/dev/disk/'.
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.
#
# After editing this file, run 'systemctl daemon-reload' to update systemd
# units generated from this file.
#
UUID=xxxxxxxx-xxxx-xxxx-99bd-a745922edadf / ext4 defaults 1 1
Whoops, my bad. Thanks, that makes sense.
I happen to have a backup of my old /etc/fstab from Arch Linux:
# Static information about the filesystems.
# See fstab(5) for details.
# <file system> <dir> <type> <options> <dump> <pass>
# (This is my old Arch Linux partition, which is now overwritten by Fedora.)
# /dev/nvme0n1p4
UUID=xxxxxxxx-xxxx-xxxx-2cc909cd1e80 / ext4 rw,relatime 0 1
# This is my EFI partition.
# /dev/nvme0n1p1 LABEL=SYSTEM
UUID=xxxx-A5DF /boot vfat rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro 0 2
It’s now my presumption that I could just add the /boot partition to fstab to fix the issue, and then reinstall GRUB 2 again to see if that fixes the issue… I’ll need to look into it!
If you have a separate unmounted /boot partition it could be added similar to the line for / . That is not needed since the / filesystem is ext4 and the output of lsblk does not show an extra partition for /boot. In fact ls /boot does show all the needed files in /boot.
Once that is done and the system booted that way then the steps I showed above for reinstalling grub should be done, as well as sudo dnf reinstall kernel* and it should work properly. Only one other thing might be needed to tell the bios to boot grub properly and that is to run efibootmgr -c which will tell the bios that grub is the boot loader.
If it cannot be booted that way then the mount from the live media and the chroot to the mounted file system then follow those steps to reinstall grub and to reinstall the kernel.
Since the root file system is ext4 and all the needed files show in /boot it is not necessary to add the extra /boot partition but simply to mount the efi partition to /boot/efi and it should be good to go. (Again after doing the reinstalls noted above.) The reinstalls will cause fedora to rebuild all the needed grub boot loader files.
So far so good! After adding an entry in fstab, it auto-mounts now, which is great:
➜ ~ mount | grep /boot/efi
/dev/nvme0n1p1 on /boot/efi type vfat (rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=winnt,errors=remount-ro)
However, I don’t think the GRUB directory is being modified in the EFI upon reinstallation… It seems completely untouched when reinstalling the GRUB packages. If I remove that folder or the config there altogether, it’s not being regenerated.
[root@fedora efi]# cd grub/
[root@fedora grub]# date -r x86_64-efi/
Wed Aug 31 08:43:18 AM CDT 2022
[root@fedora grub]# date -r grubenv
Wed Aug 24 03:29:58 AM CDT 2022
The grub/ folder is the GRUB from my old install which the PC is booting from, meaning I need to somehow get GRUB2 to install side by side (ie. to even generate in the first place).
[root@fedora nflx]# efibootmgr --create --disk=/dev/sda --part=1 --label="fedora" --loader="EFI\fedora\grubx64.efi"
Could not prepare Boot variable: No such file or directory
I think GRUB 2 is misconfigured, but I don’t know exactly why.
Move forward with trying to debug GRUB 2 (go through the live disk, reinstall the GRUB configs and see where my boot partition is detected on its side), because for some reason I’m not able to detect Fedora off of my main partition. Not sure why… And then from there maybe I can get an idea of what’s going wrong.
Add Fedora’s EFI/fedora/grubx64.efi to my current GRUB 1 bootloader that exists. Maybe I could point it with grub-mkconfig, but I know this is not a good idea to try on Fedora…
I beg you to please back up and take things one step at a time.
You can easily recover what is needed by grub with posting the output of efibootmgr and let us see what it tells you. There may easily be something hanging on in efi that is confusing the boot, but efibootmgr can tell us that.
A full 100% recovery of grub is quite possible if needed with 3 simple steps now that you have the efi partition properly mounted at /boot/efi.
Wait at least 5 minutes after the reinstall completes then reboot.
Fedora grub should be able to properly boot as long as the efi boot picks the proper loader and that can be seen from the grub boot menu. Getting the efi to pick the proper boot device is the reason for running efibootmgr so you can see and select which device to boot from.
Yes! No worries, I’m just as concerned as you are - I’ve backed up my EFI files ahead of time, and I’m trying my best to look into things as we go. I’m just a little lost as to why this isn’t working, haha
Meant to post this, whoops. I did look through this and couldn’t find anything unusual here. Note that I also removed my old/unused arch and rEFInd boots prior to this.
➜ ~ efibootmgr
BootCurrent: 0001
Timeout: 0 seconds
BootOrder: 0001,0000,0019,001A,001B,001C,001D,001E,001F,0020,0021,0022,0023,0024
Boot0000* Windows Boot Manager
Boot0001* GRUB
Boot0010 Setup
Boot0011 Boot Menu
Boot0012 Diagnostic Splash Screen
Boot0013 Lenovo Diagnostics
Boot0014 Regulatory Information
Boot0015 ThinkShield secure wipe
Boot0016 Startup Interrupt Menu
Boot0017 Rescue and Recovery
Boot0018 MEBx Hot Key
Boot0019* USB CD
Boot001A* USB FDD
Boot001B* NVMe0
Boot001C* NVMe1
Boot001D* ATA HDD0
Boot001E* ATA HDD1
Boot001F* USB HDD
Boot0020* PXE BOOT
Boot0021* HTTPS BOOT
Boot0022* LENOVO CLOUD
Boot0023 Other CD
Boot0024 Other HDD
Boot0025* IDER BOOT CDROM
Boot0026* IDER BOOT Floppy
Boot0027* ATA HDD
Boot0028* ATAPI CD
I just reran the installation and saw this here:
Running scriptlet: grub2-common-1:2.06-53.fc36.noarch 14/14
Generating grub configuration file ...
Found Windows Boot Manager on /dev/nvme0n1p1@/EFI/Microsoft/Boot/bootmgfw.efi
Adding boot menu entry for UEFI Firmware Settings ...
done
After waiting a couple minutes, my efibootmgr output still looks unchanged, unfortunately, with the same fallback: