Hello Fedora community,
I am coming here from the Qubes OS forum, seeking some guidance (or at least some answers). My original post, and full description of the situation, is there. I will attempt to summarize here with everything important from there. I do not know if there is anything anyone can do here, since I suspect that this is either a bug in the Anaconda installer or otherwise a limitation of it, but I hope I am wrong.
I want to install Qubes OS R4.0.3. Qubes OS uses the Anaconda installer and its base operating system is Fedora (atop Xen). R4.0.3 uses Fedora 25 and I am not aware of any feasible way to rebase it. I want to install it in LVM on LUKS atop raw data disk sda
. By that I mean I want no partitioning, partition table, or filesystem data on the raw sda
device, ie no VBR/MBR/GPT table or partitioning information exposed in the clear, so that the entire disk appears as random data. /boot
and the LUKS headers will be stored on sdb
, a separate USB boot drive.
After manually encrypting, partitioning, and formatting everything from tty2
in the live OS, I proceed to assign mountpoints in “Manual Partitioning”, but sda
does not show up under the list of existing partitions and neither do any of the LVM volumes. This is even after decrypting the LUKS container and activating the volume group from tty2
. Rescanning disks does not help and in fact causes the volume group to be deactivated, as confirmed by lsblk
in tty2
, with the following lines in /tmp/anaconda.log
and /tmp/lvm.log
:
# /tmp/anaconda.log
--
INFO program: Running [XX] lvm vgchange -an qubes_dom0 --config= devices { preferred_names=["^/dev/mapper/", "^/dev/md/", "^/dev/sd"] } log {level=7 file=/tmp/lvm.log syslog=0} ...
INFO program: stdout[XX]: 0 logical volume(s) in volume group "qubes_dom0" now active
INFO program: stderr[XX]: /usr/sbin/dmeventd: stat failed: No such file or directory
--
# /tmp/lvm.log
--
lvmcmdline.c:1611 DEGRADED MODE. Incomplete RAID LVs will be processed.
libdm-config.c:1064 activation/monitoring not found in config: defaulting to 1
lvmcmdline.c:1617 Processing: vgchange -an qubes_dom0 '--config= devices { preferred_names=["^/dev/mapper/", "^/dev/md/", "^/dev/sd"] } log {level=7 file=/tmp/lvm.log syslog=0}
lvmcmdline.c:1618 Command pid: 2730
lvmcmdline.c:1619 system ID:
lvmcmdline.c:1622 O_DIRECT will be used
libdm-config.c:992 global/locking_type not found in config: defaulting to 1
--
libdm-common.c:1475 qubes_dom0-pool00: Skipping NODE_DEL [trust_udev]
libdm-common.c:1475 qubes_dom0-root: Skipping NODE_DEL [trust_udev]
libdm-common.c:1475 qubes_dom0-pool00-tpool: Skipping NODE_DEL [trust_udev]
libdm-common.c:1475 qubes_dom0-pool00_tdata: Skipping NODE_DEL [trust_udev]
libdm-common.c:1475 qubes_dom0-pool00_tmeta: Skipping NODE_DEL [trust_udev]
vgchange.c:160 Deactivated 3 logical volumes in volume group qubes_dom0
activate/dev_manager.c:755 Getting device info for qubes_dom0-swap [LVM-string1]
ioctl/libdm-iface.c:1838 dm info LVM-string1 [ noopencount flush ] [16384] (*1)
activate/dev_manager.c:755 Getting device info for qubes_dom0-pool00 [LVM-string2-pool]
ioctl/libdm-iface.c:1838 dm info LVM-string2-pool [ noopencount flush ] [16384] (*1)
ioctl/libdm-iface.c:1838 dm info LVM-string2 [ noopencount flush ] [16384] (*1)
activate/dev_manager.c:755 Getting device info for qubes_dom0-root [LVM-string3]
ioctl/libdm-iface.c:1838 dm info LVM-string3 [ noopencount flush ] [16384] (*1)
activate/activate.c:1835 Counted 0 active LVs in VG qubes_dom0
vgchange.c:259 0 logical volume(s) in volume group "qubes_dom0" now active
activate/dev_manager.c:755 Getting device info for qubes_dom0-swap [LVM-string1]
ioctl/libdm-iface.c:1838 dm info LVM-string1 [ noopencount flush ] [16384] (*1)
activate/dev_manager.c:755 Getting device info for qubes_dom0-pool00 [LVM-string2-pool]
ioctl/libdm-iface.c:1838 dm info LVM-string2-pool [ noopencount flush ] [16384] (*1)
ioctl/libdm-iface.c:1838 dm info LVM-string2 [ noopencount flush ] [16384] (*1)
activate/dev_manager.c:755 Getting device info for qubes_dom0-root [LVM-string3]
ioctl/libdm-iface.c:1838 dm info LVM-string3 [ noopencount flush ] [16384] (*1)
activate/activate.c:1835 Counted 0 active LVs in VG qubes_dom0
mm/memlock.c:562 Unlock: Memlock counters: locked:0 critical:0 daemon:0 suspended:0
activate/fs.c:489 Syncing device names
cache/lvmcache.c:157 Metadata cache: VG qubes_dom0 wiped.
misc/lvm-flock.c:71 Unlocking /run/lock/lvm/V_qubes_dom0
misc/lvm-flock.c:48 _undo_flock /run/lock/lvm/V_qubes_dom0
device/dev-io.c:625 Closed /dev/mapper/luks3
metadata/vg.c:89 Freeing VG qubes_dom0 at 0xXXXXXXXXXXXX
metadata/vg.c:89 Freeing VG qubes_dom0 at 0xYYYYYYYYYYYY
--
Inspecting the logs reveals that Accordion is also “unselecting all items”. All this occurs every disk rescan, even when the volume group is manually reactivated.
When I click “Done” twice, the “Summary of Changes” states that Anaconda recognizes all the encrypting, partitioning, and formatting I have done on sda
and wants to “Destroy” every step of it just to insert an MSDOS table.
What can I do about this? For example:
- Will preventing Anaconda from deactivating the volume group upon disk rescan solve this? How can I do so?
- Is there something I am doing wrong that I can change to successfully expose the LVM volumes for mountpoint assignment without creating a partition table on
sda
? - Can Anaconda somehow be tricked into thinking
sda
has a partition table? - What can I do to hotfix or modify Anaconda to allow installation on a raw data disk?
- Is there a way to skip/bypass the “Installation Destination” step and “Begin Installation” after manually assigning mountpoints from
tty2
? - Is a
chroot
and/orunsquashfs
path for installing from thetty2
terminal possible, bypassing Anaconda altogether? - Is there some other way to force installation after manually mounting?
The setup I want is possible through other installers (such as Debian) and with minimal bootstrappers that allow you to install from scratch (like pacstrap
for Arch and debootstrap
for Debian). But it does not seem possible on Fedora, at least through an installer like Anaconda, which does not seem to be able to work on raw devices unless it creates a partition table. If not, I may have to go through the headache of reconfiguring the system post-installation or using an intermediate installation. I want to avoid doing that if I can.
I apologize for the length; this was intended to be a brief summary. If any of you have any advice or suggestions on how I might be able to accomplish the setup I want, preferably during installation from the live OS without any outside tools, I will greatly appreciate it. At this point, I am willing to accept any creative solution, or at least confirmation that this is impossible from installation due to the hardcoded limitations of Anaconda.
Best regards,
John
Expected behavior
Existing LVM volumes on LUKS encrypted raw data disk sda
(no partition table, detached LUKS header) are available for mountpoint assignment. Assign mountpoints and proceed with installation.
Actual behavior
sda
is not listed, LVM volumes are not listed, even when the LUKS container is decrypted and all volumes activated and available. Rescanning disk deactivates the volume group. The “Summary of Changes”, when the full device tree is opened, includes destroying all LUKS and LVM volumes to create an MSDOS table.
Steps to reproduce (simplified)
- Launch Qubes R4.0.3 / Fedora 25 live disk (
sr0
) with a zeroed main drive (sda
) and USB drive (sdb
). If you are running the live disk from a USB drive, too, the device labels will be different. This may be reproducible on later release versions of Fedora/Anaconda, as well. - Click “Installation Destination”, select “I will configure partitioning.” and deselect “Encrypt my data.” Click “Done”.
-
CTRL+ALT+F2
to entertty2
. -
cryptsetup
LUKS container onsda
with LUKS header onsdb
. Open the LUKS container. -
pvcreate
,vgcreate
, andlvcreate
LVM partitioning inside LUKS container. - Deactivate volume group and close LUKS container.
-
CTRL+ALT+F6
back into Anaconda GUI. - In “Manual Partitioning”, check under existing partitions. See no
sda
. - Return
tty2
. Reopen LUKS container, reactivate volume group, and verify that all logical volumes are open and available.lsblk
to see full device block tree. - Return to GUI. Check existing partitions, see no
sda
. - Rescan disk. Check existing partitions, see no
sda
. - Return to
tty2
and runlsblk
. The volume group will be deactivated.grep
both/tmp/anaconda.log
and/tmp/lvm.log
forvgchange
to confirm deactivation. - Reactivate volume group.
- Repeat steps 10 through 13 until you feel numb inside.
- Reactivate volume group.
- Click “Done” twice to see “Summary of Changes”: LUKS container and all LVM partitions are listed and scheduled to be destroyed to make way for creating an MSDOS table on
sda
. - Post to Ask Fedora.