Ask Your Question
2

One of two identical M.2 nvme drives disabling due to same NQN

asked 2018-11-01 20:09:58 -0500

jvbockel gravatar image

Hi,

Trying to install Fedora 29 on an Intel NUC8i7hvk with two Intel 760p 1TB NVMe M.2 drives.
Latest 004C firwmare on the 760p drives and version 50 BIOS on the NUC. If I disable one of the two M.2 drives in the BIOS, the other works perfectly. It doesn't matter which is disabled In the BIOS, both function well when only one is enabled via BIOS.

If I enable both, the nvme_core kernel module disables one of them, see message that follows.

[root@localhost ~]# dmesg | grep -i nvme

[ 8.067045] nvme nvme0: pci function 0000:72:00.0
[ 8.067088] nvme nvme1: pci function 0000:73:00.0
[ 8.281930] nvme nvme0: ignoring ctrl due to duplicate subnqn (nqn.2017-12.org.nvmexpress:uuid:11111111-2222-3333-4444-555555555555).
[ 8.281932] nvme nvme0: Removing after probe failure status: -22
[ 8.284565] nvme0n1: p1

Running "nvme list" with M.2 slot 1 enabled and 2 disabled in the BIOS yields the following.

Node SN Model Namespace Usage Format FW Rev
/dev/nvme0n1 BTHH81850C8W1P0E INTEL SSDPEKKW010T8 1 1.02 TB / 1.02 TB 512 B + 0 B 004C

With M.2 slot 2 enabled and slot 1 disabled in the BIOS gives:

Node SN Model Namespace Usage Format FW Rev
/dev/nvme0n1 BTHH81850BX31P0E INTEL SSDPEKKW010T8 1 1.02 TB / 1.02 TB 512 B + 0 B 004C

Note the two NVMe drives sharing a common model number but different serial numbers.
I've also experimented with GPT and DOS partition tables, making certain, with the blkid command, that the UUID's where always different between the drives.

With slot 1 enabled in BIOS, the "nvme list-subsys" command yields:

nvme-subsys0 - NQN=nqn.2017-12.org.nvmexpress:uuid:11111111-2222-3333-4444-555555555555
+- nvme0 pcie 0000:72:00.0

With slot 2 enabled in BIOS, the "nvme list-subsys" command yields:

nvme-subsys0 - NQN=nqn.2017-12.org.nvmexpress:uuid:11111111-2222-3333-4444-555555555555
+- nvme0 pcie 0000:73:00.0

The problem seems to be that something (not sure if the BIOS or Linux) is assigning the same NVMe Qualified Name (NQN) to both NVMe drives when both are enabled in the BIOS. Thinking it might be due to BIOS, I tried all kinds of variations in the BIOS settings. Not sure if the NQN is coming from the BIOS or Linux kernel. So not sure whether to blame Intel or the kernel maintainers.

I did have one occassion, amongs dozens of attempts, where the nvme_core kernel module did not disable one of the two nvme drives. Both appeared in the "nvme list-subsys" output as nvme0 and nvme1 with working /dev/nvme0n1 and /dev/nvme0n2 entries under /dev. Was excited when I achieved that result but was fagile and unable to repeat the success. I think it might have had to do with having different grub2 MBR installs on each and corresponding EFI boot records for each in the EFI cache at the time, not sure.

Maybe the BIOS maintainers are to blame. If so, I'll take this to Intel Support instead. Read a bit ... (more)

edit retag flag offensive close merge delete

4 Answers

Sort by ยป oldest newest most voted
1

answered 2018-11-02 04:11:38 -0500

G0bble gravatar image

updated 2018-11-02 04:12:33 -0500

I use a pcie ahci version of SM951 drive so I cant test this but you should experiment more with nvme-cli. See also [https://www.mankier.com/1/nvme]

there seem to be some commands like

` nvme-create-ns(1)

Create a new namespace

nvme-delete-ns(1)

Delete existing namespace`

HTH G0bble

edit flag offensive delete link more
1

answered 2018-11-03 01:14:58 -0500

jvbockel gravatar image

updated 2018-11-03 01:15:39 -0500

Hi G0bble,

Thanks for the guidance and encouragement. I was curious as to whether the problem was specific to Linux so I installed Windows 10 Pro while both NVMe M.2 drives were enabled in the BIOS. Windows had no problem using both simultaneously.

Did some more reading of the NVMe 1.3c spec and played a bit with the nvme_cli. No breakthroughs obtained so will take a break for a bit. I'll try again some months from now with Fedora 30 or another distribution.

Thanks.

edit flag offensive delete link more

Comments

Hi, have you tried reporting it on https://bugzilla.redhat.com/ or https://bugzilla.kernel.org/? It probably won't get fixed until someone capable hits this bug or gets report about it.

ozeszty gravatar imageozeszty ( 2018-11-03 06:23:19 -0500 )edit

Whoa! nvme drives are too costly and their performance too good to let lie for a future release .. up to you :) I do see some related stuff that should have been patched : https://lkml.org/lkml/2017/6/4/69 try posting your questions directly in the comments of a kernel developer blog/post doing the nvme driver stuff.

Good luck and Cheers!

G0bble gravatar imageG0bble ( 2018-11-04 01:07:03 -0500 )edit
0

answered 2018-11-16 18:37:45 -0500

jvbockel gravatar image

Hi,

Sorry about the slow reply, have been on the road. Tried Ubuntu 18.10 and had the same problem. Not surprising given that it is using substantially the same kernel version. Also tried the 4.18.18 update without success.

I did, moments ago, send a message with detail to to the NVMe kernel module maintainer's list. Will let you know if anything interesting results.

Thanks

edit flag offensive delete link more

Comments

ozeszty gravatar imageozeszty ( 2018-11-17 05:09:12 -0500 )edit

So it boils down to a hardware, or better a firmware problem. Any guess why windows was able to use both?

ed209 gravatar imageed209 ( 2018-11-21 18:53:06 -0500 )edit
0

answered 2018-11-21 15:33:55 -0500

jvbockel gravatar image

Hi,

I submitted the question to the NVMe kernel module maintainers via linux-nvme@lists.infradead.org. Keith responded by steering me towards a firmware update for the Intel 760p and 7600p Pro drives that addresses this NQN name conflict.

https://downloadcenter.intel.com/down...

The drives now have unique NQN names and all appears to be working.
A big thanks to responders here, Keith on the linux-nvme list, and Intel for being quick to fix the FW. Hoping you all have a great holiday.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2018-11-01 20:09:58 -0500

Seen: 493 times

Last updated: Nov 21 '18