"Select Audio Device" when plugging in headset

When I plug in a headset or headphones on my laptop, I see a new overlay dialog about selecting audio device. The headset is selected in Settings -> Sound, and works fine, but I still need to at least press Esc to make the dialog go away.

The dialog is unnecessary and annoying, can I avoid it somehow?

Fresh Fedora 33 installation. I don’t remember ever seeing this dialog on F32, GNOME 3.36.