Dnf upgrade crashes with no error reported anywhere

I have a system running on a cloud service to which I only have SSH console access. When I run “dnf -y -v upgrade”, the system gets as far as the Updates repo, then disconnects the SSH session. Reconnecting, I can find no error reported in the journal or syslog (dnf.log). I’ve tried “dnf clean all” and “dnf makecache”, both of which appear to work properly. This is Fedora F36 x86_64 with all updates applied – until this problem started, anyway. Here’s what the console shows:

# dnf -v -b --allowerasing upgrade
Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, groups-manager, needs-restarting, playground, repoclosure, repodiff, repograph, repomanage, reposync, system-upgrade
DNF version: 4.14.0
cachedir: /var/cache/dnf
User-Agent: constructed: ‘libdnf (Fedora Linux 36; server; Linux.x86_64)’
repo: using cache for: fedora
fedora: using metadata from Wed May 4 14:16:11 2022.
repo: using cache for: fedora-cisco-openh264
fedora-cisco-openh264: using metadata from Thu Oct 6 04:02:51 2022.
repo: using cache for: fedora-modular
fedora-modular: using metadata from Wed May 4 14:12:01 2022.
repo: using cache for: updates
Connection to remote closed by remote host.
Connection to remote closed.

This doesn’t appear to be a dnf issue, but it seems either you have an unstable network connection to the server or the SSH session is abruptly hanging up. You might consider setting a KeepAlive in you ssh client and/or using tmux to attempt to keep the connection alive.

Here are the settings you can try for keepalive for you ~/.ssh/config for the host:

    ServerAliveInterval 300
    ServerAliveCountMax 2

Lastly, it’s also possible that if your cloud instance might be running out of memory and is killing processes as a result. The minimum memory requirement for Fedora 36 is 2GB with 4GB+ recommended. If your cloud instance has less than 2GB of memory, it might explain why stuff is crashing on dnf update.

You can give us an idea of your cloud instance specs by running inxi -SCMm and pasting that info back here in code or preformatted text.

1 Like

Scott Williams wrote:

This doesn’t appear to be a dnf issue, but it seems either you have
an unstable network connection to the server or the SSH session is
abruptly hanging up. You might consider setting a KeepAlive in you
ssh client and/or using tmux to attempt to keep the connection alive.

An unstable connection seems unlikely. This server instance has been
live since F32 with no problems. Aside from the dnf issue, it continues
to work perfectly.

Lastly, it’s also possible that if your cloud instance might
be running out of memory and is killing processes as a result.
The minimum memory requirement for Fedora 36 is 2GB with 4GB+
recommended. If your cloud instance has less than 2GB of memory,
it might explain why stuff is crashing on dnf update.

Indeed it might. But I thought the kernel would log OOM errors and I
didn’t find any messages to that effect when looking for messages
containing “dnf”. Looking again, I do see an OOM message referencing
my user slice, so I wasn’t looking for the right thing.

All that said, I was surprised to find that both instances are
actually configured with only 1Gi of RAM. I can easily request an
increase. Thanks for the quick response.

Still, the workload for this instance hasn’t changed in years. Something
must be using more memory recently. Perhaps kernel 6? Unfortunately, I
no longer have any kernel 5 copies installed on this instance. However,
“dnf” did work perfectly immediately after booting just now. So some
application seems to be growing its memory over time. More for me to
investigate…

2 Likes