Understanding my system's RAM and swap use

transmission-deamon with thousands of torrents, some (paused) mpv instances, a couple of programs in Docker, two Firefox profiles side-by-side most of the time… I haven’t ran VMs in a while since I’m planning to try if QEMU/KVM meet my needs.

I also reboot and update the system (barring the security patches) quite rarely, it’s time-consuming to set up the workspaces again.

At the time free -h reports 3.3G of memory, while htop in turn displays around 50% used, for unknown reasons.

It seems obvious that something is misbehaving, sure.


I mentioned the driver since it’s not in the first-party software repository.

The AMD drivers are baked into the kernel, hardware acceleration works out-of-the-box and all.

The free memory you see withfree -h is the memory not used and not holding caches, cache is not critical and can be flushed to use the space it is occupying without problems. So the buff/cache is technically also ready to be used memory. Htop actually separates the cache from used memory.
Which means that htop is consistent with free: 50% used in htop means ~13.5 Gi, and free is reporting 14Gi used.

May be it makes sense to look into what is going on on the machine. First: " top" and sort by memory to look for memory eaters. Are you running VM’s?
It’s remarkable that you have 12G for buff/cache with full swap. There is a sysctl parameter “swapiness”, which determines how quick the system moves unused memory to swap, but this is normally not set to max. value. Another thing is tmpfs which is by default 1/2RAM. Are you storing big files in /tmp? When not accessed perfect candidates to be swapped out, but may more efficient in that case to store them in /var/tmp.
Anyhow, for testing you can just create a subvolume @swap and create a swapfile of any size in it, and add this file to swapspace. No repartitioning needed. Take care to disable copy-on-write for it. If the zram swap overflows, there is headroom in physical swap.

2 Likes

There’s also this handy resource that’s worth a read:

https://www.linuxatemyram.com/

4 Likes

Sorting by both CPU and memory use in htop, transmission-daemon appears to consume the most resources (has several PIDs and I’m not running multiple instances). Otherwise I can only immediately spot Firefox (expected due to modern Javascript on the web) and Thunderbird. I have to learn basic logging before I can share much of anything noteworthy.

No and haven’t in a while.

In a positive or negative sense? Ambiguity?

I’ve downloaded larger web video series to /tmp/ since mpv doesn’t cache network playlists. I also tried storing the DRM’d copies of DVDs and BDs made with ddrescue there, which I’ll decrypt with other programs.

I blindly increased the allocated size of the /tmp/ path to and 100GB should’ve certainly done research.

I’ll rewrite the scripts to point elsewhere and may wipe the new locations clean periodically.

Then it’s clear. Linux is eager to cache files for fast access if they are needed another time, and swap is last resort, that’s why 12G buffer/cache and 8G full swap is somewhat remarkable, the cache is emptied before swapping.
Strange that you can create 100G tmpfs without backing storage. May be you can store a file of 100G zeros compressed in 8G zram, but practical value is a bit limited I’m afraid.

In theory you can leave it as it is, provided you add a 80G swapfile to the system. But I assume direct access to a file is faster than grabbing the bits and pieces from swap storage, so storage in /var/tmp is better.
Thanks for the links, it’s interesting. Good to know that the /var/tmp is managed and unused files are deleted after 30 days.

Fedora uses swap on zram by default. When memory is full an existing unmounted swap partition springs up and when whole storage is full (app+data+swap) : system crash. Swap goes back to MULTICS when devices had very small RAM and swapped data from memory to storage being magnetic tape. UNIX followed and after BSD and Linux. On Fedora everything is automatic with Btrfs, Red Hat and CentOS Stream are different using XFS and LVM. System Monitor app gives an easy GUI vision, top as CLI console command also. With 8GB device using 2GB of memory, swap a distant preoccupation.