Backup slows down after installing F-33

[gms2]

@computersavvy

You might move the chcon part … and only run chcon if they do not match. It appears
you are running chcon on every file, everytime, even if not needed.

It seems, that I misled you by the formulation of my previous reply. While the context
might be different for each file, I need to change it only for the directories…

@computersavvy

rsync for example will copy files and maintain all attributes when using archive mode (-a). “cp -a” does the same, although it copies every file, not just those that have changed. If your backup program does not copy the attributes but instead relies on chcon then the added overhead to run chcon is built in by that lack.

… because I am in fact using “cp -a” for everything else.

Anyway, in the meantime I re-wrote the program in Python. The memory usage is now
in a “decent” area (less then 1/2 of the Perl version) and I actually managed to run
almost a full backup. I got stuck at ca. 40 GB, because one of the files has an “a Umlaut”
(the German a with 2 dots on top) in it’s name. So apart from that small detail, I am now
happy.

[computersavvy (JV)]

@gms2

… because I am in fact using “cp -a” for everything else.

Anyway, in the meantime I re-wrote the program in Python. The memory usage is now
in a “decent” area (less then 1/2 of the Perl version) and I actually managed to run
almost a full backup. I got stuck at ca. 40 GB, because one of the files has an “a Umlaut”
(the German a with 2 dots on top) in it’s name. So apart from that small detail, I am now
happy.

Just an fyi for future reference.
cp -a cannot handle file names with special characters. Rsync handles them with no problem.

[gms2]

@computersavvy

@gms2

… because I am in fact using “cp -a” for everything else.

Anyway, in the meantime I re-wrote the program in Python. … I got stuck at
ca. 40 GB, because one of the files has an “a Umlaut” (the German a with 2 dots
on top) in it’s name. …

Just an fyi for future reference.
cp -a cannot handle file names with special characters. Rsync handles them with no problem.

I can’t comment on rsync, because, while I studied the man page, I never used it.
As far as “cp” goes, YMMV, but in my experience it doesn’t have a problem. Where
the problems come in, is that it is used at shell level, and you have to escape it properly
so that the command knows where to look. (Remember, we are doing a backup, so
we are not creating new names. If the OS gives us a name, you should expect that
a copy command can take it. But don’t let me get onto the soap box talking about the
filenames you encounter nowadays.)

In fact, the issue was not with the cp, but with printing the name (I protocol the changes).
Python prints in UTF-8, and an ISO text stops it dead.

Anyway, now I have a clean version. Python is slightly, but not significantly, faster, but
the real difference is in the memory usage. The Python version uses ca. 14 MB, Perl
(thanks to the fact, that it doesn’t give anything back) uses ca. 3,500 MB.

[computersavvy (JV)]

One significant difference between the functioning of rsync and cp.

cp will not copy anything hidden, such as the many dot files and folders within ones home directory unless it is explicity named. It copies only that which is seen by ls (without options) by default.

rsync will by default copy everything existing in the directory including the hidden dot files.

Thus rsync makes a backup simpler as it does a complete copy instead of a selective copy.