[opensuse] cron and storebackup setup
Hi Everyone, It has been some time since I have been watching the list - my life got really busy over the past year and only now I think I will be coming back to read the emails more frequently. I have a little problem I hope someone can help with. I run storebackup dailyon my 13.2 box that I just got back up and running, having it set up to run in /etc/cron.daily, along with the other programs that are in there. The problem is, storebackup ends up taking a lot of resources while it is doing the backup, and can slow down the pc enough to be kind of annoying. So I would like to run it in the middle of the night. Unfortunately, electricity is really expensive here, and I don't want to run my computer all night long or I will build up a pretty big energy bill. So I was thinking, maybe I can add the line pm-suspend to the script that runs storebackup, and then set the cron.daily time to go at 1 am, or something like that. The idea being, I will leave my computer on when I goto bed, then in the middle of the night it will run the storebackup job, and when it is done, it will suspend to ram and save my energy bill. But then I am thinking, what if it finishes running storebackup before running the other cron jobs, then goes to suspend, and they don't get run? Will there be a problem if thesystem is put in suspend to ram mode while another cron job is in the middle of running? Is it possible to somehow set things up to have the storebackup cron job be the last one to run,and only after all the others have run? -- George Box #1: 13.2 | KDE 4.14 | AMD Phenom IIX4 | 64 | 16GB Box #2: 13.1 | KDE 4.12 | AMD Athlon X3 | 64 | 4GB Laptop #1: 13.1 | KDE 4.12 | Core i7-2620M | 64 | 8GB Laptop #2: 13.2 | KDE 4.14 | Core i7-4710HQ | 64 | 16GB -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 08/27/2015 10:28 AM, George Olson (SUSE list) wrote:
The problem is, storebackup ends up taking a lot of resources while it is doing the backup, and can slow down the pc enough to be kind of annoying.
which resources? RAM, CPU, disc, network, ...? I don't know storebackup, but did you try 'nice' and 'ionice'? Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 08/27/2015 05:23 PM, Bernhard Voelker wrote:
On 08/27/2015 10:28 AM, George Olson (SUSE list) wrote:
The problem is, storebackup ends up taking a lot of resources while it is doing the backup, and can slow down the pc enough to be kind of annoying.
which resources? RAM, CPU, disc, network, ...?
I don't know storebackup, but did you try 'nice' and 'ionice'?
Have a nice day, Berny storebackup uses bzip2 to backup everything, and the algorithm is able to compress and save quite a bit of disk space, and it makes it easy to restore. I keep 30 days worth of backups on a separate drive, in addition to my RAID1 drives to protect from drive failure, and that seems to be a helpful way of doing things for me.
bzip2 seems to use a lot of processor power, not too much memory. The system load viewer right now, as I am running it, shows 5 different bzip2 processes running simultaneously, all of which are taking up about 14% of the CPU. This is the script that calls storebackup in the cron file, if anyone is interested: ---------------- #! /bin/sh -e # run storeBackup on all files in /etc/storebackup.d/ sequentially # file names must consist entirely of letters, digits, underscores, and hyphens # Written by Arthur Korn <arthur@korn.ch> for Debian (http://www.debian.org), # in the PUBLIC DOMAIN. PATH=/bin:/sbin:/usr/bin:/usr/sbin [ -x /usr/bin/storeBackup ] || exit 0 configs=`find /etc/storebackup.d/ -type f \( ! -iname "*~" \)` delayed_error='' if [ "$configs" ]; then tmplog=`mktemp -t storebackup.XXXXXXXXXX` for file in $configs do if ! nice storeBackup -f "$file" > "$tmplog" 2>&1 then echo Error running backup for \"$file\" >&2 cat "$tmplog" >&2 delayed_error=1 fi done rm $tmplog || true [ $delayed_error ] && exit 1; fi exit 0 --------------------- So it does check with nice, but I am not really sure how I could adjust it so it takes less processing power. Maybe add a 'nice' command in the call line? I think this is the call line: [ -x /usr/bin/storeBackup ] || exit 0 but I am not familiar with this syntax, the bracket and then the -x followed by the command. In any case, I will also try Yamaban's suggestion of making a desktop file to run system right before going to bed. -- George Box #1: 13.2 | KDE 4.14 | AMD Phenom IIX4 | 64 | 16GB Box #2: 13.1 | KDE 4.12 | AMD Athlon X3 | 64 | 4GB Laptop #1: 13.1 | KDE 4.12 | Core i7-2620M | 64 | 8GB Laptop #2: 13.2 | KDE 4.14 | Core i7-4710HQ | 64 | 16GB -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 08/27/2015 03:03 PM, George Olson (SUSE list) wrote:
bzip2 seems to use a lot of processor power, not too much memory. The system load viewer right now, as I am running it, shows 5 different bzip2 processes running simultaneously, all of which are taking up about 14% of the CPU.
maybe there's a configuration file which would limit the number of simulataneous bzip2 processes to e.g. 2. Anyway, I'd try using an even more relaxing 'nice' value (see 'man nice'):
if ! nice storeBackup -f "$file" > "$tmplog" 2>&1
... nice -n19 storeBackup By that, all regular processes should get more percentage of the CPUs, and you shouldn't see much delay on the desktop. Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 08/27/2015 09:16 PM, Bernhard Voelker wrote:
On 08/27/2015 03:03 PM, George Olson (SUSE list) wrote:
bzip2 seems to use a lot of processor power, not too much memory. The system load viewer right now, as I am running it, shows 5 different bzip2 processes running simultaneously, all of which are taking up about 14% of the CPU.
maybe there's a configuration file which would limit the number of simulataneous bzip2 processes to e.g. 2.
Anyway, I'd try using an even more relaxing 'nice' value (see 'man nice'):
if ! nice storeBackup -f "$file" > "$tmplog" 2>&1
... nice -n19 storeBackup
By that, all regular processes should get more percentage of the CPUs, and you shouldn't see much delay on the desktop.
Have a nice day, Berny
Great, I will try that! -- George Box #1: 13.2 | KDE 4.14 | AMD Phenom IIX4 | 64 | 16GB Box #2: 13.1 | KDE 4.12 | AMD Athlon X3 | 64 | 4GB Laptop #1: 13.1 | KDE 4.12 | Core i7-2620M | 64 | 8GB Laptop #2: 13.2 | KDE 4.14 | Core i7-4710HQ | 64 | 16GB -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2015-08-27 15:03, George Olson (SUSE list) wrote:
bzip2 seems to use a lot of processor power, not too much memory. The system load viewer right now, as I am running it, shows 5 different bzip2 processes running simultaneously, all of which are taking up about 14% of the CPU.
Only 14% each? That's /too/ little. :-) Total 60%. Not 100%. Or rahter, 400%, which would be full power. Your issue is probably disk speed. Where is the backup stored, external disk? - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlXfHnEACgkQja8UbcUWM1zkugD/TvVVCxELBhzo96Aks8Q4Xehr 1WLENLDOnMTBkYu8M/4A/3KUX6q/43nV8TL2cdjTRvJBxIvw6JE4GNXy5cOoU3vq =dSYr -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 08/27/2015 10:28 PM, Carlos E. R. wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 2015-08-27 15:03, George Olson (SUSE list) wrote:
bzip2 seems to use a lot of processor power, not too much memory. The system load viewer right now, as I am running it, shows 5 different bzip2 processes running simultaneously, all of which are taking up about 14% of the CPU. Only 14% each? That's /too/ little. :-)
Total 60%. Not 100%. Or rahter, 400%, which would be full power. Your issue is probably disk speed.
Where is the backup stored, external disk?
It is on another drive in the computer. My external backup is done once a week - this is just for keeping incremental files backed up in case I mess something up and need to go back to a version from a month or so ago. But the drive it is on is a RAID1 drive, about a 900G ext4 format. -- George Box #1: 13.2 | KDE 4.14 | AMD Phenom IIX4 | 64 | 16GB Box #2: 13.1 | KDE 4.12 | AMD Athlon X3 | 64 | 4GB Laptop #1: 13.1 | KDE 4.12 | Core i7-2620M | 64 | 8GB Laptop #2: 13.2 | KDE 4.14 | Core i7-4710HQ | 64 | 16GB -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2015-08-28 12:52, George Olson (SUSE list) wrote:
It is on another drive in the computer. My external backup is done once a week - this is just for keeping incremental files backed up in case I mess something up and need to go back to a version from a month or so ago.
But the drive it is on is a RAID1 drive, about a 900G ext4 format.
I would test the write speed of this. Write a big file full of zeroes with dd. Something like: dd if=/dev/zero of=file bs=1M count=100 - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlXgSfwACgkQja8UbcUWM1zeFQD9GU2NnRdopwyIgIPcn/SYPmct eE9bVNaShH5q8udh5xcA/AoO9qLrANYI96iNTtQAr8Ct2s/2TH2Hr/SNHtbU/7Nu =8Utq -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thu, 27 Aug 2015 10:28, George Olson wrote:
Hi Everyone,
It has been some time since I have been watching the list - my life got really busy over the past year and only now I think I will be coming back to read the emails more frequently.
I have a little problem I hope someone can help with. I run storebackup dailyon my 13.2 box that I just got back up and running, having it set up to run in /etc/cron.daily, along with the other programs that are in there.
The problem is, storebackup ends up taking a lot of resources while it is doing the backup, and can slow down the pc enough to be kind of annoying.
So I would like to run it in the middle of the night. Unfortunately, electricity is really expensive here, and I don't want to run my computer all night long or I will build up a pretty big energy bill. So I was thinking, maybe I can add the line
pm-suspend
to the script that runs storebackup, and then set the cron.daily time to go at 1 am, or something like that. The idea being, I will leave my computer on when I goto bed, then in the middle of the night it will run the storebackup job, and when it is done, it will suspend to ram and save my energy bill.
But then I am thinking, what if it finishes running storebackup before running the other cron jobs, then goes to suspend, and they don't get run? Will there be a problem if thesystem is put in suspend to ram mode while another cron job is in the middle of running?
Is it possible to somehow set things up to have the storebackup cron job be the last one to run,and only after all the others have run?
To have "storebackup" cronjob run as last job, add a prefix like "zzz_" to the name of the scriptname in the /etc/cron.daily directory. And for suspend, please, do yourself a favor and use "systemctl suspend" instead of calling "pm-suspend". Also, how about creating a script / starter / .desktop-file that contains: [code] xdg-su -u root -c "/etc/cron.daily/zzz_storebackup" [/code] as command line, add this to your desktop / menu and call it before going to bed / leaving the comp to the night. That way there would be no extra wait to run storebackup, and by including "systemctl suspend" as the last command, it would do a suspend to ram after finishing. More power is safed. - Yamaban. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 08/27/2015 05:51 PM, Yamaban wrote:
On Thu, 27 Aug 2015 10:28, George Olson wrote:
Hi Everyone,
It has been some time since I have been watching the list - my life got really busy over the past year and only now I think I will be coming back to read the emails more frequently.
I have a little problem I hope someone can help with. I run storebackup dailyon my 13.2 box that I just got back up and running, having it set up to run in /etc/cron.daily, along with the other programs that are in there.
The problem is, storebackup ends up taking a lot of resources while it is doing the backup, and can slow down the pc enough to be kind of annoying.
So I would like to run it in the middle of the night. Unfortunately, electricity is really expensive here, and I don't want to run my computer all night long or I will build up a pretty big energy bill. So I was thinking, maybe I can add the line
pm-suspend
to the script that runs storebackup, and then set the cron.daily time to go at 1 am, or something like that. The idea being, I will leave my computer on when I goto bed, then in the middle of the night it will run the storebackup job, and when it is done, it will suspend to ram and save my energy bill.
But then I am thinking, what if it finishes running storebackup before running the other cron jobs, then goes to suspend, and they don't get run? Will there be a problem if thesystem is put in suspend to ram mode while another cron job is in the middle of running?
Is it possible to somehow set things up to have the storebackup cron job be the last one to run,and only after all the others have run?
To have "storebackup" cronjob run as last job, add a prefix like "zzz_" to the name of the scriptname in the /etc/cron.daily directory.
And for suspend, please, do yourself a favor and use "systemctl suspend" instead of calling "pm-suspend".
Also, how about creating a script / starter / .desktop-file that contains:
[code] xdg-su -u root -c "/etc/cron.daily/zzz_storebackup" [/code]
as command line, add this to your desktop / menu and call it before going to bed / leaving the comp to the night.
That way there would be no extra wait to run storebackup, and by including "systemctl suspend" as the last command, it would do a suspend to ram after finishing. More power is safed.
- Yamaban.
That is an excellent idea! Thank you for thinking of that - I will take it out of the cron daily and just run it from the desktop, running it from another directory. Thank you also for pointing out the appropriate systemd command for suspend. Always good to know these things! -- George Box #1: 13.2 | KDE 4.14 | AMD Phenom IIX4 | 64 | 16GB Box #2: 13.1 | KDE 4.12 | AMD Athlon X3 | 64 | 4GB Laptop #1: 13.1 | KDE 4.12 | Core i7-2620M | 64 | 8GB Laptop #2: 13.2 | KDE 4.14 | Core i7-4710HQ | 64 | 16GB -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thursday 27 Aug 2015 16:28:33 George Olson wrote:
Hi Everyone,
It has been some time since I have been watching the list - my life got really busy over the past year and only now I think I will be coming back to read the emails more frequently.
I have a little problem I hope someone can help with. I run storebackup dailyon my 13.2 box that I just got back up and running, having it set up to run in /etc/cron.daily, along with the other programs that are in there.
The problem is, storebackup ends up taking a lot of resources while it is doing the backup, and can slow down the pc enough to be kind of annoying.
So I would like to run it in the middle of the night. Unfortunately, electricity is really expensive here, and I don't want to run my computer all night long or I will build up a pretty big energy bill. So I was thinking, maybe I can add the line
pm-suspend
to the script that runs storebackup, and then set the cron.daily time to go at 1 am, or something like that. The idea being, I will leave my computer on when I goto bed, then in the middle of the night it will run the storebackup job, and when it is done, it will suspend to ram and save my energy bill.
But then I am thinking, what if it finishes running storebackup before running the other cron jobs, then goes to suspend, and they don't get run? Will there be a problem if thesystem is put in suspend to ram mode while another cron job is in the middle of running?
Is it possible to somehow set things up to have the storebackup cron job be the last one to run,and only after all the others have run?
I use rsync to back my data up. i have it configured in a script that runs at logout or shutdown. Could you not run your script at logout as well? I also keep meaning to look for a way to run the back up script when the screensaver is running -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 08/28/2015 12:36 AM, ianseeks wrote:
On Thursday 27 Aug 2015 16:28:33 George Olson wrote:
be the last one to run,and only after all the others have run? I use rsync to back my data up. i have it configured in a script that runs at logout or shutdown. Could you not run your script at logout as well? I also keep meaning to look for a way to run the back up script when the screensaver is running That is something to think about. I normally don't log out unless I have an update that requires a reboot, or I need to reset something in KDE. But it is worth thinking about if I change my strategy.
-- George Box #1: 13.2 | KDE 4.14 | AMD Phenom IIX4 | 64 | 16GB Box #2: 13.1 | KDE 4.12 | AMD Athlon X3 | 64 | 4GB Laptop #1: 13.1 | KDE 4.12 | Core i7-2620M | 64 | 8GB Laptop #2: 13.2 | KDE 4.14 | Core i7-4710HQ | 64 | 16GB -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (5)
-
Bernhard Voelker -
Carlos E. R. -
George Olson (SUSE list) -
ianseeks -
Yamaban