[opensuse] NAS home backup?
I would like to set up an older desktop system for backing up five home systems over the internal LAN. This would probably be a scheduled weekly job involving both Linux and Windows boxes. What hardware other than the computer, LAN card, and LARGE (2+ TB-wife has LOTS of photos) disk would be needed? I'm thinking that rsync would be the best for this use but would entertain other methods. I have a bash file that I got from someone on the OS forum several years ago but looks fairly simple to modify. My main question regards backing up from the two Windows boxes. Would that need Samba or some other software? The backup box has: Asus M5A?? AMD socket 3 CPU 4 GB RAM 2+ TB hard disk old PCIe video card (Nvidia ?) for setup only Thanks, Tom -- "To be free is not merely to cast off one's chains, but to live in a way that respects and enhances the freedom of others." - Nelson Mandela ^^ --... ...-- / -.- --. --... -.-. ..-. -.-. ^^^^ Tom Taylor KG7CFC openSUSE 13.1 (64-bit), Kernel 3.11.10-17-default, KDE 4.11.2, AMD Phenom X4 955, GeForce GTX 550 Ti (Nvidia 337.19) 16GB RAM -- 3x1.5TB sata2 -- 128GB-SSD FF 27.0, claws-mail 3.10.0 registered linux user 263467 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Personally, I use ownCloud on my oS 12.2 box; it is used to back up 4 different PC's - two Win7, one oS 12.3, and one oS 13.1. The software is built on OBS, and they have clients for windows, Mac, and linux. I back up (and sync) about 50GB of data, but it could handle a lot more. As for the box it's installed on, it's a Core2Duo 2x2.2GHz, 4GB memory, PDSMI+ system board, 2x250GB hard drives in RAID1, dual gig-E. The beauty of it is that it's just a PHP-based application that hooks into a mySQL database. Very simple and straightforward. I have an rsnapshot-based backup of the server on top of that, so there are at least 3 copies of a file lying around - computer, server, and backup. Chris -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thursday 10 of July 2014 15:26:58 Christopher Myers wrote:
Personally, I use ownCloud I back up (and sync) about 50GB of data, but it could handle a lot more.
Using owncloud here, too, for 240MiB. However, it has one problem: the file sync code on the server is very inefficient, which makes syncing of many small files very time consuming. This is expected to be fixed in the near future. Also, I'm using rdiff-backup and rsync for ~300GiB, but I'm planning to change the former to rsnapshot or something else in the near future, because it is no longer under development. -- Regards, Peter -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-- Greg Freemyer On Thu, Jul 10, 2014 at 3:55 PM, Thomas Taylor <linxt@comcast.net> wrote:
I would like to set up an older desktop system for backing up five home systems over the internal LAN. This would probably be a scheduled weekly job involving both Linux and Windows boxes.
What hardware other than the computer, LAN card, and LARGE (2+ TB-wife has LOTS of photos) disk would be needed?
I'm thinking that rsync would be the best for this use but would entertain other methods. I have a bash file that I got from someone on the OS forum several years ago but looks fairly simple to modify. My main question regards backing up from the two Windows boxes. Would that need Samba or some other software?
The backup box has: Asus M5A?? AMD socket 3 CPU 4 GB RAM 2+ TB hard disk old PCIe video card (Nvidia ?) for setup only
Thanks, Tom
Tom Taylor KG7CFC openSUSE 13.1 (64-bit), Kernel 3.11.10-17-default, KDE 4.11.2, AMD Phenom X4 955, GeForce GTX 550 Ti (Nvidia 337.19) 16GB RAM -- 3x1.5TB sata2 -- 128GB-SSD FF 27.0, claws-mail 3.10.0 registered linux user 263467 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
If you're a linux guy and you don't need samba for another reason, I would install cygwin / rsync on the windows boxes. Then use rsync to sync between the 2 boxex via ssh. It is a very common method of using rsync, so you should have no trouble finding howto's. If for some reason you don't want to use ssh, you can setup rsync as a daemon on the backup server and then have the clients connect to the backup server via a pure rsync method. You can setup a scheduled task on the windows boxes similar to using cron to do it on the linux machines. Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 10/07/14 16:55, Thomas Taylor wrote:
I would like to set up an older desktop system for backing up five home systems over the internal LAN. [snip]
[snip]
I'm thinking that rsync would be the best for this use but would entertain other methods. [snip] My main question regards backing up from the two Windows boxes. Would that need Samba or some other software?
[snip]
Thanks, Tom
I use rsync through a script on my Linux boxes (through a NFS share) and Create Synchronicity (http://synchronicity.sourceforge.net/) on my Windows box (through a Samba share). On some of my Linux boxes the script runs when I logout/shutdown (via ~/.kde4/shutdown) and others I use cron to start the script. On Windows, I have to manually initiate the backup in Create Synchronicity. There may be a way to schedule it, just haven't bothered to look into it. I have a DLink DNS-323 NAS with NFS support. I have been using this setup for a couple of years without any problems. HTH, Alvin -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
2014-07-10 17:25 GMT-06:00 Alvin Beach <alvinbeach@gmail.com>:
On 10/07/14 16:55, Thomas Taylor wrote:
I would like to set up an older desktop system for backing up five home systems over the internal LAN. [snip]
[snip]
I'm thinking that rsync would be the best for this use but would entertain other methods. [snip] My main question regards backing up from the two Windows boxes. Would that need Samba or some other software?
[snip]
Thanks, Tom
I use rsync through a script on my Linux boxes (through a NFS share) and Create Synchronicity (http://synchronicity.sourceforge.net/) on my Windows box (through a Samba share).
On some of my Linux boxes the script runs when I logout/shutdown (via ~/.kde4/shutdown) and others I use cron to start the script.
On Windows, I have to manually initiate the backup in Create Synchronicity. There may be a way to schedule it, just haven't bothered to look into it.
Very good, although you must first auntenticarse for shared directory Have you tried it on Windows 7 64-bit -- Saludos, cheperobert -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Thomas Taylor wrote:
I would like to set up an older desktop system for backing up five home systems over the internal LAN. This would probably be a scheduled weekly job involving both Linux and Windows boxes.
What hardware other than the computer, LAN card, and LARGE (2+ TB-wife has LOTS of photos) disk would be needed?
---- If you are going to backup "alot", I'd suggest upgrading your internal lan to 1Gbit if you are not there already. I wouldn't bother trying to use NFS on windows.. Cygwin/rsync is "ok", but even w/gbit, don't expect more than about 5-10MB/s. Knock that down to <1-3MB/s if you use anytype of compression. A problem w/rsync is that you will have to tweak the options a fair amount to backup groups, and win-permission lists. If you want to be able to do full restores, you'll find it easiest to use Windows Backup (or NT-backup for full backups on WinXP). Windows achieves 60-70MB/s doing or restoring a backup over a 1Gb net with good disks. If you just want to copy over files, which is likely for more frequent backups of data disks (maybe do image backup w/winbackup -- but would need samba setup for that ), but say you wanted to copy over the target machines "home" (/Users) directory(ies). I think you will find it faster to simply tar up files or copy them with the Win disks mounted on linux. It will usually be faster than trying to push the data on Win to a samba server, after you add in disk i/. *IF* you work hard and tune samba (I have yet to install Samba 4, so I don't know what the perf numbers are using it), I could regularly get 125MB writes and 119MB reads as reported by 'dd' (reports SI units). If you want to do compression, and if you have the room, and time, you might do the uncompressed backup so it will finish in some reasonable time, then run a compress job of choice on linux. For large archives, I think rzip will give you the most bang for the buck, BUT for large files, it can still take 1 or more days (which is why I don't bother compressing anymore). Even w/lowest settings on gzip, my backup speed was slowed to <5MB/s (latencies is a killer...and today's machines aren't fast enough compress much faster than that. Encryption schemes will slow that down more (for backups, you don't want to *have* to use encryption -- i.e. backup computer on "internal, non-routable (192.168.0-255.0-255, for example), but that could require 2 network cards (1Gb cards can be had for <$20 (or they used to)). I try to use dual intf. cards on most of my computers. One way to "stay ahead" in the backup game... is to lower the need for it. I keep all my data on a "server" (an actual dell server these days -- up from my old dual-socket P-III workstation solution after it died after almost 10 years in service). It depends on the applications and speed requirements, but I keep my Documents and Media (basically all my "content") on the linux box where I can run direct backups on it. Overall .. if you aren't looking to restore a dead system from a backup but just save copies of personal files, the running of tar on cifs mount of the remote hard disks is likely the way to go. You'll have to be sure to give 'Admin' full access to all the files you want to backup. You could try multiple backup methods and go w/what works best in your environment -- more work, but better result long term. post back if you have Q's..... Linda -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thu, 10 Jul 2014 12:55:40 -0700 Thomas Taylor <linxt@comcast.net> wrote:
I would like to set up an older desktop system for backing up five home systems over the internal LAN. This would probably be a scheduled weekly job involving both Linux and Windows boxes.
What hardware other than the computer, LAN card, and LARGE (2+ TB-wife has LOTS of photos) disk would be needed?
I'm thinking that rsync would be the best for this use but would entertain other methods. I have a bash file that I got from someone on the OS forum several years ago but looks fairly simple to modify. My main question regards backing up from the two Windows boxes. Would that need Samba or some other software?
The backup box has: Asus M5A?? AMD socket 3 CPU 4 GB RAM 2+ TB hard disk old PCIe video card (Nvidia ?) for setup only
Thanks, Tom
Thanks for all you who responded. Will do a bit more research on your methods and decide what will work best for me. If the NAS box runs a Linux kernel and can directly access an ntfs file system wouldn't it be possible to use a scheduled bash script to perform backups without needing Samba? I'm a bit hazy regarding Windows-Linux file systems integration. Know I can read the Windows partitions on my openSuSE box directly but not sure about over the network Thanks, Tom -- "To be free is not merely to cast off one's chains, but to live in a way that respects and enhances the freedom of others." - Nelson Mandela ^^ --... ...-- / -.- --. --... -.-. ..-. -.-. ^^^^ Tom Taylor KG7CFC openSUSE 13.1 (64-bit), Kernel 3.11.6-4-default, KDE 4.11.2, AMD Phenom X4 955, GeForce GTX 550 Ti (Nvidia 337.19) 16GB RAM -- 3x1.5TB sata2 -- 128GB-SSD FF 27.0, claws-mail 3.10.0 registered linux user 263467 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Fri, Jul 11, 2014 at 11:22 AM, Thomas Taylor <linxt@comcast.net> wrote:
On Thu, 10 Jul 2014 12:55:40 -0700 Thomas Taylor <linxt@comcast.net> wrote:
I would like to set up an older desktop system for backing up five home systems over the internal LAN. This would probably be a scheduled weekly job involving both Linux and Windows boxes.
What hardware other than the computer, LAN card, and LARGE (2+ TB-wife has LOTS of photos) disk would be needed?
I'm thinking that rsync would be the best for this use but would entertain other methods. I have a bash file that I got from someone on the OS forum several years ago but looks fairly simple to modify. My main question regards backing up from the two Windows boxes. Would that need Samba or some other software?
The backup box has: Asus M5A?? AMD socket 3 CPU 4 GB RAM 2+ TB hard disk old PCIe video card (Nvidia ?) for setup only
Thanks, Tom
Thanks for all you who responded. Will do a bit more research on your methods and decide what will work best for me. If the NAS box runs a Linux kernel and can directly access an ntfs file system wouldn't it be possible to use a scheduled bash script to perform backups without needing Samba? I'm a bit hazy regarding Windows-Linux file systems integration. Know I can read the Windows partitions on my openSuSE box directly but not sure about over the network
Thanks, Tom
To share a filesystem between computers you have to have a network protocol. NFS is the classic linux / unix protocol. SMB (or CIFS) is the classic Windows protocol. You can get NFS clients for Windows, or you can get a SMB server for linux. The best known linux SMB server is called Samba :) If you don't need full filesystem semantics (and I don't think you do) you can look at less functional protocols. FTP, SFTP (based on ssh), and rsync's network daemon are examples of less functional protocols. The less functional protocols are typically easier to setup and maintain. If rsync is your backup tool of choice then the rsync daemon would be what I looked at. It won't handle 100% of the NTFS permissions / extended attributes, but do you really need that? Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Greg Freemyer wrote: <deleted full page (34 lines) of wasted space due to *SUSE selfishness> *-don't blaim the user! it's the mailing list that sets the backwards policy
To share a filesystem between computers you have to have a network protocol.
Or a floppy, or a CD, or a serial line, or a parallel port, or a USB port... oh yeah, and you can share over networks as well.
NFS is the classic linux / unix protocol.
--- "classic" linux protocol? SMB was on linux at the same time or before NFS. Samba is included with most linux distros and the cifs file system is in the kernel (as is NFS).
SMB (or CIFS) is the classic Windows protocol.
---- NetBIOS would be more 'classic'...
You can get NFS clients for Windows, or you can get a SMB server for linux. The best known linux SMB server is called Samba :--)
----- BUT -- I've never been able to get NFS to anything closer than 60-80% of SMB -- on linux or Windows. Note -- I was using NFS when I was at Sun back in the late 80's and early 90's. I didn't really get into smb until I got a job in the Windows and Unix Interop group at SGI.
If you don't need full filesystem semantics (and I don't think you do) you can look at less functional protocols. FTP, SFTP (based on ssh), and rsync's network daemon are examples of less functional protocols.
--- You forget 'ssh-fs'? wasn't it a full emulated fs-layer over ssh? (not that I recommend that)... Stay away from encrypted links on private home networks -- they will kill bandwidth.
The less functional protocols are typically easier to setup and maintain.
Oh? It's pretty easy to mount a Windows Share on linux with linux out of the box and no extra softare on windows. That's why it was top on my list in the final recommendations. Can't say that about rsync, or NFS, et al.
If rsync is your backup tool of choice then the rsync daemon would be what I looked at.
Learning howto setup a daemon on windows or schedule a cron job on windows isn't a piece of cake (and it's scheduler isn't reliable).
It won't handle 100% of the NTFS permissions / extended attributes, but do you really need that?
that's why I said it depended on what his needs were -- maybe a full image for windows once a month/week, and daily's for data.. it realyl depends on what his needs are. But can you backup all the files w/o installing any SW on windows?... only thing that comes "close" (permissions and stuff are also problematic) is windows mount via CIFS on a linux system. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Couldn't get smb or netbios for Linux 0.99. Not that I didn't try. I needed them but they were not available. Didn't see a working smb implementation til Redhat 5. Even then, was flaky. nfs, however, worked fine, at least with VAXen and Suns. The Sun NFS client I bought for Win311 never worked. Hmmm... -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
jdebert wrote:
Couldn't get smb or netbios for Linux 0.99. Not that I didn't try. I needed them but they were not available. Didn't see a working smb implementation til Redhat 5. Even then, was flaky.
---- That was before I got into Linux... I didn't get into linux until RH5...so that would explain perceptions. We had smb on IRIX before that as Jeremy was employed at sgi as well for some number of years. Anyway, NFS was on most platforms I was on and I often tried to make it work as a sub for smb -- as it had better access control at the time, but it was always notably slower (even with TCP connections and larger bufs)... Wasn't until 10Gb ethernet that I hit algorithmic speed problems in smb, as it only uses 1 tcp connection per-user-server. Not much way to parallelize that. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 7/14/2014 2:15 AM, Linda Walsh wrote:
jdebert wrote:
Couldn't get smb or netbios for Linux 0.99. Not that I didn't try. I needed them but they were not available. Didn't see a working smb implementation til Redhat 5. Even then, was flaky.
---- That was before I got into Linux... I didn't get into linux until RH5...so that would explain perceptions.
We had smb on IRIX before that as Jeremy was employed at sgi as well for some number of years.
Anyway, NFS was on most platforms I was on and I often tried to make it work as a sub for smb -- as it had better access control at the time, but it was always notably slower (even with TCP connections and larger bufs)...
Wasn't until 10Gb ethernet that I hit algorithmic speed problems in smb, as it only uses 1 tcp connection per-user-server. Not much way to parallelize that.
I can't remember my first use of SMB, but prior to that the organizations I worked for were using Novell Netware. Converting to Linux Samba was pretty simple (all windows machines). When we started adding Linux workstations for the engineering group we fiddeled around with NFS, but the need (at that time) to sync userids on all those machines became a nightmare. (Probably due to our ignorance, but still a hassle). So we kept samba for the linux workstations as well. Never encountered any speed problems that couldn't be solved with faster nics. -- _____________________________________ ---This space for rent--- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Monday 14 Jul 2014 11:46:46 John Andersen wrote:
On 7/14/2014 2:15 AM, Linda Walsh wrote:
jdebert wrote:
Couldn't get smb or netbios for Linux 0.99. Not that I didn't try. I needed them but they were not available. Didn't see a working smb implementation til Redhat 5. Even then, was flaky.
---- That was before I got into Linux... I didn't get into linux until RH5...so that would explain perceptions.
We had smb on IRIX before that as Jeremy was employed at sgi as well for some number of years.
Anyway, NFS was on most platforms I was on and I often tried to make it work as a sub for smb -- as it had better access control at the time, but it was always notably slower (even with TCP connections and larger bufs)...
Wasn't until 10Gb ethernet that I hit algorithmic speed problems in smb, as it only uses 1 tcp connection per-user-server. Not much way to parallelize that.
I can't remember my first use of SMB, but prior to that the organizations I worked for were using Novell Netware. Converting to Linux Samba was pretty simple (all windows machines). When we started adding Linux workstations for the engineering group we fiddeled around with NFS, but the need (at that time) to sync userids on all those machines became a nightmare. (Probably due to our ignorance, but still a hassle).
So we kept samba for the linux workstations as well. Never encountered any speed problems that couldn't be solved with faster nics. Windows 7 supports NFS without any 3rd party software but you have to map the share using the command line. -- Paul Groves -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 7/15/2014 9:03 AM, Paul Groves wrote:
Windows 7 supports NFS without any 3rd party software but you have to map the share using the command line.
I didn't know that. Is that with "net use" command line? -- _____________________________________ ---This space for rent--- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 07/15/2014 01:18 PM, John Andersen wrote:
On 7/15/2014 9:03 AM, Paul Groves wrote:
Windows 7 supports NFS without any 3rd party software but you have to map the share using the command line. I didn't know that. Is that with "net use" command line?
Please advise what the command line would read. Would it need to be entered as admin? Thanks--doug -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 7/15/2014 11:33 AM, Doug wrote:
On 07/15/2014 01:18 PM, John Andersen wrote:
On 7/15/2014 9:03 AM, Paul Groves wrote:
Windows 7 supports NFS without any 3rd party software but you have to map the share using the command line. I didn't know that. Is that with "net use" command line?
Please advise what the command line would read. Would it need to be entered as admin?
Thanks--doug
I've found some doco on Technet: http://technet.microsoft.com/en-us/library/cc754350.aspx however this requires add-in software: Client for NFS which did not come with my Windows 7 Ultimate, NFS support is one of those optional packages that is available via the Add/Remove Software wizard in the Control Panel. -- _____________________________________ ---This space for rent--- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thursday 10 Jul 2014 12:55:40 Thomas Taylor wrote:
I would like to set up an older desktop system for backing up five home systems over the internal LAN. This would probably be a scheduled weekly job involving both Linux and Windows boxes.
What hardware other than the computer, LAN card, and LARGE (2+ TB-wife has LOTS of photos) disk would be needed?
I'm thinking that rsync would be the best for this use but would entertain other methods. I have a bash file that I got from someone on the OS forum several years ago but looks fairly simple to modify. My main question regards backing up from the two Windows boxes. Would that need Samba or some other software?
The backup box has: Asus M5A?? AMD socket 3 CPU 4 GB RAM 2+ TB hard disk old PCIe video card (Nvidia ?) for setup only
Thanks, Tom I have two boxes running freenas both with 2x 1TB 2.5" WD Blue HDDs in a ZFS mirror.
Every night the NAS wakes up the backup NAS with WOL at 1:50 then an rsync task is run for documents and media at 2am to backup the disks on the first NAS to the second one. It works perfectly and after the initial sync only takes on average 10 mins to half hour each night. Perhaps you could do something similar but with bigger disks (maybe 2x3TB in a mirror RAID??) for your multiple computers? I personally do not store any documents on my desktop / laptop, I keep them all on the NAS which I can access via SFTP, CIFS and NFS. FreeNAS supports CIFS (Samba) NFS and AFP (Apple File Protocol) out of the box as well as Rsync. All configurable with the GUI!!! :D I would personally look for a Rsync server for windows and use wolcmd on the windows pc to wake up the backup NAS (or wol from linux / wake from BSD). As for hardware, my setup is running fine with the 1Gbps NIC in each box with no bandwidth issues even with DLNA on the main NAS. Just make sure there is plenty of RAM in the NAS (8GB +) to handle the RAID and possilbility of multiple file transfers. -- Paul Groves -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (11)
-
Alvin Beach -
auxsvr＠gmail.com -
Christopher Myers -
Doug -
Greg Freemyer -
jdebert -
John Andersen -
José Roberto Alas -
Linda Walsh -
Paul Groves -
Thomas Taylor