Re: [opensuse] System slowdowns
On 10/5/2016 12:37 AM, John Andersen wrote:
On 10/04/2016 11:21 PM, Marc Chamberlin wrote:
On 10/4/2016 6:04 PM, John Andersen wrote:
On 10/04/2016 05:29 PM, Marc Chamberlin wrote:
The mouse problems seem worth investigating. Maybe dig in the junk drawer for a totally different kind of mouse (serial, bluetooth, usb-wired) and see if that changes anything.
Thanks John for your suggestion. I tried a different mouse and no joy... Marc....
Remind me again what kernel you are using, and what version of Vbox? I tend to purge my list mail fairly aggressivly. Hi John - I am running Leap 42.1 (x86_64) I am not running VBox, (which I think what you are referring to is VirtualBox?) Interesting that you should ask though because I am playing around with Docker. However, I don't think my system slowdowns has anything to do with Docker as these slowdown issues preceded my installation of Docker. And these slowdowns occur regardless of whether Docker is running or not. (The only reason I am fooling around with Docker, FYI, is so I can get the latest version of the Apache James email server up and running, something that OpenSuSE does not directly support.)
Marc....
So that was originally kernel 4.1. Are you still on that or have you upgraded?
It might be worth trying to boot with iomem=relaxed
added to the kernel command line.
It might not do anything but slowness has been detected in several distros as soon as kernel updates were applied due to tighter memory security.
Thanks John for the thought, I tried adding the iomem=relaxed option to the kernel options using the Yast Boot Loader tool, rebooted, and still no joy... In my fooling around with this issue, I have come across something that definitely triggers high CPU usage, and could be a clue. Although I cannot definitively say this is the smoking gun because I see high system CPU usage going on regardless of whether I do this or not - There is something very wrong going on when I use Dolphin to move stuff to the trash "Recycle Bin" (or sometimes it appears when I use Dolphin to simply move files from one location to another, i.e. drag and drop) and/or when I tell Dolphin to empty the trash. Doing so kicks off the 100% CPU usage and I hear the processor cooling fans kick into high speed. These tasks can take hours or even days to complete (if ever, some are still going on for over a week now. When I reboot the system, these tasks are not initially restarted, but if I do something like open the Recycle Bin to see if the files are still in it (after I told Dolphin to empty the trash) then these processes again kick off and the CPU usage goes up to 100%. The two (or more) spawned processes of interest are file.so and trash.so. However, unlike my original report and observations, these processes do report very high CPU usage also in the Process Table tab of KSystemGuard, top, atop, and htop so I am not 100% certain that this is related to the issue I reported - about something hidden is causing high CPU usage. Of interest also is doing file moves or deletes from the command line, i.e. using the mv, rm, and rmdir commands does not cause this high CPU usage to occur and those commands complete in a reasonable amount of time. Marc... -- "The Truth is out there" - Spooky -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-10-09 18:58, Marc Chamberlin wrote:
There is something very wrong going on when I use Dolphin to move stuff to the trash "Recycle Bin" (or sometimes it appears when I use Dolphin to simply move files from one location to another, i.e. drag and drop) and/or when I tell Dolphin to empty the trash. Doing so kicks off the 100% CPU usage and I hear the processor cooling fans kick into high speed. These tasks can take hours or even days to complete (if ever, some are still going on for over a week now. When I reboot the system, these tasks are not initially restarted, but if I do something like open the Recycle Bin to see if the files are still in it (after I told Dolphin to empty the trash) then these processes again kick off and the CPU usage goes up to 100%. The two (or more) spawned processes of interest are file.so and trash.so. However, unlike my original report and observations, these processes do report very high CPU usage also in the Process Table tab of KSystemGuard, top, atop, and htop so I am not 100% certain that this is related to the issue I reported - about something hidden is causing high CPU usage.
Of interest also is doing file moves or deletes from the command line, i.e. using the mv, rm, and rmdir commands does not cause this high CPU usage to occur and those commands complete in a reasonable amount of time.
What filesystem? -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
On 10/9/2016 10:42 AM, Carlos E. R. wrote:
On 2016-10-09 18:58, Marc Chamberlin wrote:
There is something very wrong going on when I use Dolphin to move stuff to the trash "Recycle Bin" (or sometimes it appears when I use Dolphin to simply move files from one location to another, i.e. drag and drop) and/or when I tell Dolphin to empty the trash. Doing so kicks off the 100% CPU usage and I hear the processor cooling fans kick into high speed. These tasks can take hours or even days to complete (if ever, some are still going on for over a week now. When I reboot the system, these tasks are not initially restarted, but if I do something like open the Recycle Bin to see if the files are still in it (after I told Dolphin to empty the trash) then these processes again kick off and the CPU usage goes up to 100%. The two (or more) spawned processes of interest are file.so and trash.so. However, unlike my original report and observations, these processes do report very high CPU usage also in the Process Table tab of KSystemGuard, top, atop, and htop so I am not 100% certain that this is related to the issue I reported - about something hidden is causing high CPU usage.
Of interest also is doing file moves or deletes from the command line, i.e. using the mv, rm, and rmdir commands does not cause this high CPU usage to occur and those commands complete in a reasonable amount of time. What filesystem?
Oh wow! Interesting question! I just took a look, and well this system has been an evolution since the days of SuSE 10.0. So apparently I have multiple file system types in play. I guess the best way to answer your question is to simply show the output from parted -l I have never taken the time to figure out what file system type is best or what are the pros and cons of each. So usually just gone with defaults when installing a new OS. I wonder if I should normalize all these, if so which file system to choose, and how would one go about doing so. Could this be the issue, having multiple file system types mounted at the same time is what is causing my system to slow down? parted -l Model: ATA WDC WD5001AALS-0 (scsi) Disk /dev/sda: 500GB Sector size (logical/physical): 512B/512B Partition Table: msdos Disk Flags: Number Start End Size Type File system Flags 1 1049kB 53.7GB 53.7GB primary linux-swap(v1) type=82 3 53.7GB 107GB 53.7GB primary ext4 type=83 2 107GB 500GB 393GB primary ext4 type=83 Model: BUFFALO External HDD (scsi) Disk /dev/sdb: 2000GB Sector size (logical/physical): 512B/512B Partition Table: msdos Disk Flags: Number Start End Size Type File system Flags 1 32.8kB 2000GB 2000GB primary ntfs type=07 Model: ATA WDC WD5000AACS-0 (scsi) Disk /dev/sdc: 500GB Sector size (logical/physical): 512B/512B Partition Table: msdos Disk Flags: Number Start End Size Type File system Flags 1 32.3kB 53.7GB 53.7GB primary ext4 boot, type=83 2 53.7GB 161GB 107GB primary ext4 type=83 4 215GB 500GB 285GB extended lba, type=0f 7 215GB 268GB 53.7GB logical ext4 type=83 5 268GB 446GB 177GB logical reiserfs type=83 6 446GB 500GB 54.5GB logical ext3 type=83 Model: ATA ST3320620AS (scsi) Disk /dev/sdd: 320GB Sector size (logical/physical): 512B/512B Partition Table: msdos Disk Flags: Number Start End Size Type File system Flags 1 32.3kB 67.3GB 67.3GB primary ext4 type=83 2 67.3GB 277GB 210GB primary ext3 type=83 3 277GB 320GB 43.0GB primary ext4 boot, type=83 Model: Linux device-mapper (thin-pool) (dm) Disk /dev/mapper/docker-8:3-3148282-pool: 107GB Sector size (logical/physical): 512B/512B Partition Table: loop Disk Flags: Number Start End Size File system Flags 1 0.00B 107GB 107GB ext4 -- "The Truth is out there" - Spooky -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 10/09/2016 04:21 PM, Marc Chamberlin wrote:
What filesystem?
Oh wow! Interesting question! I just took a look, and well this system has been an evolution since the days of SuSE 10.0. So apparently I have multiple file system types in play. I guess the best way to answer your question is to simply show the output from parted -l
Could you give the output of "df -T -a" please Thanks. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 10/9/2016 4:48 PM, Anton Aylward wrote:
On 10/09/2016 04:21 PM, Marc Chamberlin wrote:
What filesystem?
Oh wow! Interesting question! I just took a look, and well this system has been an evolution since the days of SuSE 10.0. So apparently I have multiple file system types in play. I guess the best way to answer your question is to simply show the output from parted -l Could you give the output of "df -T -a" please
Thanks.
Hi Anton - You are making me poke into corners I hadn't thought to look into! ;-) So here is the output of the df command, and yeah one disk, which is a removeable USB drive that I only use for storing backups (I use Bacula) is rather full. I am going to have to address that soon as well. But Bacula is scheduled to do backups of computers on my network, late at night, and if it were running, when I am working on this system during the day, I would know. When it is actively doing a backup, yes, it chews up a lot of CPU cycles. And even though /dev/sdb is pretty full, I think it is pretty much at a steady state in that Bacula recycles and purges storage from old backups as it goes along. FYI. I have tried stopping it, as well as other services I am running (Apache2, Apache Tomcat, Apache James, VSFTPD, Named, DHCP, etc.) to see if that would make a difference, but no it doesn't. (I will also say that I have noticed that the Java base servers (Apache2, Tomcat, and James) can report up to around 10% CPU usage each from time to time, but not on a steady basis. I expect that occurs when they get busy handling some request.) Marc... df -T -a Filesystem Type 1K-blocks Used Available Use% Mounted on sysfs sysfs 0 0 0 - /sys proc proc 0 0 0 - /proc devtmpfs devtmpfs 1015292 8 1015284 1% /dev securityfs securityfs 0 0 0 - /sys/kernel/security tmpfs tmpfs 1023372 144 1023228 1% /dev/shm devpts devpts 0 0 0 - /dev/pts tmpfs tmpfs 1023372 2348 1021024 1% /run tmpfs tmpfs 1023372 0 1023372 0% /sys/fs/cgroup cgroup cgroup 0 0 0 - /sys/fs/cgroup/systemd pstore pstore 0 0 0 - /sys/fs/pstore cgroup cgroup 0 0 0 - /sys/fs/cgroup/cpuset cgroup cgroup 0 0 0 - /sys/fs/cgroup/cpu,cpuacct cgroup cgroup 0 0 0 - /sys/fs/cgroup/blkio cgroup cgroup 0 0 0 - /sys/fs/cgroup/memory cgroup cgroup 0 0 0 - /sys/fs/cgroup/devices cgroup cgroup 0 0 0 - /sys/fs/cgroup/freezer cgroup cgroup 0 0 0 - /sys/fs/cgroup/net_cls,net_prio cgroup cgroup 0 0 0 - /sys/fs/cgroup/perf_event cgroup cgroup 0 0 0 - /sys/fs/cgroup/hugetlb /dev/sdd3 ext4 41246008 24637004 15541692 62% / systemd-1 - - - - - /proc/sys/fs/binfmt_misc mqueue mqueue 0 0 0 - /dev/mqueue hugetlbfs hugetlbfs 0 0 0 - /dev/hugepages debugfs debugfs 0 0 0 - /sys/kernel/debug /dev/sdd3 ext4 41246008 24637004 15541692 62% /slash /dev/sda2 ext4 377379368 246929440 111257048 69% /data1 /dev/sda2 ext4 377379368 246929440 111257048 69% /slash/data1 /dev/sda3 ext4 51473020 6477836 43930268 13% /var /dev/sda3 ext4 51473020 6477836 43930268 13% /slash/var /dev/sdc2 ext4 103085848 24928972 77091852 25% /srv /dev/sdc2 ext4 103085848 24928972 77091852 25% /slash/srv tmpfs tmpfs 1023372 2348 1021024 1% /var/run tmpfs tmpfs 1023372 2348 1021024 1% /slash/var/run fusectl fusectl 0 0 0 - /sys/fs/fuse/connections /dev/sdc7 ext4 51473020 1936700 46898600 4% /var12.3 /dev/sdc7 ext4 51473020 1936700 46898600 4% /slash/var12.3 /dev/sdc6 ext3 52233148 2011060 47562128 5% /james /dev/sdc6 ext3 52233148 2011060 47562128 5% /slash/james /dev/sdc5 reiserfs 173014680 79079704 93934976 46% /data /dev/sdc5 reiserfs 173014680 79079704 93934976 46% /slash/data /dev/sdc1 ext4 51481976 32999264 17417608 66% /SuSE11.4 /dev/sdc1 ext4 51481976 32999264 17417608 66% /slash/SuSE11.4 /dev/sdd1 ext4 64571180 19157016 44346028 31% /SuSE12.3 /dev/sdd1 ext4 64571180 19157016 44346028 31% /slash/SuSE12.3 /dev/sdb1 fuseblk 1953514524 1809690796 143823728 93% /media/buffalo /dev/sdb1 fuseblk 1953514524 1809690796 143823728 93% /slash/media/buffalo /dev/sdd2 ext3 201446256 86497132 104709204 46% /home /dev/sdd2 ext3 201446256 86497132 104709204 46% /slash/home proc proc 0 0 0 - /var/lib/named/proc proc proc 0 0 0 - /slash/var/lib/named/proc /dev/sda3 ext4 51473020 6477836 43930268 13% /var/lib/docker/devicemapper /dev/sda3 ext4 51473020 6477836 43930268 13% /slash/var/lib/docker/devicemapper proc proc 0 0 0 - /var/lib/dhcp/proc proc proc 0 0 0 - /slash/var/lib/dhcp/proc tracefs tracefs 0 0 0 - /sys/kernel/debug/tracing gvfsd-fuse fuse.gvfsd-fuse 0 0 0 - /run/user/1000/gvfs gvfsd-fuse fuse.gvfsd-fuse 0 0 0 - /slash/var/run/user/1000/gvfs gvfsd-fuse fuse.gvfsd-fuse 0 0 0 - /var/run/user/1000/gvfs binfmt_misc binfmt_misc 0 0 0 - /proc/sys/fs/binfmt_misc -- "The Truth is out there" - Spooky -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-10-09 22:21, Marc Chamberlin wrote:
On 10/9/2016 10:42 AM, Carlos E. R. wrote:
Of interest also is doing file moves or deletes from the command line, i.e. using the mv, rm, and rmdir commands does not cause this high CPU usage to occur and those commands complete in a reasonable amount of time. What filesystem?
Oh wow! Interesting question! I just took a look, and well this system has been an evolution since the days of SuSE 10.0. So apparently I have multiple file system types in play. I guess the best way to answer your question is to simply show the output from parted -l
Well, I see you are not using btrfs, and no filesystem is close to 90% full. It is not the problem. Maybe a corrupted filesystem, so check them all. To show disks, I like the output of this command: lsblk --output NAME,KNAME,RA,RM,RO,SIZE,TYPE,FSTYPE,LABEL,PARTLABEL,MOUNTPOINT,UUID,PARTUUID,WWN,MODEL,ALIGNMENT
I have never taken the time to figure out what file system type is best or what are the pros and cons of each. So usually just gone with defaults when installing a new OS. I wonder if I should normalize all these, if so which file system to choose, and how would one go about doing so. Could this be the issue, having multiple file system types mounted at the same time is what is causing my system to slow down?
No, I also have several types in use. I would organize in a tree, so as to not have many things hanging from the root, but that aesthetic and me. /other/SuSE11.4 /other/SuSE12.3 /other/other distro /data/something /data/another etc. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
On October 9, 2016 9:58:12 AM PDT, Marc Chamberlin <marc@marcchamberlin.com> wrote:
On 10/04/2016 11:21 PM, Marc Chamberlin wrote:
On 10/4/2016 6:04 PM, John Andersen wrote:
On 10/04/2016 05:29 PM, Marc Chamberlin wrote:
The mouse problems seem worth investigating. Maybe dig in the junk drawer for a totally different kind of mouse (serial, bluetooth, usb-wired) and see if
Thanks John for your suggestion. I tried a different mouse and no joy... Marc....
Remind me again what kernel you are using, and what version of Vbox? I tend to purge my list mail fairly aggressivly. Hi John - I am running Leap 42.1 (x86_64) I am not running VBox, (which I think what you are referring to is VirtualBox?) Interesting that you should ask though because I am playing around with Docker. However, I don't think my system slowdowns has anything to do with Docker as these slowdown issues preceded my installation of Docker. And these slowdowns occur regardless of whether Docker is running or not. (The only reason I am fooling around with Docker, FYI, is so I can get the latest version of the Apache James email server up and running, something
On 10/5/2016 12:37 AM, John Andersen wrote: that changes anything. that OpenSuSE does not directly
support.)
Marc....
So that was originally kernel 4.1. Are you still on that or have you upgraded?
It might be worth trying to boot with iomem=relaxed
added to the kernel command line.
It might not do anything but slowness has been detected in several distros as soon as kernel updates were applied due to tighter memory security.
Thanks John for the thought, I tried adding the iomem=relaxed option to
the kernel options using the Yast Boot Loader tool, rebooted, and still
no joy...
In my fooling around with this issue, I have come across something that
definitely triggers high CPU usage, and could be a clue. Although I cannot definitively say this is the smoking gun because I see high system CPU usage going on regardless of whether I do this or not -
There is something very wrong going on when I use Dolphin to move stuff
to the trash "Recycle Bin" (or sometimes it appears when I use Dolphin to simply move files from one location to another, i.e. drag and drop) and/or when I tell Dolphin to empty the trash. Doing so kicks off the 100% CPU usage and I hear the processor cooling fans kick into high speed. These tasks can take hours or even days to complete (if ever, some are still going on for over a week now. When I reboot the system, these tasks are not initially restarted, but if I do something like open the Recycle Bin to see if the files are still in it (after I told Dolphin to empty the trash) then these processes again kick off and the
CPU usage goes up to 100%. The two (or more) spawned processes of interest are file.so and trash.so. However, unlike my original report and observations, these processes do report very high CPU usage also in
the Process Table tab of KSystemGuard, top, atop, and htop so I am not 100% certain that this is related to the issue I reported - about something hidden is causing high CPU usage.
Of interest also is doing file moves or deletes from the command line, i.e. using the mv, rm, and rmdir commands does not cause this high CPU usage to occur and those commands complete in a reasonable amount of time.
Marc...
Still sounds like a disk hardware problem to me. Trash is limited to some percentage of disk , and there shouldn't be any reason that delete operation would take that much time. But I suppose it's still possible a horribly corrupted file system could be at fault, but if you never see errors that seems unlikely. Is it still slow when you unplug / shut down your network connections (physically unplug the cat5) ? -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 10/9/2016 11:18 AM, John Andersen wrote:
Thanks John for the thought, I tried adding the iomem=relaxed option to
the kernel options using the Yast Boot Loader tool, rebooted, and still
no joy...
In my fooling around with this issue, I have come across something that
definitely triggers high CPU usage, and could be a clue. Although I cannot definitively say this is the smoking gun because I see high system CPU usage going on regardless of whether I do this or not -
There is something very wrong going on when I use Dolphin to move stuff
to the trash "Recycle Bin" (or sometimes it appears when I use Dolphin to simply move files from one location to another, i.e. drag and drop) and/or when I tell Dolphin to empty the trash. Doing so kicks off the 100% CPU usage and I hear the processor cooling fans kick into high speed. These tasks can take hours or even days to complete (if ever, some are still going on for over a week now. When I reboot the system, these tasks are not initially restarted, but if I do something like open the Recycle Bin to see if the files are still in it (after I told Dolphin to empty the trash) then these processes again kick off and the
CPU usage goes up to 100%. The two (or more) spawned processes of interest are file.so and trash.so. However, unlike my original report and observations, these processes do report very high CPU usage also in
the Process Table tab of KSystemGuard, top, atop, and htop so I am not 100% certain that this is related to the issue I reported - about something hidden is causing high CPU usage.
Of interest also is doing file moves or deletes from the command line, i.e. using the mv, rm, and rmdir commands does not cause this high CPU usage to occur and those commands complete in a reasonable amount of time.
Marc...
Still sounds like a disk hardware problem to me. Trash is limited to some percentage of disk , and there shouldn't be any reason that delete operation would take that much time.
But I suppose it's still possible a horribly corrupted file system could be at fault, but if you never see errors that seems unlikely.
Is it still slow when you unplug / shut down your network connections (physically unplug the cat5) ?
Yeah John, I am kinda thinking you are right, something fishy with the file system or disk drives. Carlo's question seems to imply he is also thinking in that direction. I tried to dd clone the one drive that has reported errors and that failed apparently with a lot of disk read errors. So I need to figure out a way to copy everything from that drive to a new one and perhaps change the file system type of the partitions to a better choice. So first question I have is what file system should I choose and if I want to copy everything from the old drive with a partition using a file system of one type, to a new drive with a partition using a different type of a file system, how best to do so? Doesn't look like dd would be a good choice of a tool to use.... Also, I am going to want to preserve the MBR in the process. Thanks again for your thoughts and assistance... Marc... -- "The Truth is out there" - Spooky -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 10/09/2016 04:44 PM, Marc Chamberlin wrote:
So first question I have is what file system should I choose
For a variety of reasons I've now standardized on ReiserFS. There has been discussion on this back and forth, but my main motivation is that I think any FS that you have to pre-allocate a fixed number of inodes when you run mkfs isn't any better than the old V6 file systems of the 1970s except for things like speed. pre-provisioning is archaic. We don't do it with memory, why should we do it with disk. ReiserFS, BtrFS and XFS are all B-tree based that don't have this provisioning problem. You CAN'T run out of inodes before data or data before inodes. That ext4 has b-tree internally but still requires this fixed pre-provisioning strikes me as a serious shortcoming of an otherwise excellent file system.
and if I want to copy everything from the old drive with a partition using a file system of one type, to a new drive with a partition using a different type of a file system, how best to do so? Doesn't look like dd would be a good choice of a tool to use....
There really is no reason not use use rsync when copying a file system. Its an outstanding too and very versatile. You can. for example, sign up for a free or free for a month network store, rsync to that, then rsync back later. Securely!
Also, I am going to want to preserve the MBR in the process.
Ahm, I wouldn't do that if I were you. Its better to regenerate the MBR to match the new drive and its characteristics. Put it where you want it on the new drive and make sure it links to your bootloader (grub, grub2, lilo) and lists the version(s) of the OS you have on the new drive. Better to be safe than sorry! Yast is your Friend here :-) There are actually a number of situations where you may need to 'repair' the MBR. The one most discussed here is when it has been 'corrupted' by an installation of Windows. https://en.opensuse.org/SDB:Repair_MBR_after_Windows_install -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-10-09 22:44, Marc Chamberlin wrote:
On 10/9/2016 11:18 AM, John Andersen wrote:
Yes, bad sectors cause large slowdowns, but not high cpu. It should be wait states.
Yeah John, I am kinda thinking you are right, something fishy with the file system or disk drives. Carlo's question seems to imply he is also thinking in that direction. I tried to dd clone the one drive that has reported errors and that failed apparently with a lot of disk read errors. So I need to figure out a way to copy everything from that drive to a new one and perhaps change the file system type of the partitions to a better choice. So first question I have is what file system should I choose and if I want to copy everything from the old drive with a partition using a file system of one type, to a new drive with a partition using a different type of a file system, how best to do so? Doesn't look like dd would be a good choice of a tool to use....
If you want to image, don't use dd, use dd_rescue or ddrescue (they are different but achieve the same thing). For data storage I prefer xfs. I like reiserfs a lot, but I limit its total size. All the reiserfs partitions share the same cpu thread, reiserfs does not scale well. If you want to copy files, use rsync.
Also, I am going to want to preserve the MBR in the process.
The program or the partition layout? Or both? -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
On 10/10/2016 07:02 AM, Carlos E. R. wrote:
I like reiserfs a lot, but I limit its total size. All the reiserfs partitions share the same cpu thread, reiserfs does not scale well.
Yes that is going to be critical on a heavily multi-user system, each user having thread sunder a ReiserFS home or having their own ReiserFS partition. But for a single user where the SYSTEM is on BtrFS or ext4 having a single operation on a ReiserFS isn't a problem. In may case I have all my Photography there, specifically to get around the inode pre-allocation problem of ext4. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-10-10 14:02, Anton Aylward wrote:
On 10/10/2016 07:02 AM, Carlos E. R. wrote:
I like reiserfs a lot, but I limit its total size. All the reiserfs partitions share the same cpu thread, reiserfs does not scale well.
Yes that is going to be critical on a heavily multi-user system, each user having thread sunder a ReiserFS home or having their own ReiserFS partition.
But for a single user where the SYSTEM is on BtrFS or ext4 having a single operation on a ReiserFS isn't a problem. In may case I have all my Photography there, specifically to get around the inode pre-allocation problem of ext4.
I use reiserfs on partitions where I expect many small files. I have several partitions, and you hit the single thread problem when copying between two RS partitions. For photos I use XFS, photos are relatively big, and there is no problem with inodes: if I remember correctly they are dynamic as well. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
On 10/10/2016 08:45 AM, Carlos E. R. wrote:
I use reiserfs on partitions where I expect many small files. I have several partitions, and you hit the single thread problem when copying between two RS partitions.
While true that's not a situation I enounter.
For photos I use XFS, photos are relatively big, and there is no problem with inodes: if I remember correctly they are dynamic as well.
Yes, both those assertions are correct. But I've had more trouble with the one time I used XFS for video, in the order of hundred of megs in size compared to my photos were RAW never seem to exceed, even RAW, more than 24Mb, than the many years and many partitions using ReiserFS .... one at a time use. *sigh* If I knew more about FS design internals and such I'd try contributing and resurrecting the Resier4 project. Hmm. I note the last update at https://sourceforge.net/projects/reiser4/ was on 2016-08-09. I wonder how 'alive' that project really is? -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 10/10/2016 5:45 AM, Carlos E. R. wrote:
On 2016-10-10 14:02, Anton Aylward wrote:
On 10/10/2016 07:02 AM, Carlos E. R. wrote:
I like reiserfs a lot, but I limit its total size. All the reiserfs partitions share the same cpu thread, reiserfs does not scale well. Yes that is going to be critical on a heavily multi-user system, each user having thread sunder a ReiserFS home or having their own ReiserFS partition.
But for a single user where the SYSTEM is on BtrFS or ext4 having a single operation on a ReiserFS isn't a problem. In may case I have all my Photography there, specifically to get around the inode pre-allocation problem of ext4. I use reiserfs on partitions where I expect many small files. I have several partitions, and you hit the single thread problem when copying between two RS partitions.
For photos I use XFS, photos are relatively big, and there is no problem with inodes: if I remember correctly they are dynamic as well.
Thanks Anton, Carlos for your thoughts on file systems. I have used Yast to partition a new drive and laid it out in the same way as the drive that was showing some signs of failing. I noted that for partitions on which one wants to install an OS, it suggests using BtrFS and for data is suggests using XFS, so that is what I chose. (2 primary partitions with BtrFS for future usage, and 4 partitions with XFS for everything else.) Then I used rsync (nice tool BTW, I like the fact that it is easy to preserve all the file attributes as well, and that it is really fast!) to copy all the files from each of the partitions on the old drive to the new one. I am not sure I want to remove the old drive from my system yet as I like to keep the option to drop back to a previous OS in the event that something else fails, and I don't think my older OS's will understand these new file systems that I am now using. So I will just remove the mount points for those old partitions from my openSuSE42.1. Of note also, I noticed that Yast's Partitioner does not provide the option to create Reiser filesystems. On the downside however, I have not seen any improvement in my heavy CPU usage problem yet. :-( Too tired to do anything more tonight but will do some more testing tomorrow and see if using Dolphin to copy/move/delete files on the new drive still drives the CPU usage up to 100%... Marc.... -- "The Truth is out there" - Spooky -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-10-11 03:57, Marc Chamberlin wrote:
On 10/10/2016 5:45 AM, Carlos E. R. wrote:
Thanks Anton, Carlos for your thoughts on file systems. I have used Yast to partition a new drive and laid it out in the same way as the drive that was showing some signs of failing. I noted that for partitions on which one wants to install an OS, it suggests using BtrFS and for data is suggests using XFS, so that is what I chose. (2 primary partitions with BtrFS for future usage, and 4 partitions with XFS for everything else.)
IMO, you should not format as btrfs till the moment of installation, because the installation prepares it in a particular way, and a different version may do it differently. Also you have to remember that btrfs partitions have to be much bigger than other types. IMO, at least a hundred gigs.
Then I used rsync (nice tool BTW, I like the fact that it is easy to preserve all the file attributes as well, and that it is really fast!) to copy all the files from each of the partitions on the old drive to the new one. I am not sure I want to remove the old drive from my system yet as I like to keep the option to drop back to a previous OS in the event that something else fails, and I don't think my older OS's will understand these new file systems that I am now using. So I will just remove the mount points for those old partitions from my openSuSE42.1.
Of note also, I noticed that Yast's Partitioner does not provide the option to create Reiser filesystems.
True.
On the downside however, I have not seen any improvement in my heavy CPU usage problem yet. :-( Too tired to do anything more tonight but will do some more testing tomorrow and see if using Dolphin to copy/move/delete files on the new drive still drives the CPU usage up to 100%...
Maybe disconnect the old drive temporarily, for verification. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
On October 11, 2016 6:00:20 AM PDT, "Carlos E. R." <robin.listas@telefonica.net> wrote:
On 2016-10-11 03:57, Marc Chamberlin wrote:
On 10/10/2016 5:45 AM, Carlos E. R. wrote:
Thanks Anton, Carlos for your thoughts on file systems. I have used Yast to partition a new drive and laid it out in the same way as the drive that was showing some signs of failing. I noted that for partitions on which one wants to install an OS, it suggests using BtrFS and for data is suggests using XFS, so that is what I chose. (2 primary partitions with BtrFS for future usage, and 4 partitions with XFS for everything else.)
IMO, you should not format as btrfs till the moment of installation, because the installation prepares it in a particular way, and a different version may do it differently. Also you have to remember that btrfs partitions have to be much bigger than other types. IMO, at least a hundred gigs.
Then I used rsync (nice tool BTW, I like the fact that it is easy to preserve all the file attributes as well, and that it is really fast!) to copy all the files from each of the partitions on the old drive to the new one. I am not sure I want to remove the old drive from my system yet as I like to keep the option to drop back to a previous OS in the event that something else fails, and I don't think my older OS's will understand these new file systems that I am now using. So I will just remove the mount points for those old partitions from my openSuSE42.1.
Of note also, I noticed that Yast's Partitioner does not provide the option to create Reiser filesystems.
True.
On the downside however, I have not seen any improvement in my heavy
CPU
usage problem yet. :-( Too tired to do anything more tonight but will do some more testing tomorrow and see if using Dolphin to copy/move/delete files on the new drive still drives the CPU usage up to 100%...
Maybe disconnect the old drive temporarily, for verification.
IMHO you should not use BTRFS Period. Offers nothing about need and risks everything you put on it. I Lost data twice, requiring reinstall both times. -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 10/11/2016 12:33 PM, John Andersen wrote:
IMHO you should not use BTRFS Period. Offers nothing about need and risks everything you put on it.
I Lost data twice, requiring reinstall both times.
When was that, which version, which version kernel? I have no great love for BtrFS and despise its attitude of trying to subsume everything, "all your drives belong to me" approach to the "One File System to rule them all". When it first came out I too lost stuff. But one the design was stabilized, later kernels, well past the version on the 12x and 13.x distribution kernels, it hasn't failed me. I've given up on many of its features. That it requires bigger partitions, that its attitude towards snapshot management need more sysadmin work, the way subvolumes are handled, all annoy me. For the OS, where there is little growth, little further than updates to what already exists, so provisioning is predictable, the ext4 does a damn good job. Ext4 is a fine, an excellent file system for 'rotating rust'. Back in the days of UNIX V6, the 1970s, a mkfs of a disk drive involved pre-provisioning. The ration of inodes to data space was fixed. The layout was simplistic. Even the Berkeley Fast File System from the early 1980s, and compared to the V6/V7 file system it was fast, still had this ratio fixed at mkfs time. Today, the modern, no even the not so modern but still maintained file systems like XFS or 1993 vintage, B-tree based, avoid this problem. Ext4 is b-tree based but still has this archaism. I'm just sorry that Reiser4 seems to require a custom build of the kernel rather than being available as a kernel module. Correct me if I'm wrong about that, but I haven't been able to find it for that mode. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-10-11 19:11, Anton Aylward wrote:
I'm just sorry that Reiser4 seems to require a custom build of the kernel rather than being available as a kernel module. Correct me if I'm wrong about that, but I haven't been able to find it for that mode.
I see reiserfs and reiser4progs in filesystems repo :-? -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
On 10/12/2016 08:40 AM, Carlos E. R. wrote:
On 2016-10-11 19:11, Anton Aylward wrote:
I'm just sorry that Reiser4 seems to require a custom build of the kernel rather than being available as a kernel module. Correct me if I'm wrong about that, but I haven't been able to find it for that mode.
I see reiserfs and reiser4progs in filesystems repo :-?
Yes, I looked at those. The 'reiserfs' is the basic one that comes included in the distribution. No additional value. 'Reiser4progs' has the mkfs and fsck and performance monitors. It has nothing that allows the kernel or even a LUKS to make use of a partition or LV that is then mkfs'd with that tool. Most definitely no kernel module for Reiser4. BTDT, no joy. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Marc Chamberlin wrote:
Of note also, I noticed that Yast's Partitioner does not provide the option to create Reiser filesystems.
Nor JFS, but both are easily created (from the command line) anyway. -- Per Jessen, Zürich (6.6°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 10/09/2016 02:18 PM, John Andersen wrote:
But I suppose it's still possible a horribly corrupted file system could be at fault, but if you never see errors that seems unlikely.
"Horribly corrupted"? Surely boot runs a FSCK occasionally? You can force fsck at boot time by passing fsck.mode=force, as a kernel parameter. This will check *every* filesystem you have on the machine. If you are running systemd, as we all are now, with up-to-date systems, then setting the fsck pass number in /etc/fastab (that's the sixth column) greater than 0 will force a check at boot every time. Of course you can force a FSCK on the next boot or even run the FSCK manually while in maintenance mode. Marc, are you sure some FS isn't 'almost full'? That can cause thrashing of the allocation algorithms or, with b-tree file systems (such as the installation defaults of BtrFS and XFS or ext4) relocation thrashing. BTDT. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (5)
-
Anton Aylward -
Carlos E. R. -
John Andersen -
Marc Chamberlin -
Per Jessen