[opensuse] Strange behavior of external disk with encrypted and clear partitions
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, I'm using external USB disk with 2 partitions. One with NTFS used for exchange with Windows machines and another one has ReiserFS. Recently I've decided to replace ReiserFS partition with new one, encrypted. I did it using Yast partitioner (in oS12.1). Now I experience the following strange behavior: When I connect this disk when oS12.1 (64 bits) is running, non-encrypted partition is auto-mounted, as expected, and encrypted one is not. Now, I just want to copy some file to non-encrypted NTFS partition. Surprisingly the copying is terribly slow (estimated time for 500 MB file is 4 hours). I have oS 11.4 (32 bits) on the same computer I have not defined encrypted mount there. I reboot to 11.4, connect the same disk and voilà - the copy takes less then minute. if I unlock encrypted partition using Yast partitioner in 12.1 the copying is still crawling. I do not see any messages in system log at that time. Any advice how to debug it? What can slow down the write access (my impression is that reading is still OK, though I only transferred small files, so I can't say for sure)? Regards, - -- Mark -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJPyPnfAAoJEKWTiqSclQacBMMH/0QW/yrEZ+aYDBtRGgnz2Ilx UeJj9fJOx7mcrmsHFJkQnsIT/RW95ceLYe8wiPLRHpTyOJnuPBMDZhKbammEazCm gT4SVuSEmey2Ad2Ybes95jUQsfxCq+XK6KvMMzBEmeGL/yzsQT5lVEqiYCOIRPz+ 9b3j+SVLjoObSeylLSFbYD/m/xq4FuzLene2s2knBcbGZLNCwpeQ218aZsh/mrEN +SNeLv9njL1QKrGZKHIDU2s3zabjGDZzdKRgJ6YcsNQcUHSXlAkn5EMYM7/7FcFS AWNYlTCFkrdJQ/sF8KSxjOYYmN0xYMRYvsDTZBEHuRXMK9WPZRkqwDZdA519Bp0= =XBBF -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Am Freitag, 1. Juni 2012, 20:20:32 schrieb Mark Goldstein:
[...] Any advice how to debug it?
IMHO, the easiest thing to do is to rule out the NTFS filesystem by trying to copy more than 500MB to the encrypted partition (assuming it is not using NTFS). Then, to rule out the encrypted but non-mounted partition by replacing it with a non-NTFS partition or by using another disk with the same layout but a non-encrypted partition.
What can slow down the write access (my impression is that reading is still OK, though I only transferred small files, so I can't say for sure)?
There are some bug reports: * writing to usb flashdisk uses too much cpu and makes system unresponsive[0] * excessive swap out due to compaction/LRU aging[1] And at least two forum threads: *OpenSuse 12.1 freezes while copying data[2] * USB drives bad performance, low speed[3] BTW, only [3] seems to be specific to NTFS whil the other links describe system freezes resulting in slow write performance. HTH Jan [0] https://bugzilla.novell.com/show_bug.cgi?id=719416 [1] https://bugzilla.novell.com/show_bug.cgi?id=763399 [2] http://forums.opensuse.org/english/get-technical-help- here/applications/472088-opensuse-12-1-freezes-while-copying-data.html [3] http://forums.opensuse.org/english/get-technical-help- here/hardware/475403-usb-drives-bad-performance-low-speed.html -- God does not play dice. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 06/02/2012 12:58 PM, Jan Ritzerfeld wrote:
Am Freitag, 1. Juni 2012, 20:20:32 schrieb Mark Goldstein:
[...] Any advice how to debug it?
IMHO, the easiest thing to do is to rule out the NTFS filesystem by trying to copy more than 500MB to the encrypted partition (assuming it is not using NTFS). Then, to rule out the encrypted but non-mounted partition by replacing it with a non-NTFS partition or by using another disk with the same layout but a non-encrypted partition.
What can slow down the write access (my impression is that reading is still OK, though I only transferred small files, so I can't say for sure)?
There are some bug reports: * writing to usb flashdisk uses too much cpu and makes system unresponsive[0] * excessive swap out due to compaction/LRU aging[1] And at least two forum threads: *OpenSuse 12.1 freezes while copying data[2] * USB drives bad performance, low speed[3]
BTW, only [3] seems to be specific to NTFS whil the other links describe system freezes resulting in slow write performance.
Thank you Jan, I'll try to find some hints in the bugs and discussions you referred to, though from the first glance they look different. In the meantime I did a couple of additional experiments. First, I can confirm that read access is OK, as expected. Second, even if I mount encrypted partition (it has ReiserFS), the writing to non-encrypted one is still slow, while writing to encrypted partition itself is OK (20-30 MB/sec against 30-40 kB/sec). Third, I mounted encrypted partition in 11.4 and writing to non-encrypted NTFS partition was fast (30-60 MB/sec). Looks 12.1 specific. I can't be 100% sure, but I think I've copied large files to this NTFS partition under 12.1 before creating encrypted ReiserFS partition, and it was OK. Regards, - -- Mark -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJPygL2AAoJEKWTiqSclQacYF4IAK4vz63BNwvv3oJWvonqIzcG 2LHJCZ1rNqakUdHzyaAxzdnoiRbDpn7QdJaaJ9zKcb4g+E+bdUniJkAd4rQRDlH9 QxOFRyUOmWJXbixmdPx925ARL2WEgoFJ7rJuUU1BMmNiE84/zN2iGzq2q5/nZRux JdUaVqe4RhrDmWPBoVc85lrGPB9KrBql/MzeVAXZgMulfbPMZbFxYvszSolO48MB EXGw4g4kSuHvNPi/pDewtqZ/hAWszzUK27okWIF2YiaFgvtmtBBYZpLrtXY/U5jg FRPzcbvl0Rhti8ICBmy5ZCgM8Lz6pXhCJ0nKglOCAIxBbylGmMe5ApBnpLffQ2c= =iTjS -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Am Samstag, 2. Juni 2012, 15:11:34 schrieb Mark Goldstein:
[...] Thank you Jan, I'll try to find some hints in the bugs and discussions you referred to, though from the first glance they look different.
Yes they do. However, IMHO it is most important to reduce the complexity of your problem by ruling out the problem in these bugs and discussions.
In the meantime I did a couple of additional experiments. First, I can confirm that read access is OK, as expected.
Good.
Second, even if I mount encrypted partition (it has ReiserFS), the writing to non-encrypted one is still slow, while writing to encrypted partition itself is OK (20-30 MB/sec against 30-40 kB/sec).
You wrote similar data (wrt number of files and file size) to the encrypted partition?
Third, I mounted encrypted partition in 11.4 and writing to non-encrypted NTFS partition was fast (30-60 MB/sec). Looks 12.1 specific.
Thus, your general USB transfer speed is significally slower using 12.1.
I can't be 100% sure, but I think I've copied large files to this NTFS partition under 12.1 before creating encrypted ReiserFS partition, and it was OK.
I really cannot believe a partition that is not mounted would have any effect! However, is there any difference in the mount options actually used (between 11.4 and 12.1)? Gruß Jan -- Love stinks. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 06/02/2012 04:57 PM, Jan Ritzerfeld wrote:
Am Samstag, 2. Juni 2012, 15:11:34 schrieb Mark Goldstein:
[...] Thank you Jan, I'll try to find some hints in the bugs and discussions you referred to, though from the first glance they look different.
Yes they do. However, IMHO it is most important to reduce the complexity of your problem by ruling out the problem in these bugs and discussions. There are the references to fixes in 3.1.10-1.9.1 kernel that influence USB performance. I've checked and I have this same kernel. (desktop version).
Second, even if I mount encrypted partition (it has ReiserFS), the writing to non-encrypted one is still slow, while writing to encrypted partition itself is OK (20-30 MB/sec against 30-40 kB/sec).
You wrote similar data (wrt number of files and file size) to the encrypted partition? Yep, the same 500 Mb file.
Third, I mounted encrypted partition in 11.4 and writing to non-encrypted NTFS partition was fast (30-60 MB/sec). Looks 12.1 specific.
Thus, your general USB transfer speed is significally slower using 12.1. The speed reported by krusader was changing. Might be not very accurate. In 11.4 it looked like 60 MB/sec at the beginning, then it slowed down near the end of file. Another point is that 11.4 is 32-bit version while 12.1 is 64-bit. I would expect that since 12.1 can use more memory (the machine has 4 GB RAM) the buffering will be more effective.
I can't be 100% sure, but I think I've copied large files to this NTFS partition under 12.1 before creating encrypted ReiserFS partition, and it was OK.
I really cannot believe a partition that is not mounted would have any effect! However, is there any difference in the mount options actually used (between 11.4 and 12.1)? I'll check it next time I'll work with this disk. I did not adjust anything, just left what Yast did. My initial suspicion was that some kind of automount daemon is working in background trying to access encrypted partition. But since nothing changed after I unlocked and mounted the partition, it could be excluded.
Probably I have to try iotop and/or htop while copying. Regards, - -- Mark -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJPyiQ6AAoJEKWTiqSclQaco/IIAIVERfVEbj5w577LdXtbd2kM 781UoR76w96PDHV+Zw3qMwIOzHco88EA/JOdjnf47DysG0So0+H/EC6vkWcUurkP xC5mFeDToYw5svXCa2Zi30XXTYNgINo461u86ww2ihc9yvDLP+NNG9cENOgAZ1JQ sgZBN7Sb/tH+xhMQPCLt0oZlOj3baq0ti9jYZ2945TxH6wXAMNqtzT8q9xh/o/os GNa1Cd94n2QIf8glbDFiY4k2GRuYyFQW0zoGBjpyZQoFQM7FdPUkS9nnhvNfKM6U XgyqT5W+G0AlS2rRFi6mCtO4DrI/2C1BlXq24hlBBW+X/UbM02nUEEItcXG1hmQ= =Nz+n -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 06/02/2012 05:33 PM, Mark Goldstein wrote:
On 06/02/2012 04:57 PM, Jan Ritzerfeld wrote:
Am Samstag, 2. Juni 2012, 15:11:34 schrieb Mark Goldstein:
[...]
I really cannot believe a partition that is not mounted would have any effect! However, is there any difference in the mount options actually used (between 11.4 and 12.1)? I'll check it next time I'll work with this disk. I did not adjust anything, just left what Yast did. My initial suspicion was that some kind of automount daemon is working in background trying to access encrypted partition. But since nothing changed after I unlocked and mounted the partition, it could be excluded.
Probably I have to try iotop and/or htop while copying.
OK, some more information. This is how the mounts look in 12.1 /dev/sde1 on /media/FreeAgent type fuseblk (rw,nosuid,nodev,noatime,sync,user_id=0,group_id=0,de- fault_permissions,allow_other,blksize=4096) /dev/mapper/cr_sde2 on /mnt/bk type reiserfs (rw,relatime,_netdev) and this is how it looks in 11.4: /dev/sde1 on /media/FreeAgent type fuseblk (rw,nosuid,nodev,relatime,user_id=0,g roup_id=0,allow_other,blksize=4096) /dev/mapper/cr_sde2 on /media/disk type reiserfs (rw,nosuid,nodev,relatime) The /dev/sde1 is ntfs not encrypted partition, it is automounted in both systems. The difference seems to be that 12.1 has additional options sync and default_permissions and uses noatime, while 11.4 uses relatime. /dev/sde2 is encrypted partition. In 11.4 it is automounted (I have hald in 11.4) and in 12.1 I mount it from yast partitioner. Somehow the option _netdev is added, probably to delay mounting of this disk. Could "sync" option be the culprit? Does it mean sync after each write? Regards, - -- Mark -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJPykvrAAoJEKWTiqSclQac7agIAJEY0MfTLi14AE9NBNRjvpiz sv+BQiQCJvLzBtzBTym95JkZ69OY/L4yNFNQG/Y4BvwJ2ACz5HxcWO9lA2Vp/EKn ahXrzeqqY2UCz+Y5Z/UloWqzkwpfHPW2njfxjxuHjB0SAfsHB1zdAmIF9o1Tu9jE TAJyedDYuaH+jwN9lRvWwp8S1cqfk2R24hoI35Dbe6TY+3sbNUBpE5IGpeU/UnH7 MguHgtVVa2LW3kaQmMi32CHMpLuPamQboL/Y2vAj/3GaFpATTrKQ7BzvAnS4iAWd MhDyDA0NhYxh8UljL/jxdORirKWGUoOEA92L3DNIexQ/XMPiO/L0ojpGxi2OypI= =1Wf7 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2012-06-02 19:22, Mark Goldstein wrote:
Could "sync" option be the culprit? Does it mean sync after each write?
Yes. - -- Cheers / Saludos, Carlos E. R. (from 11.4 x86_64 "Celadon" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/KbbIACgkQIvFNjefEBxq9sgCgzdJrRgT//+jfjNRec2kiYEgG csoAnA1uJFRQB42ACBWk46KMynttx1dw =urtE -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Am Samstag, 2. Juni 2012, 20:22:51 schrieb Mark Goldstein:
[...] The /dev/sde1 is ntfs not encrypted partition, it is automounted in both systems. [...] Could "sync" option be the culprit? Does it mean sync after each write?
Yes, and the funny thing is: even if 11.4 did use it, it would have had no effect! The sync mount option was implemented in version 2011.1.15 and 11.4 uses 2010.10.2 AFAIK. So, how do you automount the NTFS partition? Gruß Jan -- Old programmers never die. They just branch to a new address. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2012-06-03 10:57, Jan Ritzerfeld wrote:
Am Samstag, 2. Juni 2012, 20:22:51 schrieb Mark Goldstein:
[...] The /dev/sde1 is ntfs not encrypted partition, it is automounted in both systems. [...] Could "sync" option be the culprit? Does it mean sync after each write?
Yes, and the funny thing is: even if 11.4 did use it, it would have had no effect! The sync mount option was implemented in version 2011.1.15 and 11.4 uses 2010.10.2 AFAIK.
Huh? I've seen the effect of the sync option more than 10 years ago, in floppies. It always worked. Not that long ago (9.x?), external media like usb was automounted as sync, and they were very slow. At the time the trick was to umount and then manually mount without sync. - -- Cheers / Saludos, Carlos E. R. (from 11.4 x86_64 "Celadon" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/LSiAACgkQIvFNjefEBxqToQCgspixF126Va6pm7I9zZYmizWm qDAAoM0Ge4gEAPLPNcLzwZZhiR6F1DPB =TtSP -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Am Sonntag, 3. Juni 2012, 13:27:28 schrieb Carlos E. R.:
[...] Huh?
I've seen the effect of the sync option more than 10 years ago, in floppies. It always worked. Not that long ago (9.x?), external media like usb was automounted as sync, and they were very slow. At the time the trick was to umount and then manually mount without sync.
Sorry, I did not make clear that this is specific to ntfs-3g! The sync mount option was implemented in ntfs-3g version 2011.1.15 and 11.4 uses 2010.10.2 AFAIK, see http://www.tuxera.com/community/release-history/ Gruß Jan -- We have met the enemy and he is us. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sun, Jun 3, 2012 at 2:27 PM, Carlos E. R. <robin.listas@telefonica.net> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2012-06-03 10:57, Jan Ritzerfeld wrote:
Am Samstag, 2. Juni 2012, 20:22:51 schrieb Mark Goldstein:
[...] The /dev/sde1 is ntfs not encrypted partition, it is automounted in both systems. [...] Could "sync" option be the culprit? Does it mean sync after each write?
Yes, and the funny thing is: even if 11.4 did use it, it would have had no effect! The sync mount option was implemented in version 2011.1.15 and 11.4 uses 2010.10.2 AFAIK.
Huh?
I've seen the effect of the sync option more than 10 years ago, in floppies. It always worked. Not that long ago (9.x?), external media like usb was automounted as sync, and they were very slow. At the time the trick was to umount and then manually mount without sync.
Jan is probably talking about NTFS-3G version? I'll check later when I'll reach the machine. I'd really like to know how this auto-mounting stuff works. It should be triggered by udev rules, I suppose, but I do not know how. Then the desktop auto-mounter should do the rest. I'm using KDE3, so I'll need to understand where and how it is defined there. It is possible that some update happened at the same time when I created encrypted partition, so this created false impression that the problem was triggered by this action. In any case, the workaround with remounting without sync option should work. I was going to try it first. Thanks & regards, -- Mark Goldstein -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Am Sonntag, 3. Juni 2012, 14:49:32 schrieb Mark Goldstein:
[...] I'm using KDE3, so I'll need to understand where and how it is defined there. It is possible that some update happened at the same time when I created encrypted partition, so this created false impression that the problem was triggered by this action.
Sorry, I do not know how KDE3 does its automounting today. However, there is a KDE3-specific list: opensuse-kde3. They will be very interested in such regressions.
[...] Thanks & regards,
You are welcome! Jan -- It is important for our friends to believe that we are unreservedly frank with them, and important to friendship that we are not. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sun, Jun 3, 2012 at 3:12 PM, Jan Ritzerfeld <suse@mailinglists.jan.ritzerfeld.org> wrote:
Am Sonntag, 3. Juni 2012, 14:49:32 schrieb Mark Goldstein:
[...] I'm using KDE3, so I'll need to understand where and how it is defined there. It is possible that some update happened at the same time when I created encrypted partition, so this created false impression that the problem was triggered by this action.
Sorry, I do not know how KDE3 does its automounting today. However, there is a KDE3-specific list: opensuse-kde3. They will be very interested in such regressions.
Sure, I was going to see how it works in KDE4 (also installed) and if it'll turn out to be KDE3 specific, I'll ask on kde3 list. One difference I'm aware of is that KDE3 in 11.4 uses hald, while in 12.1 it does not. Thanks again, -- Mark Goldstein -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sun, Jun 3, 2012 at 3:12 PM, Jan Ritzerfeld <suse@mailinglists.jan.ritzerfeld.org> wrote:
Am Sonntag, 3. Juni 2012, 14:49:32 schrieb Mark Goldstein:
[...] I'm using KDE3, so I'll need to understand where and how it is defined there. It is possible that some update happened at the same time when I created encrypted partition, so this created false impression that the problem was triggered by this action.
Sorry, I do not know how KDE3 does its automounting today. However,
a KDE3-specific list: opensuse-kde3. They will be very interested in such regressions.
Sure, I was going to see how it works in KDE4 (also installed) and if it'll turn out to be KDE3 specific, I'll ask on kde3 list. One difference I'm aware of is that KDE3 in 11.4 uses hald, while in 12.1 it does not.
Thanks again, Remounting of NTFS partition without sync option resolved the issue. I've also checked that under KDE4 auto-mounter does not insert sync
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 06/03/2012 03:22 PM, Mark Goldstein wrote: there is option, so this is KDE3 specific issue. So it has nothing to do with encrypted partition. Most probable some update at about the same time I've started using encrypted partition changed the behavior of auto-mounter. I'll follow up on kde3 list. Thanks again to Jan and Carlos. Regards, - -- Mark -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEbBAEBAgAGBQJPy5ORAAoJEKWTiqSclQac1rkH9Rr/I7mFC7l4z+HQm9i61p2N +Fs38zboUeUjjq7zL37Gi2o+UfwtWRtOT6t/n9I4bZ/9eMKul/uYLOxuYdFJJg67 JW1E1xq47HROUAS2bnpbJQrAqFaRkxo25JiFmaRaiGqHuK/n7hw/o8mTRPM6/npG 13YrD7/H73223dIGr6DkQ5o8s1ecBfVu1+voXtR+3nZtdJwxzWO2m2Q+TjeaHPvy kWof0kH2H9ZgH8Wb4cpk/PeBGYuuEchY7pUtqPr60GTkDkbcN9ZYnwwjUz2ym8uI XggjhTj9+a6YPX1rsIK4qO2i1Zs4n8s/nkroFl90szHcmHXzivQLPoRYO+bB5w== =RahY -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (3)
-
Carlos E. R. -
Jan Ritzerfeld -
Mark Goldstein