[opensuse-support][TW] hang trying to copy from NTFS filesystem to another NTFS filesystem using MC
Is this a known MC or ntfs-3g limitation? First I tried copying from an NTFS USB stick files that originally came from a .tgz file created from an NTFS filesystem. I tried using MC in the Knoppix 8.6 GUI from that stick to an installed SSD's Windows system partition. MC hung around 70% complete, leaving no clue in dmesg. Next try was from the original Windows 7 system partition on the old HD to the freshly installed Windows 7 system partition on a brand new SSD, which froze MC 4.8.23 on openSUSE Tumbleweed at 88% complete, files processed 9533/10387, and again no apparent clue in dmesg. I tried a third time, also using TW, but not selecting any symlinks to copy, and the hang happened at 9531/10370 @Target /mnt/Users/username/AppData/LocalLow/Microsoft/CryptnetUrlCache/Content/ prwxrwxrwx 2 root root 0 Oct 8 19:26 B398B80134F72209547439DB21AB308D_CCF564BE5A3C924B17DDEBDEB5236E1 Is the p in prwxrwxrwx some kind of indicator that failure can be expected absent some kind of decryption environment? There is inexplicably no such file on the source, so I'm at a loss what's happening, and not. The journal may have clues, but I don't recognize any as such: # journalctl -b | egrep -i 'ntfs|sdb|sdc|fuse' Oct 08 19:15:42 gx780 kernel: sd 5:0:0:0: [sdb] 500118192 512-byte logical blocks: (256 GB/238 GiB) Oct 08 19:15:42 gx780 kernel: sd 5:0:0:0: [sdb] Write Protect is off Oct 08 19:15:42 gx780 kernel: sd 5:0:0:0: [sdb] Mode Sense: 00 3a 00 00 Oct 08 19:15:42 gx780 kernel: sd 5:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Oct 08 19:15:42 gx780 kernel: sdb: sdb1 sdb2 Oct 08 19:15:42 gx780 kernel: sd 5:0:0:0: [sdb] Attached SCSI removable disk Oct 08 19:15:56 gx780 kernel: fuse init (API version 7.29) Oct 08 19:15:55 gx780 ntfs-3g[491]: Version 2017.3.23 external FUSE 29 Oct 08 19:15:55 gx780 ntfs-3g[491]: Mounted /dev/sda15 (Read-Write, label "WinDATA", NTFS 3.1) Oct 08 19:15:55 gx780 ntfs-3g[491]: Cmdline options: rw,noatime,noexec,nosuid,nodev,gid=1000,fmask=0111,dmask=0000,locale=en_US.UTF-8,users Oct 08 19:15:55 gx780 ntfs-3g[491]: Mount options: rw,noexec,nosuid,nodev,users,allow_other,nonempty,noatime,default_permissions,fsname=/dev/sda15,blkdev,blksize=4096 Oct 08 19:15:55 gx780 ntfs-3g[491]: Global ownership and permissions enforced, configuration type 7 Oct 08 19:15:55 gx780 ntfs-3g[491]: Warning : using problematic uid==0 and gid!=0 Oct 08 19:15:56 gx780 ntfs-3g[494]: Version 2017.3.23 external FUSE 29 Oct 08 19:15:56 gx780 ntfs-3g[494]: Mounted /dev/sda6 (Read-Write, label "WinSYS", NTFS 3.1) Oct 08 19:15:56 gx780 ntfs-3g[494]: Cmdline options: rw,noatime,noexec,nosuid,nodev,gid=1000,fmask=0111,dmask=0000,locale=en_US.UTF-8,users Oct 08 19:15:56 gx780 ntfs-3g[494]: Mount options: rw,noexec,nosuid,nodev,users,allow_other,nonempty,noatime,default_permissions,fsname=/dev/sda6,blkdev,blksize=4096 Oct 08 19:15:56 gx780 ntfs-3g[494]: Global ownership and permissions enforced, configuration type 7 Oct 08 19:15:56 gx780 ntfs-3g[494]: Warning : using problematic uid==0 and gid!=0 Oct 08 19:15:57 gx780 systemd[1]: Mounting FUSE Control File System... Oct 08 19:15:57 gx780 systemd[1]: Mounted FUSE Control File System. Oct 08 19:16:06 gx780 smartd[670]: Device: /dev/sdb, type changed from 'scsi' to 'sat' Oct 08 19:16:06 gx780 smartd[670]: Device: /dev/sdb [SAT], opened Oct 08 19:16:06 gx780 smartd[670]: Device: /dev/sdb [SAT], APS-SL3N-256, S/N:SF05C32442WL, WWN:0-000000-000000000, FW:SBFM11.2, 256 GB Oct 08 19:16:06 gx780 smartd[670]: Device: /dev/sdb [SAT], not found in smartd database. Oct 08 19:16:06 gx780 smartd[670]: Device: /dev/sdb [SAT], can't monitor Current_Pending_Sector count - no Attribute 197 Oct 08 19:16:06 gx780 smartd[670]: Device: /dev/sdb [SAT], can't monitor Offline_Uncorrectable count - no Attribute 198 Oct 08 19:16:06 gx780 smartd[670]: Device: /dev/sdb [SAT], is SMART capable. Adding to "monitor" list. Oct 08 19:16:07 gx780 smartd[670]: Device: /dev/sdb [SAT], state written to /var/lib/smartmontools/smartd.APS_SL3N_256-SF05C32442WL.ata.state Oct 08 19:18:47 gx780 ntfs-3g[2487]: Version 2017.3.23 external FUSE 29 Oct 08 19:18:47 gx780 ntfs-3g[2487]: Mounted /dev/sdb2 (Read-Write, label "Win7SYS", NTFS 3.1) Oct 08 19:18:47 gx780 ntfs-3g[2487]: Cmdline options: rw Oct 08 19:18:47 gx780 ntfs-3g[2487]: Mount options: rw,allow_other,nonempty,relatime,fsname=/dev/sdb2,blkdev,blksize=4096 Oct 08 19:18:47 gx780 ntfs-3g[2487]: Ownership and permissions disabled, configuration type 7 Oct 08 19:19:30 gx780 kernel: sd 8:0:0:0: [sdc] 312581808 512-byte logical blocks: (160 GB/149 GiB) Oct 08 19:19:30 gx780 kernel: sd 8:0:0:0: [sdc] 4096-byte physical blocks Oct 08 19:19:30 gx780 kernel: sd 8:0:0:0: [sdc] Write Protect is off Oct 08 19:19:30 gx780 kernel: sd 8:0:0:0: [sdc] Mode Sense: 5f 00 00 08 Oct 08 19:19:30 gx780 kernel: sd 8:0:0:0: [sdc] Disabling FUA Oct 08 19:19:30 gx780 kernel: sd 8:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Oct 08 19:19:30 gx780 kernel: sd 8:0:0:0: [sdc] Optimal transfer size 33553920 bytes not a multiple of physical block size (4096 bytes) Oct 08 19:19:30 gx780 kernel: sdc: sdc1 sdc2 Oct 08 19:19:30 gx780 kernel: sd 8:0:0:0: [sdc] Attached SCSI disk Oct 08 19:19:43 gx780 kdeinit5[1172]: DeviceServiceAction::execute: "/org/freedesktop/UDisks2/block_devices/sdb2" is not a StorageAccess device Oct 08 19:20:19 gx780 ntfs-3g[2612]: Version 2017.3.23 external FUSE 29 Oct 08 19:20:19 gx780 ntfs-3g[2612]: Mounted /dev/sdc2 (Read-Write, label "WinRUN", NTFS 3.1) Oct 08 19:20:19 gx780 ntfs-3g[2612]: Cmdline options: rw Oct 08 19:20:19 gx780 ntfs-3g[2612]: Mount options: rw,allow_other,nonempty,relatime,fsname=/dev/sdc2,blkdev,blksize=4096 Oct 08 19:20:19 gx780 ntfs-3g[2612]: Ownership and permissions disabled, configuration type 7 -- Evolution as taught in public schools is religion, not science. Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse-support+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-support+owner@opensuse.org
On 09/10/2019 03.53, Felix Miata wrote:
Is this a known MC or ntfs-3g limitation? First I tried copying from an NTFS USB stick files that originally came from a .tgz file created from an NTFS filesystem. I tried using MC in the Knoppix 8.6 GUI from that stick to an installed SSD's Windows system partition. MC hung around 70% complete, leaving no clue in dmesg. Next try was from the original Windows 7 system partition on the old HD to the freshly installed Windows 7 system partition on a brand new SSD, which froze MC 4.8.23 on openSUSE Tumbleweed at 88% complete, files processed 9533/10387, and again no apparent clue in dmesg. I tried a third time, also using TW, but not selecting any symlinks to copy, and the hang happened at 9531/10370 @Target /mnt/Users/username/AppData/LocalLow/Microsoft/CryptnetUrlCache/Content/ prwxrwxrwx 2 root root 0 Oct 8 19:26 B398B80134F72209547439DB21AB308D_CCF564BE5A3C924B17DDEBDEB5236E1
Is the p in prwxrwxrwx some kind of indicator that failure can be expected absent some kind of decryption environment? There is inexplicably no such file on the source, so I'm at a loss what's happening, and not.
ntfs develops new features now and then. About a year ago I became aware
of a new one that was not supported by news. Let's see if I can find it
(Reparse Points).
Found on nntp:Message-ID:
On 2018-07-28, Peter Pearson
wrote:
I've been happily backing up with rsync for many years, and haven't had this problem before, and have no idea why it started happening.
What should I do to keep permissions information from getting lost?
Any education on the larger picture (there's got to be a reason why there's a system that throws away permissions) would also be welcome.
Because Microsoft was not designed originally for multiuser and so did not think permissions were needed.
FAT32 has no notion of permissions. NTFS has a full suite of permissions. (SID, ACLs, ACE, domain stuff) It has such a large suite of features, nobody can explain all of it. Possibly only Jesper Johansson knows all of it. And if nobody can explain it, that's going to make verifying file system driver behavior a wee bit difficult. https://www.csoonline.com/article/2633083/security/our-windows-vista-securit... Web sites offering tutorials, usually punt before they complete the description. The topic is big enough to need a book. It doesn't mean the obscure features have value, but a complete knowledge of how it works, would mean better output from developers. Linux made the right call, by not implementing any of that for their NTFS driver. Since only reverse engineering was available to figure out how it worked, avoiding permissions and ownership when NTFS is mounted in Linux, makes the interworking practical and sufficient for end-users. And not likely to please any other level of the food chain. So if the NTFS permission model from Linux, looks about the same as FAT32, that's a conscious decision. Linux also does not interact with the USN Journal. And invalidating it is about all you can do, if you choose not to implement journaling on the NTFS file system. Fedora has fixed the Win10 MFTMIRR problem, by commenting out the check. Many other flavors haven't done anything about this yet. If you make an NTFS partition via Win10 Disk Management, Linux may refuse to mount the partition because the MFTMIRR isn't correct, and TestDisk can fix it. A run of CHKDSK from Windows 7 may also be able to put MFTMIRR back into a compatible state. Linux does not support Reparse Points. Nobody can support Reparse Points in any realistic way, since they are custom file system features without documentation. Reparse Points allow extending the file system feature set, without changing the file system version number. A separate metadata file keeps the information needed to handle any files stamped that way. Reparse Points are used to implement Junction Points (a kind of soft link). Even the Windows OS tools choose not to follow such links, and instead a warning is thrown when you hit one (icacls won't touch them). Windows 10 has added a new flavor of file system compression. Without changing the NTFS file system version number from 3.1. It's implemented as a Reparse Point. To allow modifying areas of a Windows C: drive using this feature, in Windows 10 do this as administrator, then take the drive to your Linux box and mount it. This doesn't clear the C: volume entirely of troublesome metadata, but removes certain portions from the Windows folder. As Administrator in Win10 compact /CompactOS:never and that will decompress some but not all, of the affected files. This should not impact data-only partitions, and this detail is only needed when hacking an NTFS C: (OS) partition. NTFS is in fact fully features -- it's too fully featured for its own good. You would swear someone was making these changes, just to make life difficult In Redmond, someone has an evil grin on their face... There may be more than one version of NTFS driver out there. Perhaps the Tuxera commercial one is better. I'm commenting on the one I get for free in a Linux distro, without doing any work. So if you see "I/O Error" in Linux, when attempting to change the filename on a certain file, that's not an I/O Error. That's an inability to handle the metadata on a new-compression format file. At least the MFTMIRR problem or the hibernated disk problem, you're likely to get a dialog explaining what the issue is. The "I/O Error" result comes with no explanation, and naturally, such an error will be scaring the shit out of somebody... (running off to the store, buying a new disk drive, and so on, all for nothing). Paul ...............++- -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
participants (2)
-
Carlos E. R.
-
Felix Miata