[Bug 836558] New: MidnightCommander: long file-names in tar archives truncated
https://bugzilla.novell.com/show_bug.cgi?id=836558 https://bugzilla.novell.com/show_bug.cgi?id=836558#c0 Summary: MidnightCommander: long file-names in tar archives truncated Classification: openSUSE Product: openSUSE Factory Version: 13.1 Milestone 4 Platform: Other OS/Version: Other Status: NEW Severity: Normal Priority: P5 - None Component: Other AssignedTo: bnc-team-screening@forge.provo.novell.com ReportedBy: d_werner@gmx.net QAContact: qa-bugs@suse.de Found By: --- Blocker: --- Created an attachment (id=554071) --> (http://bugzilla.novell.com/attachment.cgi?id=554071) a sample extfs for mc which correctly lists and copies out long file names User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0 tar archives support in MidnightCommander: tar archives containing long file-names or long path-name + file-name, e.g. resulting from nested sub-directories: file names are truncated. It seems the length of the path-name + file-name is limited to 100 characters, longer names are truncated. They are displayed truncated in the mc panel, and when the file (or its parent directory in the archive) is copied out of the archive with F5 it will have a truncated name. Therefore deeply nested tar archives can not safely be un-tarred with mc. There exist different formats of tar archives, "newer" formats (e.g. posix which is now the default) allow long file names (unlimited length). The problem happens in openSUSE 12.3 and also 13.1M4. Reproducible: Always Steps to Reproduce: mkdir testdir echo test > "testdir/testname_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789.txt" tar cJf testdir.tar.xz testdir start mc, enter testdir.tar.xz Actual Results: the displayed file name is limited to testname_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123 copy out (F5) will create the file with the truncated name. The tar archive itself is correct and tar xJf correctly restores the full file-names. Expected Results: the displayed file names or the file names created during copy out (F5) should be the correct file names recorded in the tar archive The attached VFS "untar" uses the installed tar (or star or gtar) to browse tar archives with mc and to copy out files, therefore avoiding the limitation of the mc internal tar list and copyout working rm, mkdir, rmdir partly working (only uncompressed tar archives) copyin works partly (only uncompressed tar archives, archive rootdir) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=836558 https://bugzilla.novell.com/show_bug.cgi?id=836558#c Ye Yuan <yyuan@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |yyuan@suse.com AssignedTo|bnc-team-screening@forge.pr |coolo@suse.com |ovo.novell.com | -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=836558 https://bugzilla.novell.com/show_bug.cgi?id=836558#c Stephan Kulow <coolo@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- AssignedTo|coolo@suse.com |david@dhaller.de -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=836558 https://bugzilla.novell.com/show_bug.cgi?id=836558#c1 Tomas Cech <tcech@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tcech@suse.com --- Comment #1 from Tomas Cech <tcech@suse.com> 2013-09-27 22:42:46 CEST --- Reproduced. It seems that filename size limit is hardcoded in src/vfs/tar/tar.c -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=836558 https://bugzilla.novell.com/show_bug.cgi?id=836558#c2 --- Comment #2 from David Haller <david@dhaller.de> 2013-09-28 11:28:09 CEST --- Thomas: yes. I said so some time ago ... *dig* in ah, yes: http://lists.opensuse.org/opensuse-factory/2013-08/msg00464.html and esp. http://lists.opensuse.org/opensuse-factory/2013-08/msg00466.html http://lists.opensuse.org/opensuse-factory/2013-08/msg00468.html See also https://www.midnight-commander.org/ticket/2201 Thanks Dirk, for the start of an utar script! I'll have to check it out though[1] to even put it in with a req on GNU tar. And the check for a emtpy "$TAR" should be moved up to the assignment. Your quoting of variables is fine, which is often neglected, but your awk needs some serious polishing. Parsing the 'tar tvaf' output via char-positions is just a no-no. Use awk's field-splitting as it was intended. But also: POSIX tar (at least star) does not support 'tar tvaf' and -a is incompatible anyway: -a,-atime reset access time after storing file and the format of '-t -v' output of gtar/star differs as well (and star lists files on stderr!) [tar is gtar here]: $ star -t -v -j -f /newsw/xemacs-21.4.22.tar.bz2 2>/dev/null | wc -l 0 $ tar -t -v -j -f /newsw/xemacs-21.4.22.tar.bz2 2>/dev/null | wc -l 1724 $ tar -t -v -j -f /newsw/xemacs-21.4.22.tar.bz2 >/dev/null | wc -l 0 $ star -t -v -j -f xemacs-21.4.22.tar.bz2 2>&1 | head -2 0 drwxr-xr-x acs/acs Dec 31 03:55 2008 xemacs-21.4.22/ 888 -rw-r--r-- acs/acs Mar 28 03:27 1997 xemacs-21.4.22/BUGS $ tar -t -v -j -f /newsw/xemacs-21.4.22.tar.bz2 2>&1 | head -2 drwxr-xr-x acs/acs 0 2008-12-31 03:55 xemacs-21.4.22/ -rw-r--r-- acs/acs 888 1997-03-28 03:27 xemacs-21.4.22/BUGS So, I strongly think I'll scrap the shell-script idea and use perl and Archive::Tar. It is POSIX compatible, allows access to file attributes via a hash with 'stat'-like fields, and support gzip and bzip2 compressed files. Perl has support for e.g. Lzma, Lzop, Xz, and Zip... On the other hand, that'd get us into dependency hell (requiring lmza, lzop, xz, etc.). This needs more work. mc should be able to handle the (de)compression via extfs/archive.sh or lib/util.* and then "extfs/utar" would have to handle only uncompressed archives. That should work, c.f. 'deb*' and 'uar' ... Anyway: the problem isn't fixed by just adding e.g. Dirk's "utar" script. Read the whole thread on os-factory mentioned above ... Upstream hasn't come up with a extfs-based-on-gnu-tar (or whatever) in years ... It's not easy. I moved it up on my mental priorty list and if I feel like it, I might improve your script to explicitly and only use gnu-tar (version > x) to implement a workaround. -dnh [1] I've got a dir-hierarchy with "sick filenames" (including everything I could come up with that is not ASCII-NUL or '/' so e.g. linebreaks, *, ', ", \r, ... ;) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=836558 https://bugzilla.novell.com/show_bug.cgi?id=836558#c3 --- Comment #3 from Tomas Cech <tcech@suse.com> 2013-09-28 14:55:37 CEST --- Created an attachment (id=560684) --> (http://bugzilla.novell.com/attachment.cgi?id=560684) sed script (In reply to comment #2)
Thomas: yes. I said so some time ago ... *dig* in ah, yes: http://lists.opensuse.org/opensuse-factory/2013-08/msg00464.html and esp. http://lists.opensuse.org/opensuse-factory/2013-08/msg00466.html http://lists.opensuse.org/opensuse-factory/2013-08/msg00468.html See also https://www.midnight-commander.org/ticket/2201
I already read that. I found that after my comment and looking into code a bit.
Thanks Dirk, for the start of an utar script! I'll have to check it out though[1] to even put it in with a req on GNU tar. And the check for a emtpy "$TAR" should be moved up to the assignment.
Yes, good work indeed.
Your quoting of variables is fine, which is often neglected, but your awk needs some serious polishing. Parsing the 'tar tvaf' output via char-positions is just a no-no. Use awk's field-splitting as it was intended.
I've attached equivalent for awk code using sed with use of regexp, not nice but I believe it should work.
But also: POSIX tar (at least star) does not support 'tar tvaf' and -a is incompatible anyway:
-a,-atime reset access time after storing file
and the format of '-t -v' output of gtar/star differs as well (and star lists files on stderr!) [tar is gtar here]:
$ star -t -v -j -f /newsw/xemacs-21.4.22.tar.bz2 2>/dev/null | wc -l 0 $ tar -t -v -j -f /newsw/xemacs-21.4.22.tar.bz2 2>/dev/null | wc -l 1724 $ tar -t -v -j -f /newsw/xemacs-21.4.22.tar.bz2 >/dev/null | wc -l 0
$ star -t -v -j -f xemacs-21.4.22.tar.bz2 2>&1 | head -2 0 drwxr-xr-x acs/acs Dec 31 03:55 2008 xemacs-21.4.22/ 888 -rw-r--r-- acs/acs Mar 28 03:27 1997 xemacs-21.4.22/BUGS $ tar -t -v -j -f /newsw/xemacs-21.4.22.tar.bz2 2>&1 | head -2 drwxr-xr-x acs/acs 0 2008-12-31 03:55 xemacs-21.4.22/ -rw-r--r-- acs/acs 888 1997-03-28 03:27 xemacs-21.4.22/BUGS
Is really necessary to support all tar flavours? This will make things too complicated. I have never met an archive that couldn't be opened by GNU tar. I don't say it doesn't exist, I say that in my whole Linux history I haven't seen one.
So, I strongly think I'll scrap the shell-script idea and use perl and Archive::Tar. It is POSIX compatible, allows access to file attributes via a hash with 'stat'-like fields, and support gzip and bzip2 compressed files.
Please no, do not bring more dependencies. And besides, less perl code could make the world more happy place :b
Perl has support for e.g. Lzma, Lzop, Xz, and Zip... On the other hand, that'd get us into dependency hell (requiring lmza, lzop, xz, etc.).
This needs more work. mc should be able to handle the (de)compression via extfs/archive.sh or lib/util.* and then "extfs/utar" would have to handle only uncompressed archives. That should work, c.f. 'deb*' and 'uar' ...
Anyway: the problem isn't fixed by just adding e.g. Dirk's "utar" script. Read the whole thread on os-factory mentioned above ...
I read it all but I can't see why...
Upstream hasn't come up with a extfs-based-on-gnu-tar (or whatever) in years ... It's not easy. I moved it up on my mental priorty list and if I feel like it, I might improve your script to explicitly and only use gnu-tar (version > x) to implement a workaround.
Let's use this first script to fix this bug and see if there will be issues with that, if even someone will lack such support. So yes, I'd say it's way to go.
[1] I've got a dir-hierarchy with "sick filenames" (including everything I could come up with that is not ASCII-NUL or '/' so e.g. linebreaks, *, ', ", \r, ... ;)
This sounds like good test. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=836558 https://bugzilla.novell.com/show_bug.cgi?id=836558#c4 --- Comment #4 from Dirk Weber <d_werner@gmx.net> 2013-10-12 18:59:07 UTC --- Created an attachment (id=563198) --> (http://bugzilla.novell.com/attachment.cgi?id=563198) improved untar extfs (GNU tar only, polished awk part) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=836558 https://bugzilla.novell.com/show_bug.cgi?id=836558#c Dirk Weber <d_werner@gmx.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #563198|improved untar extfs (GNU |improved untar extfs (GNU description|tar only, polished awk |tar only, polished awk |part) |part) | | | |related to | |http://lists.opensuse.org/o | |pensuse-factory/2013-08/msg | |00486.html | |do not pretend to support | |anything but GNU tar -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=836558 https://bugzilla.novell.com/show_bug.cgi?id=836558#c5 --- Comment #5 from Dirk Weber <d_werner@gmx.net> 2013-10-16 18:34:52 UTC --- (In reply to comment #4)
Created an attachment (id=563198) --> (http://bugzilla.novell.com/attachment.cgi?id=563198) [details] improved untar extfs (GNU tar only, polished awk part)
for openSUSE with GNU tar as default this is probably sufficient. for upstream and different tar implementations probably the best solution would be to have separate branches for each (common) tar implementation with the specific parameters and reformatting the specific output. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=836558 https://bugzilla.novell.com/show_bug.cgi?id=836558#c6 --- Comment #6 from Dirk Weber <d_werner@gmx.net> 2013-11-12 19:22:10 UTC --- Created an attachment (id=567156) --> (http://bugzilla.novell.com/attachment.cgi?id=567156) improved untar extfs which handles file names which happen to match permissiond, time, date, size during checking bnc#849082 I realized a weakness of the previously provided untar extfs by matching the "file name" $6 with the line. The new version uses the time $5 as hook. This pattern should not appear on the line before the correct occurrence. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=836558 David Haller <dnh@opensuse.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|dnh@opensuse.org |bnc-team-screening@forge.pr | |ovo.novell.com -- You are receiving this mail because: You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠novell.com