Hi Folks, I've got a little problem here that I bet someone here could handle. I've got a couple of thousand zip files that some knuckle-headed package created that have to be unpacked for inspection. They're named with a long date-time group with the .zip extension, as in Fri-Apr-30-185859-PDT-2021-logfile.zip Alas, they all unpack to the same filename: logfile So what's the easiest way to unzip them all and add the date-time string to the resultant logfile name so that they all don't unpack to the same filename and step on each other. Awk maybe? A clever pipeline with xargs? Or? Regards, Lew
Lew, et al -- ...and then Lew Wolfgang said... % % Hi Folks, Good morning! % ... % named with a long date-time group with the .zip extension, as in % % Fri-Apr-30-185859-PDT-2021-logfile.zip % % Alas, they all unpack to the same filename: logfile [snip] It depends on how much time you want to take. The ZIP file name appaers to be perfect* for each file, so you could either serialize it for F in *-logfile.zip do unzip $F && mv logfile `basename $F .zip` || break done or whip up a little script like #!/bin/sh set -e [ -f $1 ] B=`basename $1 .zip` mkdir $B.dir cd $B.dir unzip ../$B mv logfile ../$B cd .. rmdir $B.dir and parallelize parallel /path/to/script ::: *-logfile.zip it all. [If my parallel-fu were stronger, I'd whip out a parallel blah blah blah ::: *-logfile.zip one-liner that does it all, but ... not there yet :-] * Well, unique, at least ... Why don't people use ISO format when they embed the date in a file name? Lord knows that sorting by day of the week isn't very useful ... HTH & HANW :-D -- David T-G See http://justpickone.org/davidtg/email/ See http://justpickone.org/davidtg/tofu.txt
On 01-05-2021 05:36, David T-G wrote:
Lew, et al --
...and then Lew Wolfgang said... % % Hi Folks,
Good morning!
% ... % named with a long date-time group with the .zip extension, as in % % Fri-Apr-30-185859-PDT-2021-logfile.zip % % Alas, they all unpack to the same filename: logfile [snip]
It depends on how much time you want to take. The ZIP file name appaers to be perfect* for each file, so you could either serialize it
for F in *-logfile.zip do unzip $F && mv logfile `basename $F .zip` || break done
or whip up a little script like
#!/bin/sh set -e [ -f $1 ] B=`basename $1 .zip` mkdir $B.dir cd $B.dir unzip ../$B mv logfile ../$B cd .. rmdir $B.dir
and parallelize
parallel /path/to/script ::: *-logfile.zip
it all. [If my parallel-fu were stronger, I'd whip out a
parallel blah blah blah ::: *-logfile.zip
one-liner that does it all, but ... not there yet :-]
* Well, unique, at least ... Why don't people use ISO format when they embed the date in a file name? Lord knows that sorting by day of the week isn't very useful ...
HTH & HANW
:-D
And for people with bad eye sight $(command) instead of `command` Jos vankan@linux-wo23:~> echo $(which echo) /usr/bin/echo vankan@linux-wo23:~> echo 'which echo' which echo vankan@linux-wo23:~> echo `which echo` /usr/bin/echo -- Jos van Kan registered Linux user #152704
Jos, et al -- ...and then Jos van Kan said... % % On 01-05-2021 05:36, David T-G wrote: % > ... % > do % > unzip $F && mv logfile `basename $F .zip` || break % > done ... % % And for people with bad eye sight $(command) instead of `command` [snip] True :-) but not Bourne shell standard. I tend to shy away from fancy bash, zsh, tcsh extensions since that code is not portable. I've only found two cases where sh implementation differed across flavors, and those were edge-case enough that I can live with it :-) HANW :-D -- David T-G See http://justpickone.org/davidtg/email/ See http://justpickone.org/davidtg/tofu.txt
On 01/05/2021 04.09, Lew Wolfgang wrote:
Hi Folks,
I've got a little problem here that I bet someone here could handle.
I've got a couple of thousand zip files that some knuckle-headed package created that have to be unpacked for inspection. They're named with a long date-time group with the .zip extension, as in
Fri-Apr-30-185859-PDT-2021-logfile.zip
Alas, they all unpack to the same filename: logfile
So what's the easiest way to unzip them all and add the date-time string to the resultant logfile name so that they all don't unpack to the same filename and step on each other. Awk maybe? A clever pipeline with xargs? Or?
I would unpack one by one, with a script, and rename "logfile" to "ISODATE-logfile" somehow, no compression, to a different directory. Then, on that directory you could instead do "gzip *", not zip. Advantage? there are tools like zgrep designed to work on compressed logs. If that looks interesting, we can concoct a script. The only part I can not think yet about (early tea phase) is how to convert to isodate. I agree with David, and I would say that's the only date format that makes any sense. All others should be shot and burnt :-P for FILE in ls *zip do unzip $FILE etc... -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 2021/05/01 03:28, Carlos E. R. wrote:
If that looks interesting, we can concoct a script.
Here's part of the script. Feed the date part of the filename to the cnvtim function and it should give you a better format for sequencing the logs:
/tmp/dtcnv "Fri-Apr-30-185859-PDT-2021" 20210430-185859.log
#!/bin/bash -u # vim=:SetNumberAndWidth shopt -s expand_aliases alias my='declare ' int='my -i ' array='my -a ' map='my -A ' # For more full aliases et al, use .shh setx() { trap unsetx EXIT; set -x; } ; unsetx() { set +x;} ################################################################################ #setx array monams=(jan feb mar apr may jun jul aug sep oct nov dec) int ctr=1 my -l v map movals=() for v in ${monams[@]}; do int val=ctr movals[$v]=$val ctr+=1 done cnvtim() { my dt=${1:?} my -l mo (export IFS=-; read dn mo da t tz yr<<<"$dt" int h=${t:0:2} int m=${t:2:2} int s=${t:4:2} printf "%04d%02d%02d-%02d%02d%02d.log\n" $yr ${movals[$mo]} $da $h $m $s ) } cnvtim "$@" # vim: ts=2 sw=2
participants (5)
-
Carlos E. R. -
David T-G -
Jos van Kan -
L A Walsh -
Lew Wolfgang