[opensuse] OT-ish : dir content globbing question
Hi, all -- I know that this isn't SuSE-specific, but everyone here loves a puzzle :-) I have a collection of directories with varying contents; most but not all have files in them, and many of interest have subdirs in them. Here's a sample set: davidtg@u17383850:/tmp/TEST> du -ka * 16404 2019-0203/file 32804 2019-0203/DIR-100/content 32808 2019-0203/DIR-100 32804 2019-0203/DIR-23W/content 32808 2019-0203/DIR-23W 32804 2019-0203/DIR-A05/content 32808 2019-0203/DIR-A05 114832 2019-0203 16404 2019-0312/file 16408 2019-0312 32804 2019-0416/DIR-209/content 32808 2019-0416/DIR-209 16404 2019-0416/file 16404 2019-0416/other 65620 2019-0416 4 2019-0611 32804 2019-0817/DIR-209/content 32808 2019-0817/DIR-209 16404 2019-0817/file 32804 2019-0817/DIR-941/content 32808 2019-0817/DIR-941 32804 2019-0817/DIR-PXD/content 32808 2019-0817/DIR-PXD 114832 2019-0817 16404 2019-0924/file 16404 2019-0924/other 32812 2019-0924 I want to get a du total for all of the DIR-* subdirs, grouped by parent, without throwing any errors. The simple version obviously doesn't work: davidtg@u17383850:/tmp/TEST> for D in 2* ; do du -skhc $D/DIR-* ; done 33M 2019-0203/DIR-100 33M 2019-0203/DIR-23W 33M 2019-0203/DIR-A05 97M total du: cannot access `2019-0312/DIR-*': No such file or directory 0 total 33M 2019-0416/DIR-209 33M total du: cannot access `2019-0611/DIR-*': No such file or directory 0 total 33M 2019-0817/DIR-209 33M 2019-0817/DIR-941 33M 2019-0817/DIR-PXD 97M total du: cannot access `2019-0924/DIR-*': No such file or directory 0 total The best I've found in ten minutes of thinking is to use case to test a globbing of the list to see if it's still the input, but that seems ... ugly :-) davidtg@u17383850:/tmp/TEST> for D in 2* ; do DIRS=`echo $D/DIR-*` ; case "$DIRS" in *\* ) continue ;; esac ; du -skhc $D/DIR* ; done 33M 2019-0203/DIR-100 33M 2019-0203/DIR-23W 33M 2019-0203/DIR-A05 97M total 33M 2019-0416/DIR-209 33M total 33M 2019-0817/DIR-209 33M 2019-0817/DIR-941 33M 2019-0817/DIR-PXD 97M total What I'd really like is to be able to [ -d $D/DIR-* ] but of course that doesn't work because there can be multiple matches. Got any better ideas? HANN :-D -- David T-G See http://justpickone.org/davidtg/email/ See http://justpickone.org/davidtg/tofu.txt -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Hello, On Fri, 04 Oct 2019, David T-G wrote:
I know that this isn't SuSE-specific, but everyone here loves a puzzle :-) I have a collection of directories with varying contents; most but not all have files in them, and many of interest have subdirs in them. Here's a sample set: [..] I want to get a du total for all of the DIR-* subdirs, grouped by parent, without throwing any errors. The simple version obviously doesn't work: [..] What I'd really like is to be able to [ -d $D/DIR-* ] but of course that doesn't work because there can be multiple matches. Got any better ideas?
$ find bla/ -type d -exec du -hkc {} + HTH, -dnh -- New scheduler deployed on friday was buggy, we learn not to deploy larger changes on fridays anymore ;) -- Adrian Schröter in opensuse-buildservice -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
David, et al -- ...and then David Haller said... % % Hello, % % On Fri, 04 Oct 2019, David T-G wrote: % [..] % >I want to get a du total for all of the DIR-* subdirs, grouped by parent, % >without throwing any errors. The simple version obviously doesn't work: % [..] % >What I'd really like is to be able to [ -d $D/DIR-* ] but of course that % >doesn't work because there can be multiple matches. Got any better ideas? % % $ find bla/ -type d -exec du -hkc {} + OK, so I thought that this wouldn't work nicely, but when testing it I actually noticed the + at the end, and I kinda like it! So I'm going to have to read up on that later today and report back... Thanks! % % HTH, % -dnh HAND :-D -- David T-G See http://justpickone.org/davidtg/email/ See http://justpickone.org/davidtg/tofu.txt -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
05.10.2019 13:42, David T-G пишет:
David, et al --
...and then David Haller said... % % Hello, % % On Fri, 04 Oct 2019, David T-G wrote: % [..] % >I want to get a du total for all of the DIR-* subdirs, grouped by parent, % >without throwing any errors. The simple version obviously doesn't work: % [..] % >What I'd really like is to be able to [ -d $D/DIR-* ] but of course that % >doesn't work because there can be multiple matches. Got any better ideas? % % $ find bla/ -type d -exec du -hkc {} +
OK, so I thought that this wouldn't work nicely, but when testing it I actually noticed the + at the end, and I kinda like it! So I'm going to have to read up on that later today and report back... Thanks!
This won't summarize by top level directories if this is what you need. Something like for i in 2*/; do find "$i" -maxdepth 1 -type d -name DIR-\* -exec du -chk '{}' +; done will. Note that if you may have very large number of DIR-* entries this will result in several invocations of du and so total summary will be wrong (or better you will get several summaries for the same top level directory). P.S. your original question is literally beaten to death. At the end the only way to check whether there are files matching glob pattern is to actually expand it with real file names. Your choice is only how to check result of expansion (or that there is any result of expansion). For that matter the most simple is "ls $i/DIR-* > /dev/null 2>&1". -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Andrei -- ...and then Andrei Borzenkov said... % % 05.10.2019 13:42, David T-G ??????????: % > % > ...and then David Haller said... % > % % > % On Fri, 04 Oct 2019, David T-G wrote: % > % [..] % > % >I want to get a du total for all of the DIR-* subdirs, grouped by parent, % > % >without throwing any errors. The simple version obviously doesn't work: ... % > % >What I'd really like is to be able to [ -d $D/DIR-* ] but of course that % > % >doesn't work because there can be multiple matches. Got any better ideas? % > % % > % $ find bla/ -type d -exec du -hkc {} + % > % > OK, so I thought that this wouldn't work nicely, but when testing it I % > actually noticed the + at the end, and I kinda like it! So I'm going to % > have to read up on that later today and report back... Thanks! % % This won't summarize by top level directories if this is what you need. That is the goal, actually, as I showed in my example. ... % P.S. your original question is literally beaten to death. At the end the % only way to check whether there are files matching glob pattern is to % actually expand it with real file names. Your choice is only how to % check result of expansion (or that there is any result of expansion). % For that matter the most simple is "ls $i/DIR-* > /dev/null 2>&1". Dang... I was afraid of that. It just seems so ugly, ya know? :-) But I figured it would come down to hiding errors or doing more complex tests. My other idea was FLAGVAR=F ; for SUBDIR in $D/DIR-* ; do FLAGVAR=T ; done [ 'F' = "$FLAGVAR" ] && continue ... but that's also more keystrokes than it's worth. Ah, well. Thanks for the fun, y'all, and I picked up a new find arg anyway as a bonus :-) HANW :-D -- David T-G See http://justpickone.org/davidtg/email/ See http://justpickone.org/davidtg/tofu.txt -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 05/10/2019 à 03:59, David T-G a écrit :
I want to get a du total for all of the DIR-* subdirs, grouped by parent, without throwing any errors.
can't you simply redirect error message to NULl? jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
jdd, et al -- ...and then jdd@dodin.org said... % % Le 05/10/2019 à 03:59, David T-G a écrit : % % >I want to get a du total for all of the DIR-* subdirs, grouped by parent, % >without throwing any errors. % % can't you simply redirect error message to NULl? Well, I could, but then I get these useless totals for dirs that don't exist: davidtg@u17383850:/tmp/TEST> for D in 2* ; do du -skhc $D/DIR-* 2>/dev/null ; done 33M 2019-0203/DIR-100 33M 2019-0203/DIR-23W 33M 2019-0203/DIR-A05 97M total 0 total 33M 2019-0416/DIR-209 33M total 0 total 33M 2019-0817/DIR-209 33M 2019-0817/DIR-941 33M 2019-0817/DIR-PXD 97M total 0 total It offends the sensibilities ;-) But thanks for the thought. % % jdd :-D -- David T-G See http://justpickone.org/davidtg/email/ See http://justpickone.org/davidtg/tofu.txt -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Fri, 4 Oct 2019 21:59:07 -0400 David T-G <davidtg-robot@justpickone.org> wrote:
Hi, all --
I know that this isn't SuSE-specific, but everyone here loves a puzzle :-) I have a collection of directories with varying contents; most but not all have files in them, and many of interest have subdirs in them. Here's a sample set:
davidtg@u17383850:/tmp/TEST> du -ka * 16404 2019-0203/file 32804 2019-0203/DIR-100/content 32808 2019-0203/DIR-100 32804 2019-0203/DIR-23W/content 32808 2019-0203/DIR-23W 32804 2019-0203/DIR-A05/content 32808 2019-0203/DIR-A05 114832 2019-0203 16404 2019-0312/file 16408 2019-0312 32804 2019-0416/DIR-209/content 32808 2019-0416/DIR-209 16404 2019-0416/file 16404 2019-0416/other 65620 2019-0416 4 2019-0611 32804 2019-0817/DIR-209/content 32808 2019-0817/DIR-209 16404 2019-0817/file 32804 2019-0817/DIR-941/content 32808 2019-0817/DIR-941 32804 2019-0817/DIR-PXD/content 32808 2019-0817/DIR-PXD 114832 2019-0817 16404 2019-0924/file 16404 2019-0924/other 32812 2019-0924
I want to get a du total for all of the DIR-* subdirs, grouped by parent, without throwing any errors. The simple version obviously doesn't work:
davidtg@u17383850:/tmp/TEST> for D in 2* ; do du -skhc $D/DIR-* ; done 33M 2019-0203/DIR-100 33M 2019-0203/DIR-23W 33M 2019-0203/DIR-A05 97M total du: cannot access `2019-0312/DIR-*': No such file or directory 0 total 33M 2019-0416/DIR-209 33M total du: cannot access `2019-0611/DIR-*': No such file or directory 0 total 33M 2019-0817/DIR-209 33M 2019-0817/DIR-941 33M 2019-0817/DIR-PXD 97M total du: cannot access `2019-0924/DIR-*': No such file or directory 0 total
The best I've found in ten minutes of thinking is to use case to test a globbing of the list to see if it's still the input, but that seems ... ugly :-)
davidtg@u17383850:/tmp/TEST> for D in 2* ; do DIRS=`echo $D/DIR-*` ; case "$DIRS" in *\* ) continue ;; esac ; du -skhc $D/DIR* ; done 33M 2019-0203/DIR-100 33M 2019-0203/DIR-23W 33M 2019-0203/DIR-A05 97M total 33M 2019-0416/DIR-209 33M total 33M 2019-0817/DIR-209 33M 2019-0817/DIR-941 33M 2019-0817/DIR-PXD 97M total
I'm not clear whether the two occurrences of DIR-209 are separate and to be counted twice, or refer to the same thing and should be counted once? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Dave, et al -- ...and then Dave Howorth said... % % On Fri, 4 Oct 2019 21:59:07 -0400 % David T-G <davidtg-robot@justpickone.org> wrote: % ... % > davidtg@u17383850:/tmp/TEST> du -ka * ... % > 32804 2019-0416/DIR-209/content % > 32808 2019-0416/DIR-209 % > 16404 2019-0416/file % > 16404 2019-0416/other % > 65620 2019-0416 % > 4 2019-0611 % > 32804 2019-0817/DIR-209/content % > 32808 2019-0817/DIR-209 % > 16404 2019-0817/file ... % % I'm not clear whether the two occurrences of DIR-209 are separate and % to be counted twice, or refer to the same thing and should be counted % once? Aha! Good catch. No, each is independent because the same card was used to capture data on two different dates. I didn't mean to fake this up so that the same card was a solo in each case; that makes it tougher. But, yes, if it exists then it should get counted every time. Thanks again & HAND :-D -- David T-G See http://justpickone.org/davidtg/email/ See http://justpickone.org/davidtg/tofu.txt -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Dave, et al -- ...and then David T-G said... % % Aha! Good catch. No, each is independent because the same card was used % to capture data on two different dates. I didn't mean to fake this up so % that the same card was a solo in each case; that makes it tougher. But, % yes, if it exists then it should get counted every time. Oh, wait. Yes, on 0416 it's solo, but on 0817 it's part of a larger set. So it's a good sample that you noted well. HAND :-D -- David T-G See http://justpickone.org/davidtg/email/ See http://justpickone.org/davidtg/tofu.txt -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (5)
-
Andrei Borzenkov -
Dave Howorth -
David Haller -
David T-G -
jdd＠dodin.org