Mailinglist Archive: opensuse (2459 mails)

< Previous Next >
Re: [opensuse] /bin/rm: Argument list too long
  • From: Randall R Schulz <rschulz@xxxxxxxxx>
  • Date: Thu, 27 Mar 2008 15:53:33 -0700
  • Message-id: <200803271553.33664.rschulz@xxxxxxxxx>
On Thursday 27 March 2008 15:35, Brian K. White wrote:
...

If you run:
find /path/to/foo -type f -name foo* | xargs zgrep -i blah
in the wrong place, you will still get arg list too long.
That is why it's always:
find /path/to/foo -type f -name 'foo*' | xargs zgrep -i blah
single-forward-quotes

It's even more complicated than that. BASH has an option that determines
whether an argument with glob characters ('*', e.g.) that matches no
files is an error or is passed as entered (including the glob
characters).

Personally, though I have my shell configured to pass non-matching glob
patterns through, with only one exception I always quote glob-bearing
arguments. The exception is grep's --include= or --exclude= options.
The likelihood that I have files whose names begin with "--include="
or "--exclude=" is low enough to be considered zero (by me).


<aside>
I have seen scripts that are part of expensive commercial software
that did just that. They used xargs to process a list of
unpredictable size - good! They used shell globbing characters to
generate the list - bonehead! ...Then they go on in the same script
to use chown and chmod in backwards order such that chown undoes what
chmod did. </aside>

I see lots of very poorly written shell scripts. It's not as easy to do
right as many people seem to think.


If you don't use find -print0 |xargs -0
then any files with spaces in the names will look like seperate files
(which don't exist) to whatever command you ran with xargs.

Actually, they could exist. There's no reason a given directory could
not contain both of these file names, e.g.:

"foo"
"foo is bar"


If you have filenames with globbing characters in them (this is
legal), then they may be handled ok by find and xargs and then be
expanded by the shell on the command line that xargs created, thus
increasing the command line beyond the maximum since xargs would have
created a line that was already maxxed out.

Xargs does not use the shell to interpret the command invocations it
creates. It goes directly to the kernel's fork and exec primitives.


It's also possible for there to be discrepencies between the app
being run by xargs, and the rest of the system. Maybe the shell and
the kernel will tolerate up to 4k (I think it's more like 100 or 400
but no matter) so xargs generates 4k command lines, but maybe appfoo
can only handle 1k ?

That's not how it works. Xargs deals only in the maximum argument list
limits (number of args and / or total argument bytes). While I can't
say that no command imposes its own limits on the size of arguments
lists, there's really no reason to do so and I know of no programs that
have internal limits to the size of the argument list they will
successfully process.


...

--
Brian K. White


Randall Schulz
--
To unsubscribe, e-mail: opensuse+unsubscribe@xxxxxxxxxxxx
For additional commands, e-mail: opensuse+help@xxxxxxxxxxxx

< Previous Next >
Follow Ups