[opensuse] /bin/chmod: Argument list too long - how long is too long?
Listmates, I was trying to use chmod, but it says: 12:44 nirvana/srv/www/download> find . -type f -print0 | xargs -0 sudo chmod 0644 sudo: unable to execute /bin/chmod: Argument list too long How long is too long? How many files will chmod do, or does it depend on the length of file names? There are only 5590 files beneath this directory: 12:47 nirvana/srv/www/download> find . -type f | wc -l 5590 To accomplish what I wanted to do I had to change the find call to: sudo find . -type f -exec sudo chmod 0644 '{}' \; This is much...much... slower. How big is the chmod limitation and is there a better way to get around this limitation? -- David C. Rankin, J.D.,P.E. | openSoftware und SystemEntwicklung Rankin Law Firm, PLLC | Countdown for openSuSE 11.1 www.rankinlawfirm.com | http://counter.opensuse.org/11.1/small -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday, 2008-11-24 at 12:52 -0600, David C. Rankin wrote:
I was trying to use chmod, but it says:
12:44 nirvana/srv/www/download> find . -type f -print0 | xargs -0 sudo chmod 0644 sudo: unable to execute /bin/chmod: Argument list too long
How long is too long? How many files will chmod do, or does it depend on the length of file names?
There are only 5590 files beneath this directory:
Yes, it depends on the length of the entire string, because all those file names are passed into a single string. At 15 chars per name, it makes more than 64KB, which I think is the limit. It is a limit of bash. The question has been asked before, usually related to find, or to delete a large number of files. Search the archive... I think there is one solution using find that passes the argument in bunches of reasonable size. If I can I'll look later on my personal archive, must be somewhere. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkkrD6wACgkQtTMYHG2NR9VgzgCgh+yyeZp3imadOsItJDr2Scfc OeEAoIngf3DYHv99WwOxTGFpeUd2bqyj =38jh -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
2008/11/24 Carlos E. R. <robin.listas@telefonica.net>:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Monday, 2008-11-24 at 12:52 -0600, David C. Rankin wrote:
I was trying to use chmod, but it says:
12:44 nirvana/srv/www/download> find . -type f -print0 | xargs -0 sudo chmod 0644 sudo: unable to execute /bin/chmod: Argument list too long
It's simple, just modify your command to : sudo find . -type f -exec chmod 0644 {} \; -print good luk! -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
----- Original Message ----- From: "Ricardo Gomez" <ricgomez@ricgomez.net> To: <opensuse@opensuse.org> Sent: Monday, November 24, 2008 3:47 PM Subject: Re: [opensuse] /bin/chmod: Argument list too long - how long is too long?
2008/11/24 Carlos E. R. <robin.listas@telefonica.net>:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Monday, 2008-11-24 at 12:52 -0600, David C. Rankin wrote:
I was trying to use chmod, but it says:
12:44 nirvana/srv/www/download> find . -type f -print0 | xargs -0 sudo chmod 0644 sudo: unable to execute /bin/chmod: Argument list too long
It's simple, just modify your command to :
sudo find . -type f -exec chmod 0644 {} \; -print
That's hardly any better. xargs is the correct tool for this job just as he originally tried. The very definition of xarg's job is to collect input into chunks that are as big as the shell can handle, but no bigger. It's whole point is to coalesce commands as much as the shell and the kernel will allow, because iterating through huge lists one file or one item at a time (as above, using find alone) is slow and inefficient a lot of times. It's true that some versions of find do, or can coalesce the arguments sort of like having xargs built-in, but that is the exception not the rule and shouldn't be relied on. His original command , of the form "find |xargs" was already the ideal. The only thing I can think of is perhaps the sudo screws it up somehow. What the original command was really running was several big sudo commands, where really it's more expected to run several big chmod or other direct commands. (who wants to answer sudo password prompt 50 times?) sudo also just does a lot of funky stuff with the child environment in general and something like this result doesn't really surprise me. I would try: su - find |xargs chmod Also, there are options to reduce the buffer size xargs will assemble. find |xargs -s 32767 According to man xargs, the default size it will assemble is 131k, so 32k should be plenty safe and yet still make large enough command lines to chew through the job fast. Ovbviously you can play with the -s size and find where exactly it stops working and find the largest working value. -- Brian K. White brian@aljex.com http://www.myspace.com/KEYofR +++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++. filePro BBx Linux SCO FreeBSD #callahans Satriani Filk! -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Brian K. White wrote:
----- Original Message ----- From: "Ricardo Gomez" <ricgomez@ricgomez.net> To: <opensuse@opensuse.org> Sent: Monday, November 24, 2008 3:47 PM Subject: Re: [opensuse] /bin/chmod: Argument list too long - how long is too long?
2008/11/24 Carlos E. R. <robin.listas@telefonica.net>:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Monday, 2008-11-24 at 12:52 -0600, David C. Rankin wrote:
I was trying to use chmod, but it says:
12:44 nirvana/srv/www/download> find . -type f -print0 | xargs -0 sudo chmod 0644 sudo: unable to execute /bin/chmod: Argument list too long
It's simple, just modify your command to :
sudo find . -type f -exec chmod 0644 {} \; -print
That's hardly any better. xargs is the correct tool for this job just as he originally tried. The very definition of xarg's job is to collect input into chunks that are as big as the shell can handle, but no bigger. It's whole point is to coalesce commands as much as the shell and the kernel will allow, because iterating through huge lists one file or one item at a time (as above, using find alone) is slow and inefficient a lot of times. It's true that some versions of find do, or can coalesce the arguments sort of like having xargs built-in, but that is the exception not the rule and shouldn't be relied on. His original command , of the form "find |xargs" was already the ideal.
The only thing I can think of is perhaps the sudo screws it up somehow. What the original command was really running was several big sudo commands, where really it's more expected to run several big chmod or other direct commands. (who wants to answer sudo password prompt 50 times?) sudo also just does a lot of funky stuff with the child environment in general and something like this result doesn't really surprise me.
I would try: su - find |xargs chmod
Also, there are options to reduce the buffer size xargs will assemble. find |xargs -s 32767
According to man xargs, the default size it will assemble is 131k, so 32k should be plenty safe and yet still make large enough command lines to chew through the job fast. Ovbviously you can play with the -s size and find where exactly it stops working and find the largest working value.
Excellent feedback. I'll give it a go and report back. This wouldn't be the first time sudo has bitten me. I guess as a rule, I should eliminate it from my scripts and go back to: ROOT_UID=0 E_NOTROOT=67 if [ "$UID" -ne "$ROOT_UID" ]; then echo -e "\nYou must be root to run this script.\nUser: $USER, UID: $UID can't! See below...\n" show_usage exit $E_NOTROOT fi In order to force either a true root execution or a sudo execution of the entire script instead of splitting commands with sudo as I originally did in the find | xargs situation. I'll let you know! -- David C. Rankin, J.D.,P.E. | openSoftware und SystemEntwicklung Rankin Law Firm, PLLC | Countdown for openSuSE 11.1 www.rankinlawfirm.com | http://counter.opensuse.org/11.1/small -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday, 2008-11-24 at 21:33 +0100, Carlos E. R. wrote:
The question has been asked before, usually related to find, or to delete a large number of files. Search the archive... I think there is one solution using find that passes the argument in bunches of reasonable size.
If I can I'll look later on my personal archive, must be somewhere.
<http://lists.opensuse.org/opensuse/2008-03/msg01937.html> % find dirName(s) {findCriteria} -print0 |xargs -0 rm - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkkrLGIACgkQtTMYHG2NR9Vb4QCeNnef+Ehi9r6oGToxncmoCUd5 VJQAnAqu3mIQ6kvomWhrU94mB/Q2QNlX =bYCe -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Hello, On Mon, 24 Nov 2008, Carlos E. R. wrote:
Yes, it depends on the length of the entire string, because all those file names are passed into a single string. At 15 chars per name, it makes more than 64KB, which I think is the limit. It is a limit of bash.
It's usually 128 KB under 32bit Linux. And it's a limit of the kernel. -dnh -- "It is very easy to be blinded to the essential uselessness of them by the sense of achievement you get from getting them to work at all." Hitchiker's Guide to the Galaxy on the Sirius Cybernetics Corporation. Does that seem familiar to anything on THIS planet at THIS time? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Tuesday 25 November 2008 05:22:33 David C. Rankin wrote:
Listmates,
I was trying to use chmod, but it says:
12:44 nirvana/srv/www/download> find . -type f -print0 | xargs -0 sudo chmod 0644 sudo: unable to execute /bin/chmod: Argument list too long
How long is too long? How many files will chmod do, or does it depend on the length of file names?
There are only 5590 files beneath this directory:
12:47 nirvana/srv/www/download> find . -type f | wc -l 5590
To accomplish what I wanted to do I had to change the find call to:
sudo find . -type f -exec sudo chmod 0644 '{}' \;
This is much...much... slower. How big is the chmod limitation and is there a better way to get around this limitation?
David, There was a thread on exactly this subject (with suggested solutions) going back to March this year (subject was "[opensuse] /bin/rm: Argument list too long"). You should find it in the list archives. If not, I have 3 key messages saved and can forward them direct to you if you want. Regards, Rodney. -- =================================================== Rodney Baker VK5ZTV rodney.baker@iinet.net.au ===================================================
On Monday 24 November 2008 10:52, David C. Rankin wrote:
Listmates,
I was trying to use chmod, but it says:
12:44 nirvana/srv/www/download> find . -type f -print0 | xargs -0 sudo chmod 0644 sudo: unable to execute /bin/chmod: Argument list too long
How long is too long? How many files will chmod do, or does it depend on the length of file names?
...
sudo find . -type f -exec sudo chmod 0644 '{}' \;
This is much...much... slower. How big is the chmod limitation and is there a better way to get around this limitation?
One would expect sudo to know the real system limit and impose that, but perhaps it has its own idea of the upper limit. I found no mention of this in man page, but one thing you can do is tell xargs to use a argument limit smaller than the system's: --max-chars=max-chars, -s max-chars Use at most max-chars characters per command line, including the command and initial-arguments and the terminating nulls at the ends of the argument strings. The default is 131072 characters, not including the size of the environment variables (which are provided for separately so that it doesn't matter if your envi- ronment variables take up more than 131072 bytes). The operat- ing system places limits on the values that you can usefully specify, and if you exceed these a warning message is printed and the value actually used is set to the appropriate upper or lower limit. Given that (putative) sudo limit is unknown, you might have to experiment to find an acceptable one.
-- David C. Rankin
Randall Schulz -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
participants (7)
-
Brian K. White -
Carlos E. R. -
David C. Rankin -
David Haller -
Randall R Schulz -
Ricardo Gomez -
Rodney Baker