imagemagick: I/O error : buffer full - what does this mean?
Hi, since my upgrade to OS 15.5, when running a command line job for imagemagick I get I/O error : buffer full The task is finished anyway, as much as I can see, perfect. But on 15.3 I have never seen this message and I wonder what it could mean and how, if so, I could avoid it. Thanks for hints! Daniel -- Daniel Bauer photographer Basel Málaga Twitter: @Marsfotografo (often explicit nudes) https://www.patreon.com/danielbauer https://www.daniel-bauer.com (nudes)
Am 31.08.23 um 18:08 schrieb Daniel Bauer:
Hi,
since my upgrade to OS 15.5, when running a command line job for imagemagick I get
I/O error : buffer full
The task is finished anyway, as much as I can see, perfect. But on 15.3 I have never seen this message and I wonder what it could mean and how, if so, I could avoid it.
Actually, after googling, I now believe that it's not an imagemagick problem, but comes from the bash... The "buffer full" occurs for example when running this: #!/bin/sh for bild in *.JPG do convert "$bild" -quality 50 -verbose "${bild%.JPG}.png" done (or other similar scripts). The "I/O error : buffer full" appears for each processing image. I have never seen that on OS 15.3, but now (on 15.5) with all my bash scripts that use imagemagick. Do I need to adjust buffer size or something? And, if so, how? -- Daniel Bauer photographer Basel Málaga Twitter: @Marsfotografo (often explicit nudes) https://www.patreon.com/danielbauer https://www.daniel-bauer.com (nudes)
On 2023-08-31 12:40, Daniel Bauer wrote:
Am 31.08.23 um 18:08 schrieb Daniel Bauer:
Hi,
since my upgrade to OS 15.5, when running a command line job for imagemagick I get
I/O error : buffer full
The task is finished anyway, as much as I can see, perfect. But on 15.3 I have never seen this message and I wonder what it could mean and how, if so, I could avoid it.
Actually, after googling, I now believe that it's not an imagemagick problem, but comes from the bash...
The "buffer full" occurs for example when running this:
#!/bin/sh for bild in *.JPG do convert "$bild" -quality 50 -verbose "${bild%.JPG}.png" done
(or other similar scripts). The "I/O error : buffer full" appears for each processing image.
I suspect the problem could be in the "bild in *.JPG" wich can convert to a huge command line, and overflows the command line input buffer. It happens now probably because you have more photos.
I have never seen that on OS 15.3, but now (on 15.5) with all my bash scripts that use imagemagick.
Do I need to adjust buffer size or something? And, if so, how?
If it is what I suspect, you have to replace that line with something else that doesn't list the entire directory in one line. "find" can do that "somehow". I say somehow because I don't remember right now, but someone will with that hint. -- Cheers / Saludos, Carlos E. R. (from openSUSE 15.5 (Laicolasse))
Am 31.08.23 um 19:41 schrieb Carlos E. R.:
On 2023-08-31 12:40, Daniel Bauer wrote:
Am 31.08.23 um 18:08 schrieb Daniel Bauer:
Hi,
since my upgrade to OS 15.5, when running a command line job for imagemagick I get
I/O error : buffer full
The task is finished anyway, as much as I can see, perfect. But on 15.3 I have never seen this message and I wonder what it could mean and how, if so, I could avoid it.
Actually, after googling, I now believe that it's not an imagemagick problem, but comes from the bash...
The "buffer full" occurs for example when running this:
#!/bin/sh for bild in *.JPG do convert "$bild" -quality 50 -verbose "${bild%.JPG}.png" done
(or other similar scripts). The "I/O error : buffer full" appears for each processing image.
I suspect the problem could be in the "bild in *.JPG" wich can convert to a huge command line, and overflows the command line input buffer.
It happens now probably because you have more photos.
Possible. On the other hand I suspect (in my mini-knowledge) that if this was the case, the buffer would overflow when the script starts. But actually this message is displayed for every single loop or image. And why didn't the problem occur in former OpenSuse versions? Those scripts are really very old and I use them since many years and never had this overflow.
I have never seen that on OS 15.3, but now (on 15.5) with all my bash scripts that use imagemagick.
Do I need to adjust buffer size or something? And, if so, how?
If it is what I suspect, you have to replace that line with something else that doesn't list the entire directory in one line. "find" can do that "somehow". I say somehow because I don't remember right now, but someone will with that hint.
Isn't it possible that this buffer now on OS 15.3 is smaller than it was before? If so, how can I make it larger? -- Daniel Bauer photographer Basel Málaga Twitter: @Marsfotografo (often explicit nudes) https://www.patreon.com/danielbauer https://www.daniel-bauer.com (nudes)
On 2023-08-31 15:03, Daniel Bauer wrote:
Am 31.08.23 um 19:41 schrieb Carlos E. R.:
On 2023-08-31 12:40, Daniel Bauer wrote:
Am 31.08.23 um 18:08 schrieb Daniel Bauer:
Hi,
since my upgrade to OS 15.5, when running a command line job for imagemagick I get
I/O error : buffer full
The task is finished anyway, as much as I can see, perfect. But on 15.3 I have never seen this message and I wonder what it could mean and how, if so, I could avoid it.
Actually, after googling, I now believe that it's not an imagemagick problem, but comes from the bash...
The "buffer full" occurs for example when running this:
#!/bin/sh for bild in *.JPG do convert "$bild" -quality 50 -verbose "${bild%.JPG}.png" done
(or other similar scripts). The "I/O error : buffer full" appears for each processing image.
I suspect the problem could be in the "bild in *.JPG" wich can convert to a huge command line, and overflows the command line input buffer.
It happens now probably because you have more photos.
Possible. On the other hand I suspect (in my mini-knowledge) that if this was the case, the buffer would overflow when the script starts. But actually this message is displayed for every single loop or image.
Then it must be something else. -- Cheers / Saludos, Carlos E. R. (from openSUSE 15.5 (Laicolasse))
On Thu, 31 Aug 2023 13:41:13 -0400, "Carlos E. R." <robin.listas@gmx.es> wrote:
On 2023-08-31 12:40, Daniel Bauer wrote:
Am 31.08.23 um 18:08 schrieb Daniel Bauer:
since my upgrade to OS 15.5, when running a command line job for imagemagick I get
I/O error : buffer full
The task is finished anyway, as much as I can see, perfect. But on 15.3 I have never seen this message and I wonder what it could mean and how, if so, I could avoid it.
Actually, after googling, I now believe that it's not an imagemagick problem, but comes from the bash...
The "buffer full" occurs for example when running this:
#!/bin/sh for bild in *.JPG do convert "$bild" -quality 50 -verbose "${bild%.JPG}.png" done
(or other similar scripts). The "I/O error : buffer full" appears for each processing image.
I suspect the problem could be in the "bild in *.JPG" wich can convert to a huge command line, and overflows the command line input buffer.
It happens now probably because you have more photos.
I have never seen that on OS 15.3, but now (on 15.5) with all my bash scripts that use imagemagick.
Do I need to adjust buffer size or something? And, if so, how?
If it is what I suspect, you have to replace that line with something else that doesn't list the entire directory in one line. "find" can do that "somehow". I say somehow because I don't remember right now, but someone will with that hint.
First, can the error be reproduced with just a single execution of 'convert'? Like: $ bild=file1.JPG $ convert "$bild" -quality 50 -verbose "${bild%.JPG}.png" If, surprisingly, there is no error, then yes, 'find' can eliminate the long command line expansion: $ find . -maxdepth 1 -name '*.JPG' -exec sh -c 'F={} ;convert "$F" -quality 50 -verbose "${F%.JPG}.png"' \; -- Robert Webb
Am 01.09.23 um 09:59 schrieb Robert Webb via openSUSE Users:
On Thu, 31 Aug 2023 13:41:13 -0400, "Carlos E. R." <robin.listas@gmx.es> wrote:
On 2023-08-31 12:40, Daniel Bauer wrote:
Am 31.08.23 um 18:08 schrieb Daniel Bauer:
since my upgrade to OS 15.5, when running a command line job for imagemagick I get
I/O error : buffer full
The task is finished anyway, as much as I can see, perfect. But on 15.3 I have never seen this message and I wonder what it could mean and how, if so, I could avoid it.
Actually, after googling, I now believe that it's not an imagemagick problem, but comes from the bash...
The "buffer full" occurs for example when running this:
#!/bin/sh for bild in *.JPG do convert "$bild" -quality 50 -verbose "${bild%.JPG}.png" done
(or other similar scripts). The "I/O error : buffer full" appears for each processing image.
I suspect the problem could be in the "bild in *.JPG" wich can convert to a huge command line, and overflows the command line input buffer.
It happens now probably because you have more photos.
I have never seen that on OS 15.3, but now (on 15.5) with all my bash scripts that use imagemagick.
Do I need to adjust buffer size or something? And, if so, how?
If it is what I suspect, you have to replace that line with something else that doesn't list the entire directory in one line. "find" can do that "somehow". I say somehow because I don't remember right now, but someone will with that hint.
First, can the error be reproduced with just a single execution of 'convert'? Like:
$ bild=file1.JPG $ convert "$bild" -quality 50 -verbose "${bild%.JPG}.png"
Yes, the error occurs also with the above. It also occurs when I use filenames directly: $ convert _MG_0321.JPG -quality 50 -verbose _MG_0321.png I/O error : buffer full writing raw profile: type=icc, length=560 writing raw profile: type=xmp, length=8192 _MG_0321.JPG=>_MG_0321.png JPEG 4590x6885 4590x6885+0+0 8-bit sRGB 13.9185MiB 5.460u 0:05.459
If, surprisingly, there is no error, then yes, 'find' can eliminate the long command line expansion:
$ find . -maxdepth 1 -name '*.JPG' -exec sh -c 'F={} ;convert "$F" -quality 50 -verbose "${F%.JPG}.png"' \;
-- Robert Webb
-- Daniel Bauer photographer Basel Málaga Twitter: @Marsfotografo (often explicit nudes) https://www.patreon.com/danielbauer https://www.daniel-bauer.com (nudes)
On Fri, 1 Sep 2023 07:59:55 +0000 (UTC), Robert Webb via openSUSE Users <users@lists.opensuse.org> wrote:
$ find . -maxdepth 1 -name '*.JPG' -exec sh -c 'F={} ;convert "$F" -quality 50 -verbose "${F%.JPG}.png"' \;
Excuse me. The filename that 'find' substitutes for "{}" above is unprotected, and this will fail in many cases. Using more quotes will help, but it will still fail for filenames with quotes. Try this instead: $ find . -maxdepth 1 -name '*.JPG' -exec sh -c \ 'convert "$1" -quality 50 -verbose "${1%.JPG}.png"' convert '{}' \; -- Robert Webb
On 8/31/23 11:40, Daniel Bauer wrote:
Actually, after googling, I now believe that it's not an imagemagick problem, but comes from the bash...
The "buffer full" occurs for example when running this:
#!/bin/sh for bild in *.JPG do convert "$bild" -quality 50 -verbose "${bild%.JPG}.png" done
(or other similar scripts). The "I/O error : buffer full" appears for each processing image.
I've never seen an "I/O error : buffer full" as the result of looping though anything. This has a config bug vibe to it -- somewhere. When you loop, 'for bild in *.JPG', the command line is limited to: convert "$bild" -quality 50 -verbose "${bild%.JPG}.png" and no way exceeds `getconf ARG_MAX` characters. You can also run: xargs --no-run-if-empty --show-limits </dev/null to check the command line limit and "command buffer" size in use. -- David C. Rankin, J.D.,P.E.
Hello, In the Message; Subject : imagemagick: I/O error : buffer full - what does this mean? Message-ID : <de8f2344-1061-48cb-babb-5d00f327c9d4@daniel-bauer.com> Date & Time: Thu, 31 Aug 2023 18:08:05 +0200 [DB] == Daniel Bauer <linux@daniel-bauer.com> has written: [...] DB> since my upgrade to OS 15.5, when running a command line job for DB> imagemagick I get DB> I/O error : buffer full [...] This error message is usually displayed to warn about a hardware error. It will be with the disk (HDD?), I think. I recomend you to run fsck on the partition(s?) and use smartmontools to determine the health of the drive. Regards. --- ┏━━┓彡 野宮 賢 mail-to: nomiya @ lake.dti.ne.jp ┃＼／彡 ┗━━┛ "Maddox hopes that empowering users to pick their own algorithms will get them to think more about what’s involved in making them. " -- Bluesky's Custom Algorithms Could Be the Future of Social Media --
Am 31.08.23 um 18:08 schrieb Daniel Bauer:
Hi,
since my upgrade to OS 15.5, when running a command line job for imagemagick I get
I/O error : buffer full
Just to let you know: the problem has disappeared without leaving traces... There were some updates on OS 15.5, it seems they have solved it. -- Daniel Bauer photographer Basel Málaga Twitter: @Marsfotografo (often explicit nudes) https://www.patreon.com/danielbauer https://www.daniel-bauer.com (nudes)
participants (5)
-
Carlos E. R. -
Daniel Bauer -
David C. Rankin -
Masaru Nomiya -
Robert Webb