[opensuse] write several usb flash keys at once

jdd

6 Sep 2014 6 Sep '14

19:00

Hello, I received lot of openSUSE usb flash device (8Gb), but empty. writing one at a time with imagewriter is very time consuming (approx 10 mn each). But I have 4 usb ports on my tower's front. Is there a *safe* way to write for device at the same time. using dd makes me fear to destroy one of my hard drive :-( thanks jdd -- http://www.dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Show replies by date

jdd

6 Sep 6 Sep

19:02

Le 06/09/2014 21:00, jdd a écrit :

...

Is there a *safe* way to write for device

_four_ devices (4) sorry jdd -- http://www.dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Anton Aylward

19:24

On 09/06/2014 03:02 PM, jdd wrote:

...

Le 06/09/2014 21:00, jdd a écrit :

...
Is there a *safe* way to write for device

_four_ devices (4)

Four or more http://www.amazon.com/NEEWER-Portable-Expansion-Office-Travel/dp/B003V388Z2 -- /"\ \ / ASCII Ribbon Campaign X Against HTML Mail / \ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Anton Aylward

19:21

On 09/06/2014 03:00 PM, jdd wrote:

...

Hello,

I received lot of openSUSE usb flash device (8Gb), but empty.

writing one at a time with imagewriter is very time consuming (approx 10 mn each).

But I have 4 usb ports on my tower's front.

And I have this thing I can plug into the single USB on mine which has 4 ports. There's a lot of it about.

...

Is there a *safe* way to write for device at the same time. using dd makes me fear to destroy one of my hard drive :-(

Many tools _can_, if you use them incorrectly, damage your hard disk. Using them incorrectly often invovles using root. try this dd if=isoimage.dat of =/dev/.../usb1 bs=.. etc etc etc & dd if=isoimage.dat of =/dev/.../usb2 bs=.. etc etc etc & dd if=isoimage.dat of =/dev/.../usb3 bs=.. etc etc etc & dd if=isoimage.dat of =/dev/.../usb4 bs=.. etc etc etc & ... go have a coffee ... for appropriate output ports. Perhaps you could chown them so that all of the above can be run by a non-root user. -- /"\ \ / ASCII Ribbon Campaign X Against HTML Mail / \ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

jdd

20:48

Le 06/09/2014 21:21, Anton Aylward a écrit :

...

dd if=isoimage.dat of =/dev/.../usb1 bs=.. etc etc etc &

the good idea, here is to use the usb device name (versus /dev/sdx one), so much less risk to mix it with the main disk name. I have one archive usb 4Tb driven but it's easy to unplug it during the process. using usb hubs is not a good idea, there are shared bandwith and often probably collisions of sort of, because the write can be incredibly slow, or device not working at all. thanks (all the people that answered :-) jdd -- http://www.dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Anton Aylward

21:02

On 09/06/2014 04:48 PM, jdd wrote:

...

using usb hubs is not a good idea, there are shared bandwith and often probably collisions of sort of, because the write can be incredibly slow, or device not working at all.

LOL! Its always a trade off. In the limiting case a couple of dozen hubs and set up all 40 to run in parallel, the go out, have dinner, see a movie, spend the evening with your significant other, and come back the next day. It beats hanging around doing the swap-restart cycle over and over. You have to figure out not just elapsed/clock time but the time you spend doing and waiting. Personally I prefer going off and doing something else, even if its only reading a book or cooking a meal, and letting the 'incredibly slow' stuff grind through, to sitting at the terminal waiting, waiting, and doing the swap-restart step and repeat one by one, even if that does each unit faster. And it may be that the elapsed time for 4 in parallel, or 7, 8, 10 in parallel is the same as the elapsed time one at a time, but doing the setup all at once saves you time. -- /"\ \ / ASCII Ribbon Campaign X Against HTML Mail / \ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

jdd

7 Sep 7 Sep

08:02

Le 06/09/2014 22:48, jdd a écrit :

...

Le 06/09/2014 21:21, Anton Aylward a écrit :

...
dd if=isoimage.dat of =/dev/.../usb1 bs=.. etc etc etc &

the good idea, here is to use the usb device name (versus /dev/sdx one), so much less risk to mix it with the main disk name. I have one archive usb 4Tb driven but it's easy to unplug it during the process.

using usb hubs is not a good idea, there are shared bandwith and often probably collisions of sort of, because the write can be incredibly slow, or device not working at all.

finally I did the job with a hub. I have a 7 ports powered usb ub. I could only plug 6 openSUSE usb flash devices (a bit too thick for the 7th). Using dd bs=4096 if=/data/telechargements/download/isos/openSUSE-13.1-DVD-x86_64.iso of=/dev/disk/by-id/usb-General_USB_Flash_Disk_05311100000000A8-0\:0 & I could write 6 device at the same time, in approx 20 mn, including typing the command. Using tab completion is very usefull. The device name seems to be this, the real device number, so no one is the same as the other. one needs to *not* use the "part1" part (two choices, with and without). It's pretty obvious, but the iso have to be written on the disk, not the first partition. I sticked on the device a small round blue sticker with written "34" to remember what is on it :-) I only wrote 12 of them, next big meeting is mid November, and I expect to use factory then (or 13.2 if available in time). Factory installs very well. thanks, everybody :-) jdd -- http://www.dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

jdd

08:04

Le 07/09/2014 10:02, jdd a écrit :

...

I could write 6 device at the same time, in approx 20 mn, including typing the

and... do not expect to work on your computer during the write, 6 writes makes a huge stress on the system (i5, 8Gb ram). jdd -- http://www.dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R.

12:08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2014-09-07 10:02, jdd wrote:

...

Le 06/09/2014 22:48, jdd a écrit :

...

Using

dd bs=4096 if=/data/telechargements/download/isos/openSUSE-13.1-DVD-x86_64.iso of=/dev/disk/by-id/usb-General_USB_Flash_Disk_05311100000000A8-0\:0 &

Other concoction: cp /data/.../openSUSE-13.1-DVD-x86_64.iso /dev/disk/by-id/usb-...

...

one needs to *not* use the "part1" part (two choices, with and without). It's pretty obvious, but the iso have to be written on the disk, not the first partition.

Of course! Where did you get the idea that you could write one of the partitions? It is always, and always has been, write the whole iso file to the whole destination device. On 2014-09-07 10:04, jdd wrote:> Le 07/09/2014 10:02, jdd a écrit :

...

...
I could write 6 device at the same time, in approx 20 mn, including typing the

and... do not expect to work on your computer during the write, 6 writes makes a huge stress on the system (i5, 8Gb ram).

I would indeed expect to be able to continue using the computer, it should be no stress at all. USB 2 can transfer 480 Mbit/s maximum and total, about 50 MB/s. A good hard disk can provide double than that, so the slow component here is the USB. The rest of the time the system is just waiting for the USB to be ready again, which on a good design means doing something else (assuming that the chipset does "command and forget" things without CPU). However, a good motherboard can have multiple USB controllers, and each one can provide the max speed on the bus connected to it... Meaning that the total speed can go way over that 50MB/s in that configuration. Then, the internal/main hard disk would be the slow component in the mix, and the system would crawl... But you were using a hub, so this can not be the case at all. In that case, I would do the copy process with both nice and ionice in the line: nice ionice -c Idle cp /data/.../openSUSE-13.1-DVD-x86_64.iso \ /dev/disk/by-id/usb-... It will take longer, but you can use the machine. It the cpu load maxes (it should not) then you can use "cpulimit" instead. But considering you were using a hub, thus a single controller, I see no reason for the impact on the machine. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iEYEARECAAYFAlQMSqAACgkQtTMYHG2NR9U67gCcCIwGFg6GI8Vz/8l+MyfLiBAy D3kAoIVm7pUGwbkW6cAk9jC3woCdeFJc =68PU -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

jdd

12:37

Le 07/09/2014 14:08, Carlos E. R. a écrit :

...

Of course! Where did you get the idea that you could write one of the partitions?

because it's where it's usually mounted. Of course It's stupid for an iso, but I made the mistake at first :-( It is always, and always has been, write the whole iso

...

file to the whole destination device.

On 2014-09-07 10:04, jdd wrote:> Le 07/09/2014 10:02, jdd a écrit :

...
...
I could write 6 device at the same time, in approx 20 mn, including typing the

and... do not expect to work on your computer during the write, 6 writes makes a huge stress on the system (i5, 8Gb ram).

I would indeed expect to be able to continue using the computer, it should be no stress at all.

no reason, it is. Simply typing the mail was difficult :-(

...

However, a good motherboard can have multiple USB controllers,

here it was to an external usb ub, from one of the front plug.

...

But considering you were using a hub, thus a single controller, I see no reason for the impact on the machine.

I don't see the reason, but I see the impact :-( jdd -- http://www.dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

James Knott

6 Sep 6 Sep

19:36

On 09/06/2014 03:00 PM, jdd wrote:

...

Is there a *safe* way to write for device at the same time. using dd makes me fear to destroy one of my hard drive :-(

Using dd to write to a USB device will not affect your hard drive. Just create the command with a & at the end, so you can get multiple sessions going. Start one, then recall the command, make the changes for the 2nd, etc. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Cristian Rodríguez

19:46

El 06/09/14 a las #4, jdd escribió:

...

Hello,

I received lot of openSUSE usb flash device (8Gb), but empty.

writing one at a time with imagewriter is very time consuming (approx 10 mn each).

But I have 4 usb ports on my tower's front.

Is there a *safe* way to write for device at the same time. using dd makes me fear to destroy one of my hard drive :-(

thanks jdd

Using dd you can do that.. it will not destroy your hard-disk as long as you do not mistype the device name.. that said.. the USB bandwidth is not dedicated but shared across all devices connected to the controller..so it may or may not be faster than writing one device at time. -- Cristian "I don't know the key to success, but the key to failure is trying to please everybody." -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

jdd

20:44

Le 06/09/2014 21:46, Cristian Rodríguez a écrit :

...

Using dd you can do that.. it will not destroy your hard-disk as long as you do not mistype the device name..

that's the point :-) and the fact that I don't have only 4 device to write, but 40 (approx, for four it wouldn't be necessary to make complicated things), and that I'm not sure to get the same device name each time I plug/unplug a device, or will I? that said.. the USB bandwidth is

...

not dedicated but shared across all devices connected to the controller..so it may or may not be faster than writing one device at time.

I hope to have at least two controllers with 2 ports each jdd -- http://www.dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R.

22:37

On 2014-09-06 21:46, Cristian Rodríguez wrote:

...

El 06/09/14 a las #4, jdd escribió:

...

Using dd you can do that.. it will not destroy your hard-disk as long as you do not mistype the device name.. that said.. the USB bandwidth is not dedicated but shared across all devices connected to the controller..so it may or may not be faster than writing one device at time.

But, considering that a flash stick writes much slower than what what the USB can send, connecting a dozen sticks and writing to all of them simultaneously is actually faster ;-) Otherwise, you can connect the forty, and create a small script that writes ten of them, waits, then writes the next 10, etc - as long as you have a big enough hub, of the powered kind. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)

jdebert

7 Sep 7 Sep

01:41

On Sun, 07 Sep 2014 00:37:02 +0200 "Carlos E. R." wrote:

...

On 2014-09-06 21:46, Cristian Rodríguez wrote: But, considering that a flash stick writes much slower than what what the USB can send, connecting a dozen sticks and writing to all of them simultaneously is actually faster ;-)

Multiple devices can be written to/read from simultaneously on a serial bus? Is that possible or even legal with the USB protocol? It does not appear to be occurring on any system here. Reads & writes consistently appear to be occurring one at a time. There's only one physical data path and that happens to also be the command path. How do you configure a USB system to read/write multiple devices simultaneously? How do you avoid collisions? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R.

02:38

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2014-09-07 03:41, jdebert wrote:

...

On Sun, 07 Sep 2014 00:37:02 +0200 "Carlos E. R." wrote:

...
On 2014-09-06 21:46, Cristian Rodríguez wrote: But, considering that a flash stick writes much slower than what what the USB can send, connecting a dozen sticks and writing to all of them simultaneously is actually faster ;-)

Multiple devices can be written to/read from simultaneously on a serial bus? Is that possible or even legal with the USB protocol? It does not appear to be occurring on any system here. Reads & writes consistently appear to be occurring one at a time. There's only one physical data path and that happens to also be the command path. How do you configure a USB system to read/write multiple devices simultaneously? How do you avoid collisions?

Why do you think you have to do anything special? I have connected half a dozen disks simultaneously and write to them /simultaneously/. As far as things in computing occur really simultaneously, because the CPU can only do *one* thing at a time (unless you have several cpus). Look, basically you send, via the USB (a serial bus) a bunch of bytes to write somewhere on some disk, and milliseconds later, while that device is doing the actual write, it sends another (or the same, doesn't matter) bunch of bytes to another disk. And another bunch to another disk, etc. And command sequences, and reads commands from the devices. The devices have identifiers, or addresses. Whatever. To me, user or root, it does not matter at all if the USB is a serial bus or parallel bus, or radio, or fiber, or psy power. To me it only matters that I have connected to the computer, /somehow/, several "disks", each one named differently: /dev/sdf, /dev/sdg, /dev/sdh, /dev/sdi, /dev/sdj, etc. That's all that matters to me. I can have many dolphin windows, or nautilus, or midnight commander, or "cp" in many terminals, each copying the same or different files to any of the "disks" or sticks in any direction whatsoever, any order, simultaneously or not. And it /looks/ simultaneous because each device takes some time to do each write operation of 'n' bytes, and during that time the "bus" can talk and write to another device, instead of just waiting. You do not send data to write as single byte then wait. You send a block of some size (I do not remember how many), probably whole sector/block, as a stream of bytes, which is probably stored on a local small ram on the device, and then the bus is free for another operation with a different device. After some time I poll (ask) the device for the result of the write operation. Or the device tells me. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iEYEARECAAYFAlQLxR8ACgkQtTMYHG2NR9UpVQCdEpPi3wt53q+TX3ij2CEze86e uAcAn26Ov2AGVTUm8rQsT8fH4zhmIKJ4 =nNBQ -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Anton Aylward

12:35

On 09/06/2014 09:41 PM, jdebert wrote:

...

On Sun, 07 Sep 2014 00:37:02 +0200 "Carlos E. R." wrote:

...
On 2014-09-06 21:46, Cristian Rodríguez wrote: But, considering that a flash stick writes much slower than what what the USB can send, connecting a dozen sticks and writing to all of them simultaneously is actually faster ;-)

Multiple devices can be written to/read from simultaneously on a serial bus? Is that possible or even legal with the USB protocol?

Obviously not 'simultaneously' in the exact meaning of the word, but simultaneously in the same sense that an un-switched Ethernet backbone allows many devices to communicate 'simultaneously'; the same way a cell phone tower allows many phone users to talk 'simultaneously', the same way your wifi router allows many devices to communicate 'simultaneously'.

...

Reads & writes consistently appear to be occurring one at a time. There's only one physical data path and that happens to also be the command path.

Just like .... oh never mind.

...

How do you configure a USB system to read/write multiple devices simultaneously?

Perhaps there is a broadcast mode just like with TCP/IP. I don't know. However, at http://www.totalphase.com/support/articles/200349256-USB-Background I read <quote> A USB 2.0 host broadcasts information to all the devices below it. Low-speed and high-speed enabled devices will only see traffic at their respective speeds. Full-speed devices can see both their speed and low-speed traffic. USB 2.0 works through a unidirectional broadcast system. When a host sends a packet, all downstream devices will see that traffic. If the host wishes to communicate with a specific device, it must include the address of the device in the token packet. </quote>

...

How do you avoid collisions?

See above. It seems to me that a host sending packets 'downstream' avoids collisions in the same way that a machine gun sending bullets 'downstream' manages to avoid collisions between its bullets. Sometimes, though, the packets go faster than bullets. Sort of like Superman, eh? -- Life is like an analogy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R.

18:58

On 2014-09-07 14:35, Anton Aylward wrote:

...

...
...
How do you avoid collisions? See above. It seems to me that a host sending packets 'downstream' avoids

collisions in the same way that a machine gun sending bullets 'downstream' manages to avoid collisions between its bullets. Sometimes, though, the packets go faster than bullets. Sort of like Superman, eh?

:-) See http://en.wikipedia.org/wiki/Carrier_sense_multiple_access_with_collision_de... http://en.wikipedia.org/wiki/Carrier_sense_multiple_access_with_collision_av... Then you have time division multiplexing. A token ring network could be seen as an example of that, but not really. Rather an evolution of it. Or other bus arbitration techniques. That's for digital world. On the analog world, you can use frequency multiplexing (AKA tuning wheel in a plain kitchen radio). Same thing as used on cable networks carrying many analog TV channels. Similar to what is done on copper phone lines to transmit ADSL. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)

3527

Age (days ago)

3528

Last active (days ago)

List overview

Download

17 comments

7 participants

participants (7)

Anton Aylward
Carlos E. R.
Carlos E. R.
Cristian Rodríguez
James Knott
jdd
jdebert