April 2001 - Packet Writing - openSUSE Mailing Lists

packet-0.0.2i
by Jens Axboe 27 Apr '01

27 Apr '01

Hi, Woo, final 0.0.2i happened. Complete changelog, sort of: - Fix PKT_BUF_LIST error in fs/buffer.c - Fix pkt_find_dev in two spots, use q->queuedata in pkt_elevator_merge. - introduce pkt_make_request as READ remapper (and eventually add block remapper for WRITEs too). - Loose READ/WRITE checks in pkt_elevator_merge and pkt_request, they don't deal with anything but WRITEs after above change. - Fix pending request count in proc reading. - Fix increment usage bug for failed FMODE_WRITE opens. - Loose pd_lock/unlock, use io_request_lock for now. When block switches to per-queue lock, a search/replace operation will be sufficient. - Start to handle buffers non-uniform size buffers in a WRITE request. - Browse buffer cache hash table for possible bh victims - Fix blk_started_io to account buffers from buffer cache too - Do hole merging in regular elevator path - Fix bh_rq_in_between to look in the right direction - Remove sequencing from pkt_elevator - Use bh_cachep slab cache for private pool of buffer_heads - Remove old BH_Packet flag - Default udf to 2kB blocksize - Remove __getblk again - Remove rq->cmd checks in merge functions, only WRITEs now - Fix io_request_lock lockup when used with ATAPI (req->end_io) - Throttle down write when failed and retry, should write OPC entry when this happens - Use current time for b_flushtime in buffer.c for packet device, we don't want to postpone flushing a buffer if it causes read gathering - Remove all bh->b_private references, we could be stomping on other users (highmem, most notably). - Always use pkt_end_io_write as our bh->b_end_io callback - Added write caching config option (don't use yet) - Fixed missing brelse() for buffers stolen from buffer cache - Combine pkt_init_bh and pkt_recount_segments to only do one scan of request (pkt_init_rq) - Fix dupe buffer_head in pkt_hole_merge - Change buffer_head sequence check in ll_rw_blk, add to SCSI end_request handling - Missing MOD_DEC_USE_COUNT on failed bh list grow - Replace more pkt_find_dev with pkt_devs array lookup - Fix lockup on non-uptodate WRITE on ATAPI - Refile now cleaned dirty buffer when gotten from buffer cache - Add buffer/page cache hits statistics - Fix wait queue race due to unplug - Kill requests on buffer errors - Fixup sparc64 for new stats - Remove get_request_wait timeout, bug has been fixed - Cleanup rq clearing with pkt_end_request - Fix s/1/0 return from pkt_handle_queue on already busy - Cleanup pkt_request - Move default speed from 8 to 16 - Fix cleanup on error pkt_new_dev - Fixed nasty missing bh->b_reqnext list detachment in pkt_gather_data (solid hang, and possible corruption) - Only alloc PACKET_MAX_BUFFERS - 1 for freelist, save ~2kB mem - Remove pending bh count - Error compile on CONFIG_SCSI_DEBUG_QUEUES set, until fixed nicely - Fix pd locking, rely less on io_request_lock - Make sure only to try to throttle once per failed write - Remove kcdrwd startup/kill messages - Handle holes sanely, remove ll_rw_blk changes for that - Put pkt buffer on non-uptodate read *.kernel.org/pub/linux/kernel/people/axboe/packet/ or from the sourceforge page. -- Jens Axboe

1 1

Again "attempt to access beyond..."
by Ludovico Cavedon 26 Apr '01

26 Apr '01

I took a new cdrw, no damaged sectors, but the appmts to write beyond the devoce limit came back again. "df" reposrts size correctly, but log messages say "limit=2". Here is df output: -------------- Filesystem 1k-blocks Used Available Use% Mounted on /dev/hda4 6281212 3988236 2292976 64% / /dev/hda1 7746 7594 0 100% /boot /dev/hdc1 4210744 4086420 124324 98% /mirrors /dev/pktcdvd0 448320 34 448286 1% /cdrom ------------------ Here cdrwtool -i output: ------------------ 1148KB internal buffer setting write speed to 12x DISC INFO: erasable : Yes border = 3 Disc status = 2 number of first track = 1 number of sessions = 1 number of tracks = 1 status of last track = 1 uru = 0 did_v = 1 dbc_v = 0 disc type = 32 disc_id = 420582 OPC entries = 1 TRACK INFO: Track 1 track_number = 1 session_number = 1 damage = 0 copy = 0 track_mode = 7 Rt = 1 blank = 0 packet = 1 fp = 1 data_mode = 2 lra_v = 0 nwa_v = 1 track_start = 0 next_writable = 2560 last_recorded = 0 free_blocks = 0 packet_size = 32 track_size = 226912 (453824KB) ------------------------ My log attached. Bye, Ludovico Cavedon lumaca82(a)iniwind.it

2 1

Kernel panic with 0.0.2i-pre3
by Andreas Nowack 26 Apr '01

26 Apr '01

Hello, I tried 0.0.2i-pre3 and got a kernel panic (0.0.2i-pre1 worked fine). I tried to copy the kernel sources to a fresh CDRW. After a short while I got a kernel panic (syslog attached). After a reboot I had to write a new udf structure to the CDRW. "mkudf /dev/pktcdvd0" worked fine with 0.0.2i-pre3. I did a second attempt which also triggered a kernel panic (see second syslog). In the first case I found the message "kernel BUG at pktcdvd.c:1046!", while in the second case this message didn't appear. Best regards, Andreas

2 3

Re: [packet-writing] Results with 0.0.2i-pre2
by Ludovico Cavedon 25 Apr '01

25 Apr '01

On Wed, Apr 25, 2001 at 11:26:59AM +0200, Jens Axboe wrote: > On Tue, Apr 24 2001, Ludovico Cavedon wrote: > > lost+found is empty. Where are those 320Mb??? > > Good question :-). Did you get any UDF warnings in your logs? No, all was normal. But DirectCD sandisk reported damaged sectors... I'll try another cdrw. > > What sequence of commands of cdrwtool is equal to the "-q" option? > > Something like first a blank, then a format with fixed packet size set, > 32 block per packet, mode2 write mode. > > In commands, ala > > # cdrwtool -d /dev/sr0 -b > # cdrwtool -d /dev/sr0 -p -z 32 -s -m -u Thanx, bye, Ludovico Cavedon lumaca82(a)inwind.it

1 0

Results with 0.0.2i-pre2
by Ludovico Cavedon 25 Apr '01

25 Apr '01

Hi, I'm interested in packet writing and I've benn following thil ML for some weeks. Now I had time to try it on my computer. I followed the instructions. 1.Quick setup of a cd-rw 2.Mounted and unmounted it without writing (I hope the messages it wrote are only debug stuff, not errors...) 3. Mounted it again and copy 20 Mb in three files (sorry, my writer is only a 2x and i had not the time to thy the source kernel tree) 4. Unmounted The data is all readable, through windows too. But data is corruped: i use cmp to compare the files and they were different. I've attached the log. Bye, Ludovico Cavedon lumaca82(a)inwind.it

3 2

please test
by Jens Axboe 25 Apr '01

25 Apr '01

Hi folks, Could everybody with a cd-rw drive try the following program, and see if works (ie produces lba output) or if it complains about I/O error (and dumps a kernel message saying so, please send that too btw). axboe@burns:~/proggy $ sudo ./trigger_rahead /dev/hdc /dev/hdc: trigger LBA 1024, ra LBA 1050 Thanks -- Jens Axboe

10 16

packet-0.0.2i-pre3
by Jens Axboe 25 Apr '01

25 Apr '01

Hi, Next pre is up, changes since pre2: - Remove rq->cmd checks in merge functions, only WRITEs now - Fix io_request_lock lockup when used with ATAPI (req->end_io) - Throttle down write when failed and retry, should write OPC entry when this happens - Use current time for b_flushtime in buffer.c for packet device, we don't want to postpone flushing a buffer if it causes read gathering - Remove all bh->b_private references, we could be stomping on other users (highmem, most notably). - Always use pkt_end_io_write as our bh->b_end_io callback - Added write caching config option (don't use yet) - Added UDF empty block querying option - Fixed missing brelse() for buffers stolen from buffer cache - Combine pkt_init_bh and pkt_recount_segments to only do one scan of request (pkt_init_rq) - Fix dupe buffer_head in pkt_hole_merge - Change buffer_head sequence check in ll_rw_blk, add to SCSI end_request handling and it's been merged to 2.4.4-pre6. I'll hang on for final 0.0.2i until 2.4.4 is really released. *.kernel.org/pub/linux/kernel/people/axboe/packet/ As usual, there's a pre2 -> pre3 ugrade patch too. However, this one doesn't catch pre5 -> pre6, so I suggest getting 0.0.2i-pre3 from scratch. SCSI (Plextor 12/4/32) and ATAPI (Sony CRX100E) tested to the extent that large file copying was verified correct (100MB random data file), untarring 2.2.19 kernel source (and compiling it!), and various small files. No data loss of corruption detected, and it was all checked down to the very last byte. ATAPI used DMA mainly, PIO has also been tested good on this drive (just a large file copy, sanity verify). Note: although stuff like copying/untarring a kernel tree seems like fun, it isn't really a decent way to use packet writing. The pktcdvd module will do all it can to merge small writes together, but for decent performance one really should tar up small files before writing them to disc. For example, doing a make dep run on the 2.2.19 source ends up reading half the stuff from disc. Some writers are better at doing this quickly than others, the Sony beats the Plextor hands down in this sort of test (turning off laser, reading blocks, turn on and write). For the 100MB file copy, read gathering is almost exclusively due to getting the remaining parts of the packet that the UDF sparing table is written to. This is something we can't avoid (the WRITE), however I have plans for at least pinning the remaining blocks in memory. Right Ben? :-) Then arge file writes should go as fast as packet writing can go on your drive. Don't ask how long it takes to run make dep bzImage on a cd-rw stored kernel :-) (Un) fortunately I haven't had any failed writes on this particular disc just yet, so the write throttling path has not been exercised at all. It could be severly broken :-) Adam, I'll take a look at your scatter gather segment miscount next. I suspect this happens because of a requeue of the request. -- Jens Axboe

3 12

better performance
by Jens Axboe 24 Apr '01

24 Apr '01

Hi, For better performance when doing stuff like untars or kernel tree copies, run something like $ while true; do sync; done in a 2nd term. The reason for the boost is simple -- we want to keep our request queue full at all times, to allow the best merging possible. This usually happens (the request free batching helps a lot), but not always because bdflush doesn't push enough buffers out at once at seemingly random intervals. I've thought about doing pull-flushing before (rather than push flushing as it happens now) for other reasons in fact, however that would solve this issue completely too. More on this soon :-) -- Jens Axboe

1 0

attempt to access beyond end of device
by Jens Axboe 24 Apr '01

24 Apr '01

Hi, Could the folks seeing this error please do: # cdrwtool -d /dev/my_cd_device -i with the disc used inserted, and send me the output? Thanks. -- Jens Axboe

1 0

Results with 0.0.2i-pre3
by Adam Huffman 24 Apr '01

24 Apr '01

Hmm, this time I get the return of some errors I used to get with a much earlier version of the patch. Instead of copying a kernel tree, I tried a 33Mb RPM on a SCSI Zip disk. It did copy, eventually, after the torrent of messages, but it cannot be queried or verified. Here's a sample of the messages: pktcdvd: kernel thread pktcdvd0 started pktcdvd: writer sr0 sucessfully registered pktcdvd: inserted media is CD-RW pktcdvd: Fixed packets, 32 blocks / packet, Mode-2 disc pktcdvd: enabled write caching on pktcdvd0 pktcdvd: speed (R/W) 6/4 pktcdvd: 452288kB available on disc UDF-fs INFO UDF 0.9.3-rw (2000/12/25) Mounting volume '', timestamp 2001/04/24 00:46 (103c) attempt to access beyond end of device 0b:00: rw=0, want=710, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=712, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=714, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=716, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=718, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=720, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=722, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=724, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=726, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=728, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=730, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=732, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=734, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=736, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=738, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=740, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=742, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=744, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=746, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=748, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=750, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=752, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=754, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=756, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=758, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=760, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=762, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=764, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=766, limit=2 attempt to access beyond end of device 0b:00: rw=0, want=768, limit=2 huh, 1416 not uptodate huh, 1420 not uptodate huh, 1424 not uptodate huh, 1428 not uptodate huh, 1432 not uptodate huh, 1436 not uptodate huh, 1440 not uptodate huh, 1444 not uptodate huh, 1448 not uptodate huh, 1452 not uptodate huh, 1456 not uptodate huh, 1460 not uptodate huh, 1464 not uptodate huh, 1468 not uptodate huh, 1472 not uptodate huh, 1476 not uptodate huh, 1480 not uptodate huh, 1484 not uptodate huh, 1488 not uptodate huh, 1492 not uptodate huh, 1496 not uptodate huh, 1500 not uptodate huh, 1504 not uptodate huh, 1508 not uptodate huh, 1512 not uptodate huh, 1516 not uptodate huh, 1520 not uptodate huh, 1524 not uptodate huh, 1528 not uptodate huh, 1532 not uptodate 1416 not uptodate 1420 not uptodate 1424 not uptodate 1428 not uptodate 1432 not uptodate 1436 not uptodate 1440 not uptodate 1444 not uptodate 1448 not uptodate 1452 not uptodate 1456 not uptodate 1460 not uptodate 1464 not uptodate 1468 not uptodate 1472 not uptodate 1476 not uptodate 1480 not uptodate 1484 not uptodate 1488 not uptodate 1492 not uptodate 1496 not uptodate 1500 not uptodate 1504 not uptodate 1508 not uptodate 1512 not uptodate 1516 not uptodate 1520 not uptodate 1524 not uptodate 1528 not uptodate 1532 not uptodate Sorry for sending bad news twice in one evening... Adam

2 1