On 2023/04/19 08:34, Per Jessen wrote:
for iSCSI over ethernet, we experimented with 9000, which is the max both nics and switches would take, but it caused to much latency. Not enough IO.
---- iSCSI didn't look especially efficient for high speed file transfers -- it looked pretty bad because instead of being able to send a large file with a large TCP window, all files seemed to be broken down into iSCSI's idea of a block size. I found 9kb packet sizes gave about 50% more throughput over CIFS/SMB. The limiting factor was CPU and number of send/rcv buffers on client. I had to reduce number of send buffers on a win7 client considerably to get reasonable latency as the send buffers filled up. More send buffers on client resulted in lower cpu load on client, but higher cpu load on linux server. As for latency -- you're probably right, 1.5kb would likely help w/latency, though other param, like coalescing and how long to wait to coalesce might also have a strong effect. Also -- strong effect -- speed of link. Takes alot less time to send a packet (of any size) over a 10Gb link vs. a 1Gb link. Also of note was that combining 2x10Gb links resulted in no throughput improvement *in my setup* due to larger CPU load in recombining/re-ordering traffic. Eventually, I've had to settle for 1.5kb packets for compat with IoT (like media equipment in my living room).