bdflush - Streaming Buffer-to-Disk vs. Burst I/O? 11
A not-so Anonymous Coward asks: "I'am trying to sustain major data input&output on a ftpserver. I have been tweaking with /proc/sys/vm/bdflush and buffermem settings, but the IO is still very bursty. Disks are IDE and use lots of CPU when the update decides to flush the dirtybuffers. The opions differ on the net, what can I do to leave some CPU to the NIC while for example 75% is doing the data-to-disk stream? I have read the Documentation/sysctl/vm.txt and /sysctl/proc and a couple of other hints, but no real improvements so far. Please include detailed motivations and maybe even test results."
your time versus your money (Score:4)
Re:Try this... (Score:1)
Try this... (Score:3)
Enable DMA (Score:4)
Using 'hdparm' may help, but only if your kernel has been compiled with the relevant drivers. Without these DMA may work, but not at the optimum performance. You will have to investigate which IDE chipset you are using.
Neglecting to use DMA is the most common setup up problem for IDE hardware accross all operating systems. I know of one major UK company who rolled out Windows95 and NT, and none of the machines had DMA enabled.
A poor choice of NIC can also cause high CPU utilisation. Avoid NE2000 clones at all costs as these use PIO.
Re:Try this... (Score:2)
It is advantageous to run hdparm very early in the boot sequence, ideally before fsck checks the filesystem. Recompiling the kernel to automatically enable DMA, or passing "ide0=dma" as a boot-time option will also help.
Buffer Cache (Score:3)
Asynchronous I/O is helpful if the operating system supports it.
A good bus-mastering SCSI host adaptor will take much of the load off the CPU.
SCSI-IDE (Score:2)
so... check the Arena Array, it's a harware RAID box, with a SCSI interface, but with IDE bays! it uses a separate IDE channel for each drive, so the througput is as high as possible, and the host interface is SCSI, so you can use those low-CPU-demmanding cards.
Re:SCSI-IDE (Score:3)
No, the throughput is not as high as possible. It's better, sure, and it will take a load away from the CPU, which might be sufficient, but not the best. The problem is that each IDE drive can still only service one request at a time; thus, optimally, you can handle n requests for n drives. Since the article specified input and output, you can't do a simple RAID1 system to handle n reads of any data. Instead he has to go with striping, which would lead to having n reads and/or writes distributed across n drives if each request is for data on a different drive.
My advice? If you can afford new hardware, buy the best you can afford; the Arena Array may be sufficient, and it will certainly be better, but you'll need to do some resarch - how many concurrent connections, how are the file requests and uploads distributed, etc. - to see if you can expect an IDE array to fix your problems.
It boils down to either a) the problem has to be solved, and you'll just have to do whatever it takes to fix it (and I don't think you'll fix it with software), or b) you'd like to fix it but it's not that important. In the case of b), hdparm might be your best bet, as well as making sure you have a NIC that doesn't need a lot of hand-holding by the CPU.
Hdparm to the max (Score:5)
Most others here suggest turning dma on, which is obvious, but there are many other things you can do with hdparm to help performance.
This is the command I use on startup:
hdparm -d1 -c1 -u1 -A1 -a255 -m16 -X66 -W1
-d1 -- turn on dma
-c1 -- enable 32bit transfers (helps alot)
-A1 -- enable readahead
-a255 -- set readahead to 255 (maximum on ide drives)
-m16 -- permits the transfer of multiple sectors per interrupt (16 in this case - max for my hd)
-X66 -- set UDMA33 transfer mode. 67 is uata66, 68 is uata100.. dont bank on that, however.
-W1 -- enable write cacheing (this would help you alot) make sure your hard drive is stable with all other settings before trying this, as it's dangerous.
Also try getting the program powertweak for setting all kinds of tuning parameters. There is a powertweak-gtk with descriptive tooltips as well.
Re:Hdparm to the max (Score:2)
-u1 -- a setting of 1 permits the driver to unmask other interrupts during processing of a disk interrupt, which greatly improves linux's responsiveness.
Re:Try this... (Score:2)
Actually it's probably a good idea to fsck your system before making changes to your harddrive setting anyway. ;)
_____________