Date: Wed, 23 Jan 2002 10:38:43 -0600 From: Don Holmgren Subject: Re: Intel 860 PCI bandwidth problem To: Dan Yocum On Wed, 23 Jan 2002, Dan Yocum wrote: > Don Holmgren wrote: > > > > Yup. Both the programmed I/O and the DMA modes suck big time. I found > > a small tweak to the chipset last summer that Myricom put into their > > driver, which boosts the DMA rate from 140/210 MB/sec > > (incoming/outgoing) to 210/300 MB/sec. Still really lousy for 64/66 > > PCI (should be 520/520). Wire speed on Myrinet is about 250 MB/sec; the > > DMA rates throttle the bandwidth down to about 170 MB/sec. > > > I'm assuming this is per PCI bus, or is this the aggregate bandwidth? > > I wonder how this will affect the disk IO rates on our soon-to-be-delivered > storage servers... > > i860-based motherboards usually have a 32/33 bus with four slots, and a 64/66 bus with either two or four slots (though I've never seen a four slot version). The problem is per-bus. My DMA tweak only fixes the 64/66 bus. I've not checked the 32/33 bus, though I know the 850 chipset has a poor 32/33 bus (90 MB/sec throughput vs 130 nominal). Maybe there's a fix for this 32/33 bus. I assume you're using IDE raid controllers again? If so, if they're 64/66 you'll probably be fine. They'll top out at about 200 MB/sec. In the 32/33 bus you'll be stuck at about 90 MB/sec. > > > > In fact, the Myrinet throughput on my dual Xeons is identical to the > > throughput on my dual Athlons, which have 64/33 PCI buses. The Xeons > > still kick serious Athlon butt on our code, however, so I'm about to > > order 48 dual Xeons w/ the 860 chipset.