Eugene Leitl wrote:
> Dan Clemmensen writes:
>
> > Again, look at the usable bandwidth per node for the given solutions.
> > It's below 100Kbps. Radio is wonderful, but I want 100Mbps per node,
> > and I want a high node density.
>
> A full-duplex FastEthernet link gives you roughly twice that/port.
> Using multiple NICs/node gives you even more bandwidth, and one which
> scales up linearly with node size (ditto disk bandwidth, if each
> node is equipped with individual disk(s)). Multiple-port NICs, even
> with switching functionality, are not unprecendented. In fact, I'll
> build my 8-node Beowulf(cub) switchless, with at least 3 NICs/node
> early next year. Much unexploited potential lies hidden in U-Net
> and 3D Now! SIMD instructions as well. Of course you can start
> instantly with MPI/PVM and PVFS, and use the cluster as a vanilla
> parallel supercomputer, using the advanced features peu a peu.
>
We are in voilent agreemment. wire and fiber beat radio by orders
of magnitude wher they can be used. Therefore, use them when you can.
My gut feeling is that two NICs per node is the optimal number unless
the problems you are working on requires relatively high IPC/cycle ratio.
For such problems, you'll have trouble no matter what.