Comparing Blades vs. 1U Systems for Compute Node Use

We use fairly basic compute node configurations --- dual CPU, 2 GB per core RAM (which might be considered large, but is not particularly unusual or specialized configuration in 2008) , simple disk (2 SATA) and network requirements (dual 1Gb/s Ethernet). Currently Intel CPUs and 1U Dell PowerEdge 1950 nodes are providing a good match to our needs and good pricing . Various vendors are selling compute "blades" that are targeted at compute intensive and "scale-out" applications. The primary characteristic of a blade computer is that it is designed to plug into a blade chassis that provides support functions such as DC power supply and air flow. Additionally most vendor's blade systems provide internal communications for management and high speed networks, this eliminates the need to individually cable the compute blades. Each vendor's blade chassis has a proprietary interface so there is vendor "lock-in" involved. However, as we would likely buy fully populated blade systems, future upgrades are unlikely minimizing this issue. A comparison of PE1950 and M600 systems in configurations that we might use is given here. The blade system has some distinct advantages. There are some issues related equipment --- racks, PDUs etc.

Basic Node Configuration

Item	PE1950	M600
CPU	2x Intel 5430 2.66GHz	Same
RAM	8x 2GB DIMM (16GB total)	Same
Disks	2x 750GB 3.5" SATA	2x 146GB 2.5" 10k RPM SAS
Ethernet	2x 1Gb/s external port, Broadcom chip	2x 1Gb/s "internal" port, Broadcom chip
Power Supply	1	n/a
Management	IPMI 2 BMC	???
Dell List price

Attach spec.

Power Comparison

Figures are taken from http://www.dell.com/calc/ The "SPEC" usage level has closely matched measurement made on nodes running our jobs.

Full Racks

The power usage of a full rack of blades is basically the same as for a full rack of 1U systems. So, the input electric currents and airflow requirements are similar as well. The advantage of the blade is that a full rack has 4*16*2 = 128 CPU sockets (512 job slots for quad core CPUs) vs. 80 CPU sockets for the 1U solution. The blade system is providing 48/80 = 60% more CPUs or job slots in the same space. A rack with 3 blade chassis still a higher CPU socket count (96) and reduced power compared to the 1U solution.

The PE1950 rack has 40 nodes, 2 PC6248 switches, APC Door, 2 APC AP7998 PDUs using Hubbell CS8365C outlets

Node Config	c13	c19	total normal amps	total worst amps	normal outlet amps	worst outlet amps
1 pe1950	1	0	2.1	2.5?	2.1	2.5
1 md1000e 3PS	0	3	22	25?	7.3	14?

Input Power Connections

The primary issue is that the power supplies are rated at 14 Amp input power at 208V. This is to much for a C13 outlet. C19 outlets are used (though something like an L6-20R could be used as well). Each chassis needs 3-6 power supplies depending on redundancy desired.

APC solutions...

For 1U compute nodes, we prefer to have switched power outlets as a fall back method for rebooting them. This is not needed for the c19 outlets for the blade chassis. APC offers "basic", "metered" and "switched" power strips, but there are actually more options in the metered type than in the switched type. In particular some of the power strips that are attractive for blade chassis are available in metered type but not switched (makes sense).

-- TomRockwell - 03 Jun 2008