[mduell]

If you combine Intel's chipset docs ("The memory interface for the Intel 5000X MCH supports four memory channels with four Fully Buffered DIMMs per channel, enabling a total system memory bandwidth of up to 17 GB/second on a 64 GB total capacity.") with Apple's chipset diagram (below), it appears that the optimal configuration (for bandwdith) is to populate one FB-DIMM in each channel (which would be every other slot on both cards).



Is it even possible to only have a single FB-DIMM in each channel?
Is it correct to say that for maximum bandwidth you need at least one FB-DIMM in each of the four channels?
Since FB-DIMMs are chained in serial, rather than parallel like dual channel DDR, does having a second FB-DIMM in each channel help?

[P]

Do you mind linking those chipset docs? I was wondering that myself.

Apple eventually puts out a technote explaining details like this, but for now we'll have to make do with this quote:

Mac Pro uses 667MHz DDR2 fully buffered ECC memory, a new industry-standard memory technology that allows for more memory capacity, higher speeds, and better reliability. To take full advantage of the 256-bit wide memory architecture, four or more FB-DIMMs should be installed in Mac Pro.

It suggests that your assumptions are correct. I found it in the Applestore, in the "Learn More" text that shows up under Memory on the configure page for the Mac Pro.

[mduell]

Originally Posted by P

Do you mind linking those chipset docs? I was wondering that myself.

http://www.intel.com/design/chipsets...umentation.htm

I'm sure you have to fill all 4 channels for max performance. My only real question remaining is if you have to put a pair of FB-DIMMs in each channel, or if one is possible.

[OldRocketGuy]

Let us compare the Mac Pro's memory architecture with Dell's dual dual core (quad) boxes. The Mac Pro is 256 bits wide 667MHz DDR2 vs. Dell's what? If the Dell's don't match the I/O bandwidth of the Mac Pro, then the Mac Pro has a definitely superior architecture. Is Dell even using Woodcrests?

[mduell]

Originally Posted by OldRocketGuy

Let us compare the Mac Pro's memory architecture with Dell's dual dual core (quad) boxes. The Mac Pro is 256 bits wide 667MHz DDR2 vs. Dell's what? If the Dell's don't match the I/O bandwidth of the Mac Pro, then the Mac Pro has a definitely superior architecture. Is Dell even using Woodcrests?

Why bother comparing? They're identical except the Dell workstations have 16 slots on 4 channels instead of 8 slots on 4 channels, so you can have 64G in the Dell but only 32G in the Mac Pro.
Of course Dell is using Woodcrests. They've were shipping them before Apple even announced the Mac Pro.

Don't hijack my thread with your trolling.

[Spliffdaddy]

Originally Posted by mduell

http://www.intel.com/design/chipsets...umentation.htm

I'm sure you have to fill all 4 channels for max performance. My only real question remaining is if you have to put a pair of FB-DIMMs in each channel, or if one is possible.

You can use a single (or 3) DIMM but you won't get the benefit of dual (um, 'multi-channel', rather) channel operation - on either bank if you use 3.

While I can't confirm this fact firsthand - my source for this information has never been wrong. She has a machine with the same specs that was running 3 DIMMs at one time.

[mduell]

Originally Posted by Spliffdaddy

You can use a single (or 3) DIMM but you won't get the benefit of dual (um, 'multi-channel', rather) channel operation - on either bank if you use 3.

While I can't confirm this fact firsthand - my source for this information has never been wrong. She has a machine with the same specs that was running 3 DIMMs at one time.

That's how it was with DDR/DDR2 DIMMs. With FB-DIMMs you have channels made up of modules, rather than banks made up of channels (or Intel just decided to switch terminology on me). The FB-DIMMs are chained in serial within a channel, and the channels run in parallel; thus 4 channels with 64-bit wide modules makes a 256-bit wide memory bus. I could be totally wrong and confused, but what you're saying doesn't jive with what I've read.

Does your source actually have a Xeon 51xx setup with FB-DIMMs? How were the 3 FB-DIMMs arranged? All 3 in one channel, 2 in one channel 1 in another, or all 3 in different channels?

[thetman]

hope this doesnt break any rules

[Tuoder]

I wonder if it will work in quad channel with different amounts of RAM in each channel. I'd bet it would, but I wonder.

[mduell]

Originally Posted by thetman

hope this doesnt break any rules

I'm probably does, but thanks!

I've talked to a few people who bought them; as soon as they get their hands on them, I'll ask them to play around with the memory setup and see what boots and what doesn't.

[P]

Apple did indeed release a technote on the memory expansion in the Mac Pros. It notes that bank 1 and 3 are on the same bus and bank 2 and 4 on the same bus. If you put that together with the picture thetman provided, it's obvoius that putting one DIMM on each bus is acceptable - of the four combinations, only the last one fills all the buses.

[Gossamer]

Perhaps this is why Apple makes the 2GB option on the Pros 4x512MB instead of 2x1GB, that way the multi-channel advantages are realized.

[Simon]

From http://developer.apple.com/documenta...608/index.html



Judging by this image I'd say you install in pairs starting with A1A2, then A1A2B1B2, A1A2B1B2A3A4, A1A2B1B2A3A4B3B4.

[Edit: Reduced image size so it will even fit on your ColorClassic's screen ]

[mduell]

Ah-ha, they played with the memory slot arrangement.

Only question left is if you need to fill both channels on a branch or not. IOW, will a Mac Pro boot with only one FB-DIMM?

I'm happy to see the optical drives remain ATA/100; SATA optical drives are available, but expensive and limited.
Also curious that they show the two exra SATA busses (the southbridge supports 6) terminating to no where... perhaps it's a hint that eSATA is coming soon.

Unfortunate that all four Firewire ports are bottlenecked over a single 133MBps PCI bus.

Any chance you could post the PCI Express section? I'm curious about the configurable bandwidth setup.

[Simon]

Originally Posted by mduell

Also curious that they show the two exra SATA busses (the southbridge supports 6) terminating to no where... perhaps it's a hint that eSATA is coming soon.

I saw that too. eSATA would be very nice.

Any chance you could post the PCI Express section? I'm curious about the configurable bandwidth setup.

Not much there yet.

PCI Express Architecture
The Mac Pro has four internal, 2.5 GHz, PCI Express links connected to the North Bridge IC and South Bridge IC. The PCI Express slots are system and user configurable. The Mac Pro’s standard configuration is one 16-lane, double-wide graphics slot, two 4-lane expansion slots, and one 1-lane expansion slot.

[OreoCookie]

@Simon
You're cutting it close here, the diagram barely fits my browser window

[Simon]

Originally Posted by OreoCookie

@Simon
You're cutting it close here, the diagram barely fits my browser window

I'm sorry. I know I'm really pushing it.

That said, you need to get yourself a decent screen. On my 23" ACD the diagram fits about four times.

[Simon]

Wait a moment. After all, there's imageshack...

Fixinated.

[OreoCookie]

Originally Posted by Simon

I'm sorry. I know I'm really pushing it.

That said, you need to get yourself a decent screen. On my 23" ACD the diagram fits about four times.

I have a MacBook Pro with plenty of screen estate. I just prefer small browser windows (lots and lots of windows!)

BTW, I don't want to be a pain, but please shorten your sig to four lines. Thanks

[mduell]

Two 4x and a 1x? That's ridiculous. Even the PowerMacs had two 4x and an 8x.
The Intel southbridge has got to have more lanes available than that.

[MaxPower2k3]

I thought I read somewhere that the PCIe controller had 40 lanes total; 16 for the video card and the rest configurable between the rest of the slots.

[mduell]

After doing a bit more reading, the memory performance deal is this: 4 FB-DIMMs is the optimum configuration; this maximizes memory bandwidth while minimizing latency. With 2 FB-DIMMs you get the same latency but half the bandwidth. With 6 or 8 FB-DIMMs you get the same bandwidth but higher (at least 3-5ns) latency.

I still don't know about having an odd numer of FB-DIMMs.

Originally Posted by MaxPower2k3

I thought I read somewhere that the PCIe controller had 40 lanes total; 16 for the video card and the rest configurable between the rest of the slots.

That was my guess, based on past chipsets. The Mac Pro actually has 44 PCIe lanes: 16 for the graphics slot, 8 on each the northbridge and southbridge to talk to each other, and 12 for the remaining 3 slots.

[Simon]

Umm, actually you also lose one lane to the AP module. Apple gives these possibilities:
default: 16-1-4-4
optional: 8-8-1-8, 8-8-4-4, 16-1-1-8

16 (GPU) + 2x8 (NB-SB) + 1 (AP) + 10 (max in slots 2-4) = 43. Did they 'waste' a lane somewhere?

http://developer.apple.com/documenta...inkElementID_3

[mduell]

Weird... for some reason they don't support 2x slots at all.

[Simon]

Yeah, I noticed that too.

[mduell]

Someone tested it, and yes, the Mac Pro will boot with an odd number of FB-DIMMs.

Thanks tobyg!