The chipset supports 4GB of physical memory; due to some PCI addressing issues you can only use 3.2GB of it. The Macbook does not require memory to be installed in pairs, it's just a good idea for graphics performance.
But as far as I know, Apple has limited all the Core Duo Macs to 2GB memory with the firmware. The Core 2 Duo Macs support 3GB with the same chipset due to a change in the firmware. It has nothing to do with 64-bit.