Well even if the battery is not charging, the power supply circuit in the computer has to manage the incoming power, and that creates heat. So yes, it's normal for the computer to run hotter.
Consider also that Mac OS X's default Energy Saver settings are to run at increased performance on AC power than on battery. This, too, makes a difference.
tooki