That picture definitely means "plug into the wall outlet." Some computers don't provide enough power to properly charge an iPod, and the 4th gens (in my experience) want more juice when they're low than when you're just topping off the battery.
If a wall charger doesn't fix the problem, try different outlets. In older buildings I've found "poor" wiring to cause noticably low voltage levels at outlets.