If you've rated the Album but not the songs in them they get assigned the album rating in "hollow" stars. It's stupid that the hollow stars "count" in smart albums, I agree. Unrate the album is the best bet, but I've had the darn things stick around even after.