it could be either, depending on how much your headphones can take (ie. whether they're the stock headphones or aftermarkets which might be able to handle more, and how old / worn they are). amps can often put out a lot more volume than they can produce cleanly, without excessive distortion. cranking it all the way up may well have been pushing the amp beyond what it can produce without getting a bit wobbly.
as for damaging it, i'm not sure if that's likely, though i really couldn't say... probably very little chance, i reckon, at least unless you're going nuts on the thing. if the amp is producing lots of distortion there's more chance of damaging the phones though, i believe.
sminch