When the SMS app is left open, and as long as the screen hasn't gone to sleep mode, it'll play a quieter sound, similar to the default sound for an incoming message in iChat. I've missed many SMS messages this way as well. I'd really like to set a custom sound as an alert, but that's another topic.