If the Mac decides it needs to do a RAM test, you will experience these exact symptoms. After a few minutes, depending on how much RAM you have and how fast the CPU is, the system should boot.
If it doesn't boot after five or ten minutes, you can safely assume the cause is something else. Could be the on-board PRAM battery is dying. If this is the case, usually holding down the keyboard PRAM reset sequence will get it to boot. Eventually, though, this too will fail (when the battery is sufficiently discharged).