This could be one of 2 things: your GPU or your dongle.
Hopefully it's not the GPU, but if there was something wrong with your video RAM (VRAM), the chip would not be able to pump out anything other than a single display, which is what you are are seeing by not being able to go into dual screen mode and closed lid mode working. This is not likely, but it is a possibility.
More likely is an issue with your DP to VGA dongle. I'd try another one just to make sure that's not the problem.
Finally, are you using software like gfxCardStatus (
http://www.codykrieger.com/gfxCardStatus)?] This tool lets you have more control over switching between integrated graphics and discrete graphics. I've noticed that if I'm using integrated graphics on my 2010 MBP, I can't go into dual display mode either. When I switch to discrete, it works. Normally the system would do this for me automatically, but using this software allows you to force integrated or discrete.
In any case, hopefully this has given you some ideas on things to check.
Steve
Edit: There's even something about this in the FAQ:
http://codykrieger.com/gfxCardStatus/faq