OS X apps should not have problems with <2GBs of RAM. That may have been a limitation in the Windows consumer world until fairly recently, but it's never been a problem on our platform.
Bad memory can manifest itself in various ways, but usually it causes applications to crash or worse. If you have any doubts, take the RAM out and see if the problems continue.