Soundsoap was a bit of a
PitA when it was a plug-in, but we use it more often now that it is a stand-alone app.
Obviously, the more background noise, the more processing has to be done, making the audio sound tinnier. But we have had acceptable results.
You will rarely get "perfect" results. After all, it is called SoundSoap, not SoundOxyClean.

However, you adjust the slider to trade off between hearing the background noise and making your audio clip sound "artificial."
Also, we have had the best results so far with removing continuous low frequency rumblings, like AC fans.