As far as I know that is the only way to delete a file, and it makes sense: one song might be on several playlists so just deleting it on a playlist should only remove it there.
The problem is finding it in playlist order in the master list and that isn't too difficult - just add the 'Date Added' column and sort on it. If your songs aren't on multiple playlists they will be in playlist and list order making it easy to delete them.