I bought the 8500/180 when it was new. It never did a 'stellar' job of video editing, but for the time it was decent.
Now you are likely to not get 30fps rates when grabbing full-screen video of 640x480. You should be able to get lower resolution video without a problem.
The issues with the video is related to the system bus maxing at 50 or 55mHz and the internal drive speeds. If you had a fast/wide scsi card in their and decent fast or wide scsi drives you may be able to get the video framerates you want.
To minimize your stuttering, have a clean (freshly erased) drive for when you do grabbing and a better option then would be additionally having the drive on its own scsi bus (like a third party card).
Keep your segments you grab short (no longer than a minute or so), then you can splice or make edits and transitions in software to put it all back together. The longer segments stutter when the drive can't lay down all the data without recalibrating (AV class scsi drives will not stutter in their writing of data during long writes). I didn't fully understand the hardware technical explanation of this, but the a/v drives are set up for this type of work.
Other than that you'd need to upgrade it with higher cost than getting a great machine (iMac) for about the same as upgrades.