Not sure, but I think Unreal required OS8.6 as a minimum. Might have been OS9, though.
Quake, on the other hand, can be run perfectly well in OS8.1. I do it on my old PowerComputing clone, just for kicks.
The OS you use really depends on the requirements of the apps you want to run. In general, you should be able to run any OS8-or-later app in OS9.