From your post, it's a little hard to know what the problem might be, but for reference, I've seen OSX Server running on an external UW/SCSI3 Medalist using BootX for switching between 8.5 and X during startup. Point is, it seems unlikely that it's the drive itself... So, I'd start, as it sounds like you did, by reformatting the drive using Drive Setup, but here are three things you should be sure to do:
1. Format for HFS+ -- OSX can't be installed onto an HFS drive.
2. Update the drivers using 1.7.2 after reformatting.
3. Under "Customize" in 1.7.2 select automount on startup (just to be sure)
You might also consider giving the drive a lower SCSI id# but I don't know if this will make much of a difference.
Additionally, is your SCSI card an Apple card (I know you said Adaptec, but did Apple put it in there -- they seem to be very good about supporting their own cards)?
Two more things to think about:
1. If booting from a SCSI disk, Mac OS X Server can only start up from a disk attached to the first SCSI card.
2. http://developer.apple.com/techpubs/macosxserver/macosxserve r.html
And finally, the rest of Apple's online documentation is pretty good and worth checking out and there's a discussion list that might be able to help you out.
Later.