Does Apple System Profiler see the extra drive? If it's a kernel driver problem, you can check by booting into OS9 and see if ASP sees the drive from 9.
Make sure power and data cables are plugged into both drives. A power plug incompletely plugged in will make the drive invisible to the G4.
If you are using a cable-select cable, make sure both drives are jumpered to cable-select. If you are using a standard cable, the drives must be jumpered as Master and Slave.