No. The Radeon 9600 from the G5 is an AGP Pro card. It will only work in the G5s.
If you're upgrading a G4, I'd suggest an Apple/nvidia GeForce 4 Ti or (though you'd still be CPU limited) a retail Radeon 9800 Pro.
Edit- or if you want a cheaper but still significant bump, find an Apple OEM GeForce 3 (much faster still than the GF4MX).