Currently, I doubt that's possible. It seems that iChat enforces a one-to-one session. Video conferencing monopolizes the video and audio inputs. Audio conferencing monopolizes the audio input. The thing is, I haven't seen many SIP-based clients that offer "true" conferencing. H.323-based applications like MS NetMeeting support this (as the protocol itself supports it), but quality is a far cry from iChat AV + iSight.
You might want to check out
X-Ten's offerings if you want to do multi-party conferencing.