It looks like BtMM is an Apple provided dynamic DNS service plus authentication handled via .Mac using .Mac credentials rather than from machine to machine using machine authentication.
An encrypted tunnel provided via IPSEC is actually a good idea, as is a challenge/response Kerberos based authentication system. Many enterprise-grade systems rely on Kerberos for authentication, including Microsoft's Active Directory which uses a Microsoft variant.
In short, it seems like BtMM is well designed from a security standpoint.