The two main projects like what you are describing are
MythTV as was mentioned, and
Freevo. There is also Microsoft's Windows Media Center, but I don't think that is what you are looking for. Both MythTV and Freevo are skinable and extensible, so installs can look very different, and even have added features that are not part of the standard install.
Both projects have ports to MacOS X, so it is as easy as installing a binary to get playback running (neither has full support for recording yet, but both are working hard on it... and MythTv has the concepts of front-ends and back-ends so your back-end could be on another box altogether.