Your biggest problem nowadays is from mail server blacklisting - one of the stupidest anti-SPAM techniques ever invented.
For example, say you're a cable modem user and you set yourself up a dynamic DNS address and a mail server.
Guess what? Many mail servers will refuse to accept email from your mail server, because every DHCP IP address used by your cable company has been put on a "blacklist". The maintainers of these blacklists assume that only spammers would run a mail server on a dynamic address.
Like you, I want to run my own email server so I can receive very large attachments and not have to constantly delete email out of my IMAP account. But, finding a service provider that can give me an address that is not blacklisted is extremely difficult. For most service providers, that means moving to a "business" account, which typically start at $250 a month.
Additionally, most service providers (especially the cable providers) have a service agreement that says you cannot run any type of server whatsoever when using their service. They assume that if you want to run a web server, you'll be posting huge warez images rather than perhaps wanting to post a few pictures of your kids.
In short, it's a very frustrating situation for the technically-skilled home user that wants to run a few small servers.
Wade