I think that can be a problem with the mail provider. I once shared my email address across two computers, powerbook and imac. I would use the imac mostly, and mail would work OK. Then I would go use the powerbook and fetch mail with mail.app on it, so when I came back to the iMac it would redownload all the mail on the server again. Probably because the POP server is what keeps track of new mails when your mail.app asks it to send a list of new messages.
Instead you could set mail.app to delete messages from the server after just one day or straight away so then there are only a small number of emails on there to fetch. It might stop the mail server or mail.app from getting confused and downloading duplicates. Those settings are in the Accounts preferences underneath Advanced.