It doesn't really download mail, it just syncs with your mail folders on the server. So, one way or another all that mail is on your account.
It's hard to guess what is really going on without knowing what kind of email account you have, GMail, MobileMe, Yahoo?, etc. But here are a couple of things you can do.
Under Settings >> Mail, Contacts, Calendars, go to the Mail section and change Show to a lower number of Recent Messages. Doing this will keep Mail from downloading more than that many messages in a particular mail folder, unless you go to that folder and tell it to download more.
Depending on your account type, this might help.
Go to Settings >> Mail, Contacts, Calendars, and choose your account under Accounts. There may be options to change how many days back to sync and or what folders will push mail to the iPad. The only account I have that does this is set up as an Exchange account, but there may be others types.
If this doesn't address your problem, please describe it in more detail, and let us know what type of mail account you have. Someone probably has the same account and will be able to give you more detailed help.