I'm going to assume you're using Gmail with the stock Mail - mainly because that's the only app I know that automatically makes you start with Archive versus Delete. So, here's some tips (that I "stole" from Apple and Gmail):
Explanation and Removing Option:
When you archive a message, it moves to the All Mail folder. It does not delete it from the Gmail account. In addition, it replaces all delete options when using a Gmail account, including using the Trash icon while viewing a message and selecting Edit to manage multiple messages.
You can disable this feature in Settings > Mail, Contacts, Calendars > Gmail Account > Archive Messages
Where to Find Archived Mail:
Any message you've archived can be found by clicking the "All Mail" label on the left side of your Gmail page (on the Internet). You can also find a message you've archived by clicking on any other labels you've applied to it, or by searching for it.
I, personally, hate that feature. If I don't want the Mail, I want it deleted, not archived. Of course, YMMV. So, there you go. If you're not talking about Gmail and the Mail app, my bad and as Leelai asked, please let us know which app you're discussing.
Hope some of this helps.
Marilyn