If you mean that you've actually selected the "Open In..." option within the Mail app - then you have left the Mail app and a back button will not help you. There is no Back button to go back to an app.
If you have them activated, you can use multi-gestures to do a four-finger swipe (left or right on the screen) to go back to the Mail app. Or, you can do a four-finger swipe up on the screen (or double tap the Home button; same thing) to reveal the multi-task bar. The Mail icon should be There and you can press the icon to go back to Mail.
Hope this clarifies.
Marilyn