Swipe up on them. They will only come back if you open an email in it's own view. As long as there are no extra open emails, the pop-ups will not appear when you open Mail. You can still force them open by tapping on the three dots at the center top of apps.
There is no way to disable this feature. It's part of the new multitasking environment meant to help you quickly switch between different open documents in the same app. There will always be at least two thumbnails. The current view, and a + one for adding another view/thumbnail.