If you swipe up on the popups, all but the current screen and the New Window popup will go away. The next time you open the app they won't appear. They will remain hidden until you accidentally, or on purpose, add another active window to the app.
You can tap on the tree dot icon at the top center of the screen to bring up the pop-ups at any time. If you lose a document or window you were working with, try that to see if it became a second window by accident.
You can not turn off the feature. it is part of the new multitasking environment.