Even though magazines end up on your newsstand shelf, they are still apps. To delete specific issues you need to open the app and go to whatever view shows all of your downloaded magazines. From that point on methods vary a bit.
Try tapping and holding on a magazine you no longer want. If an x appears at the top left corner, tap it to delete the magazine. Usually all the magazines will get the x, and possible jiggle (like when deleting apps). If so there will probably be a Done button to exit this mode. Alternately there may be an Edit button that will let you select and delete magazines (and possible other actions.
Good luck.