Ok, took me a while to figure out what you were talking about, but it does seem to be a bit of a pain. I did find a way to make it work, but it's less than obvious.
Once you get to the artist page (listing of tracks next to album covers), if the progress bar area is not blank tap the back arrow until it is. Then tap Play. If you had shuffle turned on, it will. If not it may not, so you can turn it on, pause, and tap the back arrow to make things blank again, then play to start over with shuffle.
The key is a blank progress bar at the top. If you select a track it will only play that album.
It seems quirky and a bit unpredictable, so YMMV
Edit: You can use the back button to get a blank progress bar if the replay icon is on. Turn it on after you get things started, if you had to use the back button. If not, worry not.
Second Edit: if shuffle is turned off it plays the albums in some order unrelated to the list, but it plays each song on the album in order. Weird.
