You can only restore a previous version of the iOS if you have saved the SHSH blobs for that older[er] version. Although the odds are good that if you don't know what SHSH blobs are, you probably haven't been waving them, see here for details on SHSH blobs: http://www.ipadforums.net/showthread.php?t=59291
Oh, and if you really mean that you are on iOS 5.1 - try upgrading to iOS 5.1.1. A lot of people have stated that the incremental upgrade to 5.1.1 IS an improvement over 5.1.
Sorry, but, as far as Apple is concerned, once you upgrade, there you'll stay.