If the back arrow is grayed out, that means that Safari does't think there is a previous page to go back to "in that tab". If a link you tapped opened a new tab, and you for some reason closed the old one, then you can't go back. Well, not by that method.
I don't know that this is what is happening, but without actually seeing what you did this is my best guess.
You can still get back to older content in two ways.
The simplest is to tap and hold on the + icon (top right of the tool bar). This will show you a list of recently opened tabs. It's only good for that session. As soon as Safari has been cleared from RAM, the recent tabs are reset.
Use the Bookmark icon, choose the Book tab, use the back arrow in the column until you see the History. Unless you've been using Private Browsing mode (dark gray toolbar), all your visited sites are here. It's not the easiest place to navigate, but if you visited the site recently it's not overly hard.
Of course, it could be a problem with Safari itself. In that case I'll just offer the usual fixes (below) and suggest you go to Settings > Safari and Clear History and Website Data. Sometimes old data stored on the iPad can cause problems with a site that makes heavy use of it.
If the problem persists please give us more details, like what iOS version you are using, the site that's causing the problem (if you don't consider it private), and anything else you can think off.
------------------------------
The Usual Fixes
------------------------------
When you have problems with an app it's a good idea to clear it from RAM.
Close/Clear the app:
iOS 6: With the problem app closed, double tap on the Home button to get the multitask bar. Tap and hold on any icon in the bar until they jiggle, then tap the minus sign on the problem app.
iOS 7 & 8: Double tap the Home button to get the multitasking display. Swipe up on the problem app's thumbnail image (not the icon).
If that does not work try a restart,
Restart: Hold down the power button until the power off slider appears. Slide it off. When the iPad finishes shutting down, hold down the power button again until the Apple logo appears. Apple recommends you restart the iPad at least once a month.
and then a reset if that does not work.
Reset: Hold down the power and home buttons until the Apple logo appears. The iPad will reboot. This usually takes a bit longer than a restart.
Reset All Settings: Another, somewhat more drastic thing you can do is Reset All Settings. You'll find this under Settings > General > Reset > Reset All Settings. It won't erase any of your content, apps, or accounts; but you will have to go through most of your other settings and re-tweak them back to your liking.
Do not confuse this with Erase All Content and Settings.
More detailed information at:
http://support.apple.com/kb/TS3281?viewlocale=en_US&locale=en_US
