There is a very noticeable difference, and I have screenshots to prove it. Yes, you are right that the difference on a big TV is much more visible than on a smaller screen, but HD videos on the iPad are definitely worth it.
I have season 3 of The Big Bang Theory on DVD and Blu-ray, ripped both, and put the episodes on my iPad. The DVD was ripped in 480p and the Blu-ray was ripped in 720p. I took two screenshots of each file, one with the black bars and the other zoomed in.
EDIT - I didn't realize the pictures would be this small, but here they are, in this order: 720p zoomed, 480p zoomed, 720p wide, 480p wide.