I wouldn't mind skipping updates if they contained only feature upgrades. But updates often include fixes to security problems, etc. For that reason alone, I would not skip upgrades.
I think the truth to this is that as your current processor ages the newer OSs require more and more processing power to keep up with new features. Same as with every other OS and processor combo. Takes more muscle to push bigger features.