You might want to replace it. Silicone cases come in varying qualities. Even pricey ones can suck. If you like silicone, try to stick with silicone TPU cases, which tend to be much less sticky, but still have some give. I like them, but YMMV.
If you buy through Amazon, the customer reviews tend to help, though there are so many choices that it might end up being time-consuming. (On iPad, Amazon's Windowshop app is handy, with searching and scrolling.)
For cleaning, you might also try running a lint roller over your case from time to time.