A bit more info on the charging alert sound. As Johanna says, it's one of the system sounds and can not be individually controlled. The Mute option will mute it. Mute is either the side switch, or an icon in the Control Panel (iOS) or Multitasking bar (iOS 6 and earlier).
The Mute option is only supposed to affect system sounds, alerts, and notification, but some apps will also mute, depending on how the developer set it up.
The system/alert sounds will not adjust with the volume controls unless you have Change with Buttons turned on in Settings > Sounds. With Change with Buttons turned off the system and alert sounds will stay at the volume level set in the slider. This is handy if you want your alerts to always be loud (or quite) no matter how you have your music or games set.
With the Change with Buttons turned on, zeroing the volume will stop all sounds from the iPad, including the charger alert.
I tested all of this on my iPad 3 running iOS 7. While I remember it doing the same on iOS 5 and 6, I can't test to make sure my memory (not always reliable) is correct.
This assumes we are all talking about the same thing, the beep that happens when you plug your iPad into power.