It's been brought up several times , especially on April 20th at Where 2.0 conference in Santa Clara.
While we ourselves may not be any threat to anyone i find it to be a violation of our privacy because personal or not some companies and people do get stalked, this is going on with out our permission and we have no option to turn it on or off while yet paid allot for our devices
However, the information does not actually leave your device according to Allan and Pete Warden in this link.
Why is Apple recording iPhone and iPad users’ locations? | VentureBeat
Sent from my iPad2 64GB wifi 3G using iPF