In the list view or column, swipe across the podcast as if were drawing a line through it. A red delete button should appear. Tap the button to delete, or anywhere else to cancel.
This should work in the Music app for iOS 5, or in the Podcast app in iOS 6 (must be downloaded).