Only iBook Store books will sync.
The books themselves aren't actually synced/copied between devices. What happens is that each device looks in the iBook Store and sees what books you have purchased. Those books are available on all your iOS (and OS X) devices, as long as they are signed in with the same Apple ID.
If you have sync Collections turned on, then the collection as an organization is synced. So you get all the collections, and the collections know what books belong in them, but they don't automatically download those books. Even iBooks Store books don't automatically download; though if you have Online Content turned on you will see all of your books. Those not stored on the device will have a cloud/download symbol next to them.
If you have it enabled in the iTunes & App Store settings, books purchased on one device will automatically get downloaded to your other devices. Still, the download comes from the iBook Store, not from your other device. And it only happens at the time of purchase. Older books won't be downloaded when you turned the feature on.
Suggestions:
If you already use iTunes on the computer to sync, you can add all your non-iBook Store books to the computer first, in iTunes on Windows or the iBooks app in OS X. Those books will be synced when you sync with iTunes, as per your Book sync settings.
Use a third party app for your PDF books. Most of the good ones will sync through cloud services like DropBox or Box; or even iCloud. I like PDF Express for it's friendly markup tools, but there are lots of these apps.
If it's only a few books, you can email them to the other device. If a book can be emailed you will see a share/action icon (box with an up arrow) at the top left corner of the open book. It's next to the library button and contents icon. To the best of my knowledge only PDF files can be emailed. You won't see this option for ePubs, even if they aren't from the iBook Store.