I also had a lot of books, journals, papers, etc. to move, and not a whole lot else.
I did consider the USPS book rate/ 'M Bag' option, but in the end, I found it was easier-- and felt somewhat safer-- to ship them via a moving company, along with the rest of my things (a few boxes of photos, keepsakes, family china, etc.). I wanted to make sure I could trace the shipment.
I briefly considered UPS/FedEx/DHL, but they don't actually deliver to the islands. They just tack on massive surcharges, then hand the deliveries off to local couriers in Glasgow or Inverness. So, I can't speak to how the cost would compare for UK mainland delivery.
In the end, I went with a company that offered pallet space in shared shipping containers. (As opposed to many international moving companies, that seem to only deal in full or half-sized containers.) I had just one pallet, and by the time it was all said and done, the international shipping/customs charges/etc. came to, I think, around $1100. (That's not including the other £120 to have it brought in from Glasgow.) Be aware that all of these companies seem to quote a base rate, but mention that there 'may be' additional charges up to $xxx if your items have to go through customs inspection. It seems they always go through, so just figure on paying for the full, higher amount.
I wasn't too worried about things being broken (other than the china), but was quite worried about damp/water damage. I considered a lot of packing options, but ultimately, I bought several Rubbermaid-style, plastic storage totes. They're a bit pricey on their own, but there are often multi-buy deals, and if you start shopping for them just now, you'll find them quite cheaply as part of the Back-to-School sales. Check places like Staples & Home Depot, in addition to Target, etc.
I think I filled 8 or 9 totes, and then used heavy-duty strapping tape (the kind with strings running through it) to secure the lids. Lots, and lots of tape. In addition to being damp-proof to protect books and photos, and rigid enough to protect breakables, another advantage was that, unlike cardboard boxes, I wouldn't have to worry about the bottoms falling out under the weight of lots of books and papers. Plus, I've still got the totes. Which is good, because most of my stuff is still in storage, since there's no room for it. They stack up very neatly in a corner of our very damp bedroom.
So, that's what I'd say about how to move them. But I'll add one more, gentle suggestion: think hard about your library.
I know what it's like to be absolutely attached to your books. I'm sure I had at least 500 books before I moved, and I couldn't have imagined parting with any of them. But the cost of shipping, combined with the knowledge that I'd have virtually no storage space, made me take a hard look, and pare that number down to probably ~150. (My husband donated around 20 boxes of his old paperbacks to the local library, so I can't really complain.)
To be fair, about the only reason this was possible, was that between the two of us, we had a lot of duplicate titles. We also both invested in Kindles, and we were able to replace a lot of our paperback fiction with e-books, which take up far less space. I wasn't sure I'd like an e-reader at first-- actually, I was sure I'd hate it. But I've found I actually don't mind, and kind of enjoy the convenience of not lugging a backpack full of books around every time I travel, just so I don't run out of reading material.
In the end, what I kept were most of my academic/non-fiction/reference titles, the comics/graphic novels that my husband didn't already have, any paperbacks that weren't already in his collection, that were out-of-print or unavailable as eBooks, and anything that was signed or otherwise collectible. This meant that I got to give away many of my favourite books to friends and family, which was a bit sad, but also kind of fun in its own way, as I enjoy matching books with people.
There are a few things that I do miss, or wish I'd kept-- some old textbooks or critical editions from college, for example. But I've found used copies of those available on Amazon pretty cheaply, so if we ever get our own space, I can replace them easily.
So, anyway, it could be worth going through your collection to see if there's anything you can shed.