I agree with everyone else, you're not being unreasonable. It's your wedding and people need to be more understanding of that and not take everything so personally. What is it about weddings that makes everyone think it should be done their way when it's not even their wedding?
I'm in a slightly similar situation... I have a huge extended family (thanks to my dad's italian side) but a lot of them I rare ever see so I hardly know them, they don't really need to be at my wedding. I'm not the type of person who likes to be the center of attention but at the same time, I like a big party. So we decided we'd have a small ceremony followed by a larger reception. I limited the ceremony to only "immediate" family - parents, siblings, grandparents, very close friends, 1st cousins and 1st aunts and uncles ONLY. No Great Aunts and Uncles, no 2nd cousins, no 1st cousins once removed, no casual friends, no long time friends we haven't seen in 15 years, etc. All those people would be invited to the reception, not the ceremony.
Well my Grandpop was not pleased about that - he seems to think that because HE'S very close with a lot of the people who got ruled out of the ceremony, that means they should be invited to the ceremony... never mind that I've met them all of about 10 times in my life and I don't really want near strangers watching a very personal moment. I even asked some of my 2nd cousins who I have been closer to over the years and they said they would not be offended if they were only invited to the reception (since that's the best part for most people anyways lol)
So anyways, I know how you feel, I've been getting a hard time for something similar but just like someone else said, you can't please everyone so you just need to do what you want to do.