In your wee town, it's you are a new face. Perhaps.
But it could also be that you appear to have an attractive face and head full of thick hair (going by your avatar) and frumpy or not doesn't stop the facial/head appeal.
As a man, I like to cast a glance a pretty face regardless of the body it is attached to. But I tend to look at men's faces as well. Faces are interesting. People are intersting. I do tend to make eye contact with men and woman. In a city, things are different.
But these are glances. Occasionally a person may require a few discrete glances.

A stare I would consider offensive and would not tend to do that as staring is something kids, weirdos and creepy men do.
Now if I were with a male friend, things are a little different as men in packs of 2 or more do tend to be more observant of the fairer species. But even then staring is still a no-no.
But on the other side of the coin, I do find people tend to look at me, maybe because I looked at them? It could be the head full of long grey hair, my infectious smile, my raccoon black bags, or my wonderfully good looks

Some people are lookers, others are not.