To be perfectly honest it's all about the context for me. If it's used in a derogatory manner towards homosexuals, then, yes, it bothers me. Otherwise it's too entrenched in common-lexicon to try and police. As a general term both have lost meaning, much like motherfucker usually doesn't refer to someone who fucks mothers, but a general disapproval with severity. Much of my offense to it just depends on my mood as well. If I'm in a good mood I'll let a lot of things slide, but other times it irks the ever living hell out of me.
XI Wiki


