I never had an issue with other people touching my cards until the day I saw a woman I was in an Adult Day Treatment program for mental health patients, and who I did frequent readings for come out of the stall in the ladies' room and leave without washing her hands. For a while, I wouldn't let anyone touch them unless they used hand sanitizer first.
Energetically, I don't have an issue with it. I want the querent's energy in the cards. I mean, how do you do a reading without you get their vibes? I generally don't have them shuffle unless I know they're a card player. People who don't know how to shuffle, I feel pain when they mangle my cards. Also, I don't use reversals, and I've found that human beings have problems following simple instructions like "Be careful not to reverse any of the cards", even when I've gone so far as to take a marker and mark one end of the deck and show them the mark so they can keep the cards all pointed the same way.
I usually just have them hold them a minute, do the shuffling myself (I've dealt blackjack in Vegas, so I'm damn good at it) then fan the cards out and have them draw however many I need for the spread I'm using and hand them to me.