I think these "challenges" are often more imagined than real, at least in the case of reputable merchants. It's when there's *actual* fraud involved, such as with "shady" merchants, or with shady staff of otherwise "sunny" merchants, that problems seem to arise, and they get all the publicity to give the others a bad name.
I should think that's pretty obvious. If you withdraw your authority for them to help themselves to your money, then if they continue to do so, they are committing theft, or fraud, or something.