Libertarians joke: Most people are libertarian toward those they regard as their equals; they are paternalistic toward those they care about but view as lower status; and they are authoritarian toward those they despise. In contrast, libertarians are libertarian toward everyone.
Throughout history, almost everyone has wanted the state to enforce virtue among citizens. Almost everyone everywhere wanted the state to impose uniform culture and religion for the sake of stability and community. Most want the state to stop people from making imprudent choices.
Some people believe we should all live in godly communities that punish sin and reward faith. Others believe we should live in societies in which no one offends the weak and marginalized. Others believe we should only eat the healthiest foods and never watch stultifying television. Others think we must not be allowed to drink alcohol, use recreational drugs, buy sex, or gamble. Most people think some such goals like these are so noble that they are willing to use violence to achieve them.
Libertarianism advocates tolerance. It is thus a demanding doctrine. It asks people to rise above normal human nature. Most of us are moral busybodies who want to control how others live. The libertarian says, “Live and let live.”