I grew up in the South. I know firsthand about the Klan and Nazis.When I was a kid, these haters were called by their names. There was the Klan and there were the Nazis. No skinheads, no neo-Nazis, this was before the advent of new white nationalism. Today we lump them together and they are called white supremacists.In my town, there was always the recognition that simmering under the false sense of calm was something that might, at any moment, erupt. The Jewish community was aware of that potential. The black community shared the same fear. It could happen at a high school football or basketball game. It could happen downtown on a Friday night or Saturday night. It could happen in a local mom-and-pop grocery or liquor story.Jews were established and well integrated in my hometown. In a nod to our Southern roots, we never called it a city, even though it was the state capital. We even had a Jewish mayor so active in our segment of the community that he would regularly attend Shabbat services.We were not what Northerners call part of the “deep south.” We were south of the Mason-Dixon line which divided the North from the South in the Civil War.Actually, to this day, the Civil War is called “the War Between the States” in the South.In our town, natives – even members of the Jewish community – could trace their lineage back five and even six generations. That was the argument we used to show that we belonged, that we had a right to be active members of the greater community. Three generations and you were still a newcomer. People on my block – and we were north of Washington, DC – spoke with a southern drawl and everyone used the expression “y’all.”I was first among my siblings to attend a desegregated school. I remember conversations around the Shabbat table about busing. To me busing just meant “how I was getting to school.” I knew that my siblings drove off to school in one direction and I was going in the opposite direction.Only in high school did I understand why the two junior highs were so closed to one another. Because one was on one side of the road in an exclusively black neighborhood, where, during the hot summer nights, people would hang out outside their tenements. I discovered that what was now the junior high was once the black high school, the white high school was on the other side of the street and the white junior high was down the street.Our rabbi had a collection of local antisemitic photos.Private clubs were bastions of white elitism and one of his photos was of a beach club – the beach club my family belonged to. The sign in the days before we were members read “No Blacks [now I’m being PC and substituting a word here], No Dogs, No Jews.” It perplexed me that dogs were listed before Jews. My understanding is that the order was more an issue of style than hierarchy.We knew who the people were who hated us because we were Jewish. We stayed away from them. I remember my older brother briefing me about this person and that person. We stayed away not because we were afraid (although sometimes we were) but because that was the way we got along. It was the status quo. It was a way of avoiding crises. The big kids would lift up my baseball cap and search for horns. Honest. I didn’t see it as a threat. They didn’t punch or curse at me, they looked for my horns.As a kid I reasoned that people from more rural areas of the county were more likely to affiliate with the Klan and the other groups. I thought the same of hunters.I was so very wrong. Hate groups were from in town and from the farms. Luckily for us, they were small in numbers and got smaller over the years. As I got older I deceived myself into thinking that in the United States, we lived in a post antisemitic society.I was wrong. The tragedy of Charlottesville was in the works for several years. Politics of race and fear are part and parcel of the political scene. Fear and hate are the subtext of political campaigns today. The hate we see today is not unique to this past election – it was the foundation of Obama’s first and second campaigns too.The lessons I learned growing up in a world where the Klan and Nazis existed are important moral lessons.I learned the importance of being unequivocal when it comes to hate groups. I learned about their right to speak. And I watched them go underground because the rest of society said that even though they did have the right to speak, what they said was abhorrent and socially unacceptable.Leaders must condemn these groups and so must we the people. Society requires us to protect the rights of people with whom we disagree. At this stage in our democracy these ideas are not a serious threat to democracy, but they are a threat to young impressionable minds.This is not an argument of nuance.The author is a political commentator. He hosts the TV show Thinking Out Loud on JBS TV. Follow him on Twitter @MicahHalpern.