Most people talking about "punching Nazis" are advocating punching people who are certainly far right but who have not themselves committed or threatened any acts of violence.
The Nazis of the early 20th century were committing genocide and conquest, and the only way to stop them was through violence. But if you're talking about "punching Nazis" in the context of the UK and Frogage, their odious policies are a long way from justifying threats of violence which are rightly not legal.
In the US where Trump is attempting a fascist takeover, time will tell whether it will take violence to dislodge. But from our position it's easy to see why a lot of these people get banned.