Guys like that are usually pussies, who don't think the person they're tormenting would do that. So good job.
In my freshman year of high school, there were a few guys, older than me but much shorter than me, that I swear targeted me because of their own insecurities with their height (I'm 6'4" roughly same at the time of this) Wasn't anything serious, really, until this one day when I ran into them one day when I was running late to class- and the guy that always gave me the most shit hit me with his backpack, gave me a bloody nose and black eye. After that, didn't see much of him, but he would still make fun of me, or just insult me in general. Still, I went about my life and never thought about it, or did anything about it. Ignored him.
Fast forward around roughly 7 years
I was out with my girlfriend in my hometown, went to a Subway that was built recently, close to my high school. When I got up to the register to pay, he came around the corner and recognized me, I didnt- at first. As he rang me up he did the usual "heyyyy I know you- you went to my high school" and at first I did the whole polite "oh.. yea! ummm, yea I remember you" at which talked about our time in school and was like "yyeaaaa, remember that time I hit you in the face!? hahaha" and it was only then that I remembered him, and I said like "...oh, oh yea. Wow, that was 7 years ago, why do you remember that?" Then he kind of got quiet, and I paid, and left.
Didn't even have to ask him the whole "hows that working out for you?" kind of spiel, this guy hasn't gone anywhere. He's stuck in his hometown, working an unskilled retail job, and still
cares about hitting some freshman when he was in high school. Not to mention I forgot about the guy since then. He's a nobody, like the guy that you snapped at. Don't pay them any attention.
TL

R- I had a bully, he was a douchebag, still is and isn't going anywhere in life. Your bully will be the same