Should I fight this guy?

I'm a sopre and there's a freshman that got in my face a couple weeks ago in the hallway because I "disrespected one of his friends". Which I didn't at all, this guy just has a rep for over ananlyzing things. Anyway, he ended up shoving me, but I walked away cuz I didn't think I should fight him. Well now lately he's been provoking me whenever he's sees me by doing things like insulting me when I walk by but pretending he doesn't know I'm there and shouldering me in the hallway and I'm really getting pissed the **** off now. he's about 3 or 4 inches taller than me, but I'm stronger. So.. What should I do? People say that I would kick his ***, but I've never actually fought before. I'm pretty good at wrestling/grappling though, I just don't know cuz those extra inches of height intimidate me a little bit. But I'll fight him if I need to. What do I do??