What do I do?

As Dr. Bo said , this is not a fallacy. It's more psychology related.
First of all, by saying "I was JUST trolling" what they do is MINIMIZING. They are trying to present it like you're overreacting, and basically they are blame shifting. All that time they saw you get annoyed and still continued saying nonsense until you lose your temper, while at the same time they remain completely calm. That may make them feel superior and powerful by seeing how much they can affect your emotional state.
And that's sadism.
Something I had found interesting, which expands to a much wider range of behaviors besides trolling. is the concept of "Baiting and bashing". What I had read is:
"Abusive people enjoy this framing game. They provoke their chosen target for a reaction, then claim it as evidence of mental instability, evil-mindedness, or something else that implies it is the victim who is at fault. Diverting all attention away from his own behavior, the bully seeks support from others, turning them against his target. It can be devastating for an individual who already is suffering from mistreatment to be blamed, slandered, rejected, and isolated, as well. 
The abuser enjoys the sense of power and control he derives from tormenting with impunity, and the positive attention he gets from playing the victim and fishing for sympathy. 
It is also an effective method of intimidating his target from attempting to speak up and expose the truth."
It doesn't apply entirely to trolling but it has some interesting points.

To answer your question, I don't think you can do much against a troll. As with all ill-willed behaviors, you need to develop a philosophy of evil so that you're no longer a victim of it. Once you feel that you're wasting your time you should disengage from the conversation. But to do that you have to remember that some conversations are futile, whether you're dealing with a troll or someone truly stupid.

I'd say a possible non sequitur since it doesn't necessarily follow that when one is trolled, they get annoyed and angry. Amusement at the troll is also an appropriate response, as is indifference, etc. The intent of trolling may be to annoy the trolled, but the intention resulting from being trolled is not necssarily to feel annoyed. So, what you do is you choose a different response. Boom, problem solved.

I'd also like to add that imprecise language betrays imprecise thinking: the differences between there/their/they're and your/you're should be mastered and applied by everyone attempting to communicate clearly.

What's so hard about that? Seriously.

There are no fallacies here.