Parent Custody Problems?

Here's the situation. I am 14, turning 15 in june. My mother is incarcerated and has been for about 3 years now, I live with my grandparents, not my father because he moved about 200 miles away. I am now having probelms with my dad, my mom has full custody of me and my father has visitation rights for 2 weekends of the month, and my grandmother has power of attorney (given to her by my mom which means she can act on my mom's behaf). My dad has agreed that I should go see him once a month instead, the problem is that I dont want to go see my dad, I have been blowing him off since January by making excuses, but now he is threatening to go to court if I don't go next month. I really dont want to go see him! My mom says that when they say he has visitation rights, that means that he has the right to see me and she (or anyone else) cant stop him from seeing me if I want to go, but if I don't want to go no one can make me. Is this true? and if he does go to court, what can he do about it (legally) if I straight out tell him that I dont want to see him again? Am I old enough (in the laws eyes) to make my own decision? Can a judge actually make me go somewhere I dont want to go?