car loan

My ex and I broke up almosy 2 years ago and til this day he hasn't refinanced the car to get it off my name. The car loan is only under my name and I've been extremely patient and have given him enough time to refinance but he claims he can't get a loan bcz of bad credit. The problem is that the insurance is also under my name, he's gotten quite a few tickets, and the most recent thing is he hit a pedestrian. This situation has gotten way to stressful for me and I want to take the car but heard that I can't just take it even though I put $6000.00 down payment and the car and insurance is only under my name because he's been making payments since car was purchased and he can turn around and end up taking me to court. Please any information on what I can do to end this stressful situation will be greatly aprreciated..