Can a co-signer return a car without the other persons permission?

My brother purchased a car in April and i co-signed for it. Since he's had the car he's been driving drunk...so yeah helping him get the car was a bad idea. I want to take it back to the dealer but don't know if I need his permission. Please HELP!