My wife hasn’t taken my last name and I couldn’t care less. I actually think it’s fairly patriarchal. I’m fairly confident that the failure of your relationship didn’t have anything to do with your decision not to change your name. I’d ignore his argument (as you rightly are) completely.