Who will be our whipping boy if you are not around?

Well I guess its official.
My brother the peacemaker has dubbed me,
"The King of the Bastards" -- if I have overthrown someone I'm sorry my brother took it upon himself to be the sole voting member of the naming committee.
I will be abdicating soon anyway so the title will be available. Applicants, The line forms to the left please.
I didn't bother to point out to him that by the meaning of the word that I wasn't one because our parents were married at the time of my birth -- I don't think he was in the mood for it. Being that bastard was the nicest thing he called me during or conversation.
Basically this verbal campaign against me is due to the fact that I won't apologize for something I didn't do, but am being blamed for by those who would have made me a bastard had they not wed.
After his ranting on about what a lousy human being I am in very colorful words and catch phrases he picked up from the movies he watches.
He attempts to patronize me by saying that I'm smarter than he is and I should be able to see that I'm wrong.
I actually have to give him credit there being how he was able to say I was smart and a dumb-phuck at the same time. Of course that may have been an accident.
His next shot was to use my words against me. This I expected because the "peacemaker" is, if not anything else, the supreme keeper of all that is said.
This actually made me mad. Not because he was making a point but because he was going back to when we were teenagers and pulling things from then for now.
When this failed to get me he pulled out the guilt card. Then the I dare you to do it you bastard was his last attempt.
Daring me to admit I'm wrong -- how strange is that.
Oh OK buddy -- I'm right but because you dared me to say I'm wrong I guess I have to do it now or I'll be ...
a coward (yikes)
I know I'm being very cynical and flippant about it -- I have to rant somewhere
He did succeed in getting me very angry, but I guess those who are closest to you have an ability to push the buttons we don't let other people near.
The odd thing is that when I could have made a verbal assault on him I didn't say anything I just let it go. Hence the cynicism now, but I am glad that even in my extreme anger I didn't say anything that I will need to take back later.
The thing is that they are all much happier when I'm not around.
I guess its because I chose not to deal with them anymore and not vice-versa -- like have you ever almost quit a job because you hate it so much but the boss talks you into staying so he or she can fire you in a week.
That's how this feels -- oh we need you to come back so we can stomp on you some more who will be our whipping boy if you aren't around.
That was my evening --how was yours?