Because for him walking away would mean giving up so much stuff beside the public spotlight.
He COULD give everything up, but he accepts the public nonsense as the price of pushing ahead with Tesla, SpaceX, and personal fiscal profit.
Just because he accepts that cost doesn't mean he likes it, and in appearance after appearance I see him resigning to it.