If you predict a flat 50% on every prediction, your brier score won't be 0 for that bin unless your questions themselves are split evenly to have positive and negative outcomes. The most obvious trick to achieving this is to randomize each question's positivity. Flip a coin, if it's heads then you ask a positive question (will X happen?) otherwise use the negative version (will X not happen?)

I'm wondering if there are other techniques like this