Fast neuronal oscillations (>30~Hz) are very often characterized by a dichotomy between macroscopic and microscopic dynamics. At the macroscopic level oscillations are highly periodic, while individual neurons display very irregular spike discharges at a rate that is low compared to the global oscillation frequency. Theoretical work revealed that this dynamical state robustly emerges in large networks of inhibitory neurons with strong feedback inhibition and significant levels of noise. This so-called `sparse synchronization' is considered to be at odds with the classical theory of collective synchronization of heterogeneous self-sustained oscillators, where synchronized neurons fire regularly. By means of an exact mean field theory for populations of heterogeneous, quadratic integrate-and-fire (QIF) neurons -- that here we extend to include Cauchy noise -- , we show that networks of stochastic QIF neurons showing sparse synchronization are governed by exactly the same mean field equations as deterministic networks displaying regular, collective synchronization. Our results reconcile two traditionally confronted views on neuronal synchronization, and upgrade the applicability of exact mean field theories to describe a broad range of biologically realistic neuronal states.