We describe the Ruijsenaars' action-angle duality in classical many-body integrable systems through the spectral duality transformation relating the classical spin chains and Gaudin models. For this purpose, the Lax matrices of many-body systems are represented in the multi-pole (Gaudin-like) form by introducing a fictitious spectral parameter. This form of Lax matrices is also interpreted as classical-classical version of quantum-classical duality.