As people retire from the workforce, that leaves gaps lower down the workforce to replace those that retire. A lower birthrate means less people to take up jobs in a few years. We are already seeing this.
Granted if we didn't have such inequality then everyone would be able to find good employment as they would also have access to education to help build the skills to fill those jobs.