SCOTUS already pointed to the answer: early history of the US that helps us see the meaning of the Constitution as written.
The Federal Reserve has roots in the early history while contemporary agencies operate very differently from the Fed and have no real analog among the agencies created by those who set up the Constitution.
We could do more to make sure the operation of the Fed matches constitutional guardrails, but your question her has been addressed by the SCOTUS.