@KristianHarstad Depends on what you wish science to obtain.
I have quite an utilitarian view of science, I believe that scientific knowledge should be ultimately used to do something useful.
As long as a theory can be used to do something useful, that theory is useful for its purpose; it does not matter if it has been proven wrong centuries ago or if it's the most modernly conceived view of the world.
Fitting experimental data is not a good parameter to stop using a theory, or you'll quickly find yourself using extremely complex theories to solve very easy problems.
It is normal to use simpler theories despite knowing these don't fit all experimental data or even to simplify complex theories by ignoring some small effects and particular cases which would complicate the treatment of the problem at hand.

Theoretical musings can be useful as well.
I'd rather evaluate a theory on a problem basis: the simplest and more general theory is the correct one to use in a field. You have some choice in how simple and how general you want it, but I would not advise using quantum mechanics to calculate the speed of the flow of a liquid through a pipe.