There is legit scientific prototypes of solar panels made from non-mined biodegradable materials... if they will become viable and when is another question. But there is no fundemental limits in physics that would make it out of the realm of possiblity.
I think I even remember solar panels made with microorganisms that reproduce.
I haven't researched the specifics. Though I'd be interested to talk to a materials scientist regarding non-metal conductors.
Metal is biodegradable, and doesn't necessarily need to be mined (for example it can be extracted from ocean water, though its not efficient). But the conductors might be hard to do non-metal... only non metal conductors I know of are ceramics and they still need mining I think.
It's an interesting challenge. However, the point of sustainability is thinking & developing long-term solutions. Therefore, finite resources and pollution is "off the table".
Sooner or later a system that uses finite resources will either have to find alternatives or try to recycle what it can. In the context of fossil fuels, there isn't anything left that can be reused.
"but overall having a finite amount isnt the top concern"
Have you heard of peak oil?
Have you watched this video? https://youtu.be/-xr9rIQxwj4