Alright so after some exploratory and survey activities I realized that my workstation's memory speed is capped. Had to unlock it and got some small but decent improvements just now 🤣
On the other hand, about scalability of local hosted AI models, an idea came to mind and some brainstorming with Jarvis, I realized this is the next big thing for my projects.
I'll start doing the scalability project soon and share it on github as an open-source project.