Stark from Earth @ LS @tonyfromls@qoto.org
Continuing the AI Engineering course for Day 3 at the moment.
Not Happy.
At 14 billion parameters for Qwen2.5-coder, its not only slow, it's like the hallucinating ChatGPT 4.x that I tried from last year. I still need to be clear on my prompts in detail.
Apart from that, I still need to install an AI Agent and configure things and yet it still far from what Cursor or IntelliJ with Codex offers but of course with a fee 👀
Too slow, although the AI agent can actually do tasks, even the lower 7 billion parameter is slow.
Not feasible.
It's really slow but still usable. It's like chatting with an AI model but in a slow motion speed 🤣
Wrestling with the configuration for the models in Continue extension in VS Code.
Weird that none of the configurations worked, except when I set autodetect for Ollama in the Continue extension and suddenly the AI model is introducing themselves to me after I asked them everytime I switched models.
14 billion parameter model is very slow to answer.
PS C:\opt\ai-projects\llm_engineering_in_python> ollama pull qwen2.5-coder:14b
pulling manifest
pulling ac9bc7a69dab: 100% ▕████████████████████████████████████████████▏ 9.0 GB
pulling 66b9ea09bd5b: 100% ▕████████████████████████████████████████████▏ 68 B
pulling 1e65450c3067: 100% ▕████████████████████████████████████████████▏ 1.6 KB
pulling 832dd9e00a68: 100% ▕████████████████████████████████████████████▏ 11 KB
pulling 0578f229f23a: 100% ▕████████████████████████████████████████████▏ 488 B
verifying sha256 digest
writing manifest
success
Time to setup our VS Code with these open source or open weight AI models and hopefully, the AI coding assistance capabilities should be just fine. Who knows I don't need to pay for Codex for the next month 😆
Anytime now!
PS C:\opt\ai-projects\llm_engineering_in_python> ollama pull qwen2.5-coder:14b
pulling manifest
pulling ac9bc7a69dab: 99% ▕███████████████████████████████████████████ ▏ 8.9 GB/9.0 GB 242 KB/s 8m55s
So we are almost there:
PS C:\opt\ai-projects\llm_engineering_in_python> ollama pull qwen2.5-coder:14b
pulling manifest
pulling ac9bc7a69dab: 95% ▕█████████████████████████████████████████ ▏ 8.6 GB/9.0 GB 294 KB/s 24m30s
😊
Alright did 2 python exercises under that AI Engineering course and already submitted the PR.
So let's check our Ollama pull for that 14 billion paramater model:
PS C:\opt\ai-projects\llm_engineering_in_python> ollama pull qwen2.5-coder:14b
pulling manifest
pulling ac9bc7a69dab: 51% ▕██████████████████████ ▏ 4.6 GB/9.0 GB 7.9 MB/s 9m11s
Going to get some rest and then we do a 10km run.
Also, I have notified my martial arts master that I won't be around again for the training because of the preparation of the technical interviews and technical tests.
First ever work on a forked project. PR Already submitted. Will be working more on the next coding exercises under AI Engineering.
We still at
PS C:\opt\ai-projects\llm_engineering_in_python> ollama pull qwen2.5-coder:14b
pulling manifest
pulling ac9bc7a69dab: 35% ▕███████████████ ▏ 3.1 GB/9.0 GB 9.1 MB/s 10m41s
PS C:\opt\ai-projects\llm_engineering_in_python> ollama pull qwen2.5-coder:14b
pulling manifest
pulling ac9bc7a69dab: 35% ▕██████ ▏ 3.1 GB/9.0 GB 573 KB/s 2h50m
Lets defer the long runs later as I feel the hunger at the moment. I need my lunch 🍗🍽🥤
And I noticed changing the VPN server cuts my ollama pull damn 😅
Time to prepare for a Sunday 10km run. I need a lot of 10km runs before I can get back to 21km runs.
Well, its the VPN server where I am connected to. Changing it made my internet connection faster 😅
Could be my internet connection. FB and Insta on my end is slow including YT.
Just woke up a few minutes already and my pull for that 14 billion parameters AI model was cut off. Connection reset by the server WTH 😅
Restarted that pull again:
PS C:\opt\ai-projects\llm_engineering_in_python> ollama pull qwen2.5-coder:14b
pulling manifest
pulling ac9bc7a69dab: 27% ▕████ ▏ 2.4 GB/9.0 GB 339 KB/s 5h23m
Sleepy now. I'll leave the Ollama pull for the 7 billion parameter AI model for now and do the pull for the 14 billion parameter tomorrow.
Just a bit worried on using 14b parameters in my local. Might be sluggish waiting for 5 to 15 seconds response from an AI coding assistance but let's give it a try soon.
Pulling more AI models using Ollama that I can use for a local AI Model on IDE AI Code assistance, in this way I dont need to pay anything 😅
Fork done and did a git clone but then some issues when I opened the project from Cursor as it is not able to find the appropriate kernel.
Turns out I need to do uv self update which took a new version and then do the update sync which creates the .venv and then from there I was able to select the appropriate Python kernel.
