Stark from Earth @ LS @tonyfromls@qoto.org

Now that I am not only using AI-First Engineering approach into creating systems but also started working on a new AI Model and with my studies in the second undegraduate degree in BS Computer Engineering to complete my hardware engineering aspect, this alone actually already makes me an AI Engineer.

Not a senior one but jus starting so we can classify me these days as a Jr. AI Engineer 🤣

I was planning to do a 10km run but the area is submerged to a low level flood brought about by the heavy downpour last night.

I'm thinking if I should do a 10km later but with my mountain bike to get past the floods and start the run somewhere, or I just do my martial arts.

For quite sometime now I been seeing the word Tensor in Physics and now in LLMs and Machine Learning in general.

I just realized what it is today 🤣

Reviewing the prototype code and later will continue for the next iteration.

In a while I will be on weekend warrior mode.

Pre-release tag v0.0.1-pre released 😅

This initial release is a prototype, to prove that the architecture works. Very small C project for now.

Later iterations will be small evolutions from the base code. Being cautious because although the architecture has been theoretically proven, it remains to be seen in the real world with actual data. So small iterations from hundred thousand of parameter to 1 million for the next round.

Prototype LLM pushed to GitHub. Preparing for release tag v0.0.1

I never imagined me getting into the mess of AI model engineering 🤣

Deciding on the C toolchain considering future portability between OS and CPU architectures.

Alright, so now I am working on an alternative architecture for LLMs and is going to be written in C.

Not sure yet if this is going to be better in reality although in theory the theorem for the architecture is solid. It borrows and aggregates ideas from unrelated different projects.

If this goes well, we will be seeing new AI Models that are at the same level as the frontier models but has a small computing resource footprint hence can be run on consumer PCs.

Trying to investigate alternative implememtations of AI models apart from the transformer architecture or maybe a step away from the use of binary floating point representation of the embeddings vector for cheaper arithmetic operations in the computer.

That means I'd be looking at research papers soon on this area.

On the other hand, I am tempted to write an AI model using pure C language but it turns out that I need to train the model after and since there is literally zero training tools for models written in C but in Python, I'd be doing all these tools in C from scratch which is a no trivial matter

This also explains why Python is the language of choice for AI model development. It's not just because it abstracts a lot of complex coding stuffs if done in lower level programming language but because the development ecosystem of AI including the development of AI models are rich for Python and not for other languages and definitely not for C.

Lightning strikes from afar and before I could even hear the thunder my computer screen blips. A few more and then power outage.

Same old electric grid problem. I wonder why it's not being addressed when we got BSEE people working on the grid.

I maybe asking a trivial question at the moment as to why embeddings in LLMs has to be numbers in R and not in Z.

Looking at cases where adjustments maybe more appropriate and "smooth" for R but not for Z.

Best hardware upgrades will be 2x Kingston fury modules of 16GB each to push my RAM at 64GB and a separate NVIDIA GPU maybe NVIDIA Tesla P40 or NVIDIA RTX 3090.

Interesting, Qwen2.5-coder:14b failed on task nor in chat with a simple problem. It may have something to do with the extensions Continue and Cline but when I used Qwen2.5-coder:14b as stand alone inside Ollama and just chatted with it and gave the coding problems, it's able to solve it and far from the slow response I had from last time.

Then I opened VS Code for Insiders and turns out it comes with a GPT 5 mini in the form of raptor.mini and though is free and also capped, the credits are a fraction of from those larger frontier models.

Basically the AI model and agent that I needed integrated with VS Code, a problem I tried so solve for days now is already solved with raptor.mini 🤣. No need to integrate qwen2.5-coder:14b with VS Code as it never worked for me.

End the end, what I have now are the following:

1.) OpenAI's ChatGPT Chat in the web which is already powerful enough but no integration with IntelliJ and VS Code

2.) OpenAI's Codex which is integrated with my IntelliJ but is subscription based, that means I pay for a plan.

3.) If say my internet went down, I can resort to using Alibaba Cloud's Qwen2.5-Coder:14b on my local running on Ollama and with no integration with my IDEs, I can use it's chat features like ChatGPT Chat on the web.

4.) VS Code with integration of raptor.mini (basically a GPT 5 mini tweaked by Microsoft)

So, closing this problem now.

What comes next is running llamaC++ on my old laptop AMD A8 and on an old Oppo phone, also trying the same with AirLLM.

Then a project on scaling local AI models via clusters.

Alright so after some exploratory and survey activities I realized that my workstation's memory speed is capped. Had to unlock it and got some small but decent improvements just now 🤣

On the other hand, about scalability of local hosted AI models, an idea came to mind and some brainstorming with Jarvis, I realized this is the next big thing for my projects.

I'll start doing the scalability project soon and share it on github as an open-source project.

Engineering mode again. Brainstorming my idea with Jarvis regarding scaling AI models without expensive hardware.

After AI-first development mode, my brain is now on survey / exploratory mode on some problems and AI.

Currently at the metaverse while just in a awaking state 😅😅😅

My original taste for music was alternatives, pop, rock, and OPM. After getting into a circle of friends and acquaintances from different parts of the world but mainly from US and Europe, I got immersed in old school hip-hop, trap, house and electro genre, including at times industrial goth, and metal music 🎸🎹🎼🎧

Also got immersed into Turkish music and middle eastern musical influences too.

BTW, I got VS Code for Insiders, the icon where the color is green as opposed to the regular VS Code where the icon is blue.

Heard interesting stuffs about the Insider hence I downloaded it but have not used it yet. The plan is to connect llama.cpp with it soon.

Alright, I should have invested on a dedicated NVIDIA GPU's back in 2022 with a lower end CPU instead of getting an AMD APU.

Although I am still happy with my AMD Ryzen 7 5700G, there is a heavy constraint on what AI model's I can load on my local machine.

Anyways, I think I could buy a brand new NVIDIA GPU soon if not the dedicated AI accelerator chips so I can run more powerful AI models locally.

On the other hand, I got two machines sitting by my side doing nothing. These are old machines. One old laptop with an AMD A8 CPU and an old Oppo smartphone with a bulging battery.

I have prepared a project pipeline for these 2. The first project is to load AirLLM for both and see how a local AI model will perform using AirLLM architecture. The next project is to use llama.ccp on both.

Whatever works, that will make both of these old machines a separate AI nodes in my home network.