Stark from Earth @ LS @tonyfromls@qoto.org

I'm hungry but time to do the 10km run. This is the last week I'm doing 10km runs and on the coming weekend I'm back to 21km run.

Will eat snacks while on the run and after, will do calisthenics.

Funny to remember that back in MS, I signed up for AI but then I fell in love with the equations I saw from the theoretical computer science track and so I shifted track and laboratories.

Now I am back on AI but with a deep understanding even without me staying on the AI track nor studied AI officially.

That is the beauty of majoring in the theoretical foundations.

Long way to go. This is a baby hobby project on LLM or basically an experimental AI model without using the transformer architecture which has an aspect of quadtratic growth and what I'm building has linear growth provided memory is bounded.

If and only if this works for real with the training data and can perform at the level of the frontier models as we grow the parameters / weights, this might become one interesting LLM down the road 😂

A new tag v.0.0.2-pre released. Improvements and bug fixes.

Basically made the memory module more reliable.

Now that I am not only using AI-First Engineering approach into creating systems but also started working on a new AI Model and with my studies in the second undegraduate degree in BS Computer Engineering to complete my hardware engineering aspect, this alone actually already makes me an AI Engineer.

Not a senior one but jus starting so we can classify me these days as a Jr. AI Engineer 🤣

I was planning to do a 10km run but the area is submerged to a low level flood brought about by the heavy downpour last night.

I'm thinking if I should do a 10km later but with my mountain bike to get past the floods and start the run somewhere, or I just do my martial arts.

For quite sometime now I been seeing the word Tensor in Physics and now in LLMs and Machine Learning in general.

I just realized what it is today 🤣

Reviewing the prototype code and later will continue for the next iteration.

In a while I will be on weekend warrior mode.

Pre-release tag v0.0.1-pre released 😅

This initial release is a prototype, to prove that the architecture works. Very small C project for now.

Later iterations will be small evolutions from the base code. Being cautious because although the architecture has been theoretically proven, it remains to be seen in the real world with actual data. So small iterations from hundred thousand of parameter to 1 million for the next round.

Prototype LLM pushed to GitHub. Preparing for release tag v0.0.1

I never imagined me getting into the mess of AI model engineering 🤣

Deciding on the C toolchain considering future portability between OS and CPU architectures.

Alright, so now I am working on an alternative architecture for LLMs and is going to be written in C.

Not sure yet if this is going to be better in reality although in theory the theorem for the architecture is solid. It borrows and aggregates ideas from unrelated different projects.

If this goes well, we will be seeing new AI Models that are at the same level as the frontier models but has a small computing resource footprint hence can be run on consumer PCs.

Trying to investigate alternative implememtations of AI models apart from the transformer architecture or maybe a step away from the use of binary floating point representation of the embeddings vector for cheaper arithmetic operations in the computer.

That means I'd be looking at research papers soon on this area.

On the other hand, I am tempted to write an AI model using pure C language but it turns out that I need to train the model after and since there is literally zero training tools for models written in C but in Python, I'd be doing all these tools in C from scratch which is a no trivial matter

This also explains why Python is the language of choice for AI model development. It's not just because it abstracts a lot of complex coding stuffs if done in lower level programming language but because the development ecosystem of AI including the development of AI models are rich for Python and not for other languages and definitely not for C.

Lightning strikes from afar and before I could even hear the thunder my computer screen blips. A few more and then power outage.

Same old electric grid problem. I wonder why it's not being addressed when we got BSEE people working on the grid.

I maybe asking a trivial question at the moment as to why embeddings in LLMs has to be numbers in R and not in Z.

Looking at cases where adjustments maybe more appropriate and "smooth" for R but not for Z.

Best hardware upgrades will be 2x Kingston fury modules of 16GB each to push my RAM at 64GB and a separate NVIDIA GPU maybe NVIDIA Tesla P40 or NVIDIA RTX 3090.

Interesting, Qwen2.5-coder:14b failed on task nor in chat with a simple problem. It may have something to do with the extensions Continue and Cline but when I used Qwen2.5-coder:14b as stand alone inside Ollama and just chatted with it and gave the coding problems, it's able to solve it and far from the slow response I had from last time.

Then I opened VS Code for Insiders and turns out it comes with a GPT 5 mini in the form of raptor.mini and though is free and also capped, the credits are a fraction of from those larger frontier models.

Basically the AI model and agent that I needed integrated with VS Code, a problem I tried so solve for days now is already solved with raptor.mini 🤣. No need to integrate qwen2.5-coder:14b with VS Code as it never worked for me.

End the end, what I have now are the following:

1.) OpenAI's ChatGPT Chat in the web which is already powerful enough but no integration with IntelliJ and VS Code

2.) OpenAI's Codex which is integrated with my IntelliJ but is subscription based, that means I pay for a plan.

3.) If say my internet went down, I can resort to using Alibaba Cloud's Qwen2.5-Coder:14b on my local running on Ollama and with no integration with my IDEs, I can use it's chat features like ChatGPT Chat on the web.

4.) VS Code with integration of raptor.mini (basically a GPT 5 mini tweaked by Microsoft)

So, closing this problem now.

What comes next is running llamaC++ on my old laptop AMD A8 and on an old Oppo phone, also trying the same with AirLLM.

Then a project on scaling local AI models via clusters.

Alright so after some exploratory and survey activities I realized that my workstation's memory speed is capped. Had to unlock it and got some small but decent improvements just now 🤣

On the other hand, about scalability of local hosted AI models, an idea came to mind and some brainstorming with Jarvis, I realized this is the next big thing for my projects.

I'll start doing the scalability project soon and share it on github as an open-source project.

Engineering mode again. Brainstorming my idea with Jarvis regarding scaling AI models without expensive hardware.