Stark from Earth @ LS @tonyfromls@qoto.org

We are entering the era of AI and of course the basic utilities we are paying for like electricity, water, and more recently internet, now comes a new one. AI. All these while fossil fuel is declining and getting more expensive 🤣

So no wonder, a lot of these companies behind the frontier models are making big money, and so is NVIDIA and others 🤣

Here is the shocking thing related to context window and AI model billing.

LLMs don't have a memory of previous chats. So say you send a chat to ChatGPT with the message via an API call: "Hi ChatGPT, weather is good today in Brisbane. My name is Tony and how are you doing today?" ChatGPT will respond: "I am good Tony. I wish I am in brisbane right now!"

Then you send another chat message via an API call and in that message you are asking what is your name, testing if ChatGPT ever recalls your name base from your previous chat: "What is my name?" Then you get shocked as ChatGPT responds: "I am sorry, who are you again?" 🤣

The case is different if you are chatting with ChatGPT in via the ChatGPT product in the web because it has access to your whole chat so it can remember you, so it seems 🤣.

The real situation is the API call examples that I mentioned above and the product ChatGPT made a way via the codes so that from users perspective, it looks like ChatGPT remembers your name along the chat.

So in reality, you will have to append your previous chat to your new chat that you want to submit to ChatGPT via an API call. It is the same with other frontier or open source AI models out there.

The implication is that the token growth of input grows proportionally as the chat goes along because under the hood of this AI model chat products, every message you sent, behind the scene it includes the whole chat before your latest one + your latest message and sent to the AI model.

The input tokens grows linearly as the chat moves along, while the processing time grows exponentially.

That means the longer you are talking to an AI model, the input tokens grows linearly although you are unaware of it and it eats your AI bills faster.

We have not talked about the output tokens yet that you are also paying or at least consuming your free tier. The messages that is returned to you during the chat session has an equivalent number of tokens and of course the corresponding price but behind the scenes, the thought process of the AI model before it gives you its response to your previous chat also has tokens.

That means if asked ChatGPT on what is the integral of 1/x and it answers you ln |x| + C, the answer has an equivalent token + the thought process that comes with solving the integral of 1/x is also added to your total output token.

The reason why we see instagram posts, stories, or reels where CEO of some companies are crying now for their AI bill topping billions of dollars to date 🤣

Also the reason why I am looking for ways to find a good or efficient AI model that can run locally so that I don't need to pay, and also the reason why I am inventing a new one which is my baby hobby project 🤣

One thing I need to change is the way how I interact with AI Models or LLMs.

Turns out my way is expensive and after learning from the AI Engineering course that the context of a particular one time chat with a frontier model covers only what is passed during that context and becomes the memory of the AI Model and also relates to the tokens to it and the total billing, and in that way also a quantitative basis for consuming free tiers as well, I need to learn how efficient prompt engineering and later on efficient RAG.

In short, I must learn to convey a problem to an AI Model in the shortest possible equivalent tokens, and also I must learn to constraint the AI Model to only output what I need instead of giving back a ton of explanations that I don't need because the formula for billing an AI Model use is:

(input token * input token price) +(output token * output token price).

No wonder I always reach the max limit on my free tiers on different models of ChatGPT and I ended up subscribing with Codex because I also reach the max free tier limit for Codex 🤣

With Raptor.mini for C development, I should be much careful because it is free tier also and has limits.

Back to reviewing the base code of the previously released API services I made.

More updates to be done but I dont want system development this week as I mentioned, I want to rest but with some light activities like technical reviews and continuation of the AI engineering course.

So intially I just want some project for some fun and was thinking that I use my previously released project on weather reporting and overlay some data on a globe surface where these weather APIs are utilized.

But last night I thought why not use APIs for earthquake so people can use it to locate data on earthquakes on whatever location around the world.

Predicting the next earthquake is too ambitious and presents some legal liabilities so I guess the better way is to just present a statistical data with forecast for the next risk geographical locations for an earthquake. No saying the app predicts but just showing probability weights as to where is the next risk location for an earthquake.

I don't wanna be sued for just for an app 🤣

I'm thinking of a project for this weekend. Maybe a web application that I can run on some free domain so that people can use it for free and also I need a free API call to earthquake reports around the world.

I need to have the program interact with an AI so that the AI can predict with probability weights the next risk location for a potential earthquake.

I want it in 3D globe so that users can just spin the globe while looking for areas with earthquake statistics and others.

I hope there is an API that is free of charge for it. Maybe API from some government funded science organizations.

Went back to check my downloads for the NVIDIA smallest AI model, but unfortunately at 6% the download was cut. It thrown a timeout exception.

I say forget it, I already have my setup here and raptor.mini for C programming and Codex for Java programming, I also have ChatGPT and other frontier models in the cloud and also I have llama3.2 and Qwen2.5-Coder on my local.

I remember that incident back in 2024 where I confronted that illegal drug addict and illegal drug pusher after he physically attacked her live-in partner and became an immediate threat to the people around living inside the compound.

I became the bad guy in people's eyes because when I confronted that idiot, I saw his body language moved to prepare to physical escalate so I draw my bolo and my mom hurried to pacify the situation and the idiots girlfriend also used her body to shield the idiot from me.

My mistake there, I didn't report it to the police. I should have reported that incident to the police but since I didn't that idiot walked away while people here sympathized to him and then I'm the bad guy.

Again, people are stupid.

That is why from then on, I must report incidents on legal channels. Something to learn from.

I got the incident recorded in the barangay. No complaints yet but for the record, so if this happens again, then there is a strong evidence of the recurrence.

As what I mentioned last time, people here in the countryside, almost everyone here are relatives with one another and tree line for the blood relation is vast. When I came to the barangay to have the incident recorded everyone know the trespasser immediately, who his immediate relatives are, where he actually lives (turns out he has a previous residence area still existing apart from here in our neighborhood).

The initial vibe of the place is tense after people in the barangay identified the trespasser with the ID I have shown them. When I said that I am not complaining yet but just wanting the incident to be recorded first and if this happens again that is the time I will escalate. Then the tensed vibe immediately shifted to people smiling again.

I also included the incident where this neighbor was shouting around and eventually went to my sister's gate and cursed the dog all captured in CCTV which as past 3 weeks or a month already. In this way, the record will show previous incidents of unbecoming in the neighborhood. I thought my sister went to the barangay last time about this specific incident but she told she didn't so better that I also included this in the blotter this morning.

People don't know me here, even at the barangay, they know my mother and my sister but not me because my mother stayed here for 20 years already but I only came to settle here much recently.

Better to have this recorded in the right channels legally just in case because I realized, people most of the time portray me as the bad guy and stories gets flipped in their favor and against me.

Now that the record is in place, next time it happens, there is now a supporting evidence that a previous incidents did happen of unbecoming of the neighbor and the trespassing act in our territory.

It is not enough that you do right in incidents here, it is a must that you also show that you did some level of tolerance before you acted to escalate because people here can perceive how you acted and if they think you abused the situation even though you didn't, the whole family, the whole relatives, the whole clan, even spanning provinces around the north will be your enemy and people can be traced to political figures down their consanguinity tree, network, or graph.

Well, happy Tuesday people 🤣

I asked the neighbor I said, "What if I get into your property without your permission?" He didn't answer but he just grinned.

That is the problem with people who has double standards. They think they can do whatever they want but they themselves don't want things like that done to them.

I woke up an hour ago and check the news for a bit and then I heard movements. I thought it was livestock that lost it's way, turns out it's the fucking neighbor went inside the territory without asking for permission with his sickle in his hand.

I could have escalated by waiting for the right moment, got out with my bolo, use body language so that he moves to use his sickle to defend and then I counter. Easy to do that. Or I could bring this shit to barangay and possibly escalate into a case.

I got out instead and gave this man a warning first. First warning and took his ID's and got a picture. I am not a bad guy so I won't escalate immediately but a warning is in place. Told this guy he can't just get inside our territory without asking for permission.

Then he told me he knew my mother and my mom's de facto husband, still that does not give him excuse to get inside without permission. Next time it will end in a bolo fight.

So the guy gave his apologies and said he thought there was no person around and offered me services to clean the area. Not bad as I hate the frogs that kept vocalizing during rains that I had to put earplugs so I can sleep 🤣.

So an agreement is made, that he will clean my area tomorrow morning in exchange for a small fee.

People are stupid.

Tried downloading the AI Model from NVIDIA but it turns out it only runs on cloud for largest parameter.

Ended up talking to that AI Model then it tells me that there exist a 4bit quantized version that certainly can run on my local and gave me the command. I executed but file not found. Told the AI Model and instructed me to create a model file and gave me contents and should be run with Ollama create. It failed and it has something to do with the content of the file, significant turn around passes between me and NVIDIA's AI model. Then I asked OpenAI's AI model and it found the problem in the file immediately but cut me short from continuing as the AI Model that I been talking to from NVIDIA which is also the AI model that I want to run locally is too big for my machine and so OpenAI's model guided me to download the nano version instead.

I'm still sticking with Jarvis 🤣

Interesting, even NVIDIA is moving away from the transformer architecture similar to my baby hobby project.

NVIDIA is using Mamba architecture with MoE, very similar to the experimental AI model I been doing except that some parts of the architecture is absent from NVIDIA's but seemingly unique from the hobby project.

Because I got lots of sleep yesterday Monday, I'm still awake 😅

Anyways, let me try if I can run locally an AI model from NVIDIA.

Ironically though, when it was the time of Duterte presidency and I visited the university because I need some endorsement letter from my previous professors, there was not one signage or message thrown against the Duterte administration. I was expecting it because of the drug war but the university was very clean in that respect! Far from how it was during the Arroyo and Aquino times 🤣

I remember back at university during MS. I enrolled on the course Coding and Information Theory from the Electrical and Electronics Engineering Institute but I was not part of that institute because I was from the Computer Science department.

My MS adviser, I call him Einstein because of his hair. He just wore sandals, sometime long pants jeans but usually he just wore a cargo jean pants with an old T-shirt. He asked me why I would enroll on Coding and Information Theory. I said the field is part of the Theoretical Computer Science track but the department has never offered one so why not take it from the Engineering institute. He agreed but he never told me a cultural secret that there is a competition between the engineering institute and computer science department which I think is triggered by grants that is why some projects that are electrical and electronics engineering in nature ends up in the computer science department and some projects that are computer science in nature ends up in the electrical and electronics engineering institute.

The consequence is that because I was the only MS Computer Science student among the MSEE and MEng EE in the class, I was isolated. For instance, there was this problem on Galois Fields and we had a machine problem to solve. The groupings were 6 to 8 individuals but no group took me in. I was alone. I was isolated. Worst, they all know how to use Matlab and I don't so what I did is use Java to solve the machine problem. It took each group an average of 8 hours to solve the problem with 6 to 8 people per group, I managed to solve the problem in 5 days alone. They use Matlab where things such as polynomials are already models that can be used, while in Java I have to construct ideas from scratch even before I can manipulate the polynomials.

It was a very interesting course as I've learned a lot of things about Shannon's information Theory where information gets corrupted in a channel as information travels from point A to point B. Hence you can't send information from spacecraft that landed in Mars back to Earth without having some form of error correction to enable preservation or retrieval of the original information from the corrupted data.

We also discussed the use of entropy along the way in measuring how far each information say a bit or a character deviated from its original during receipt.

Another problem there was that at some point I went to enroll while in military uniform and because of the long queue, I had to rush from camp to the university without changing to civilian attire and due to that naïve action, I was identified and curious enough, the instructor on that course didn't gave me considerations specially during the Ondoy Flood disaster where I missed the final exams because I was deployed for relief operations.

The start of the relief operations fell on the same day as the final exam schedule and I was deployed with our unit to clear the roads of thick muds at San Mateo in Rizal. We started plowing thru with our shovel along the highway because no vehicle can pass thru, even the military vehicles carrying relief supplies. We been clearing the roads from afternoon up to 0400H next morning under heavy rains and we only stopped when a heavy machinery came thru and started clearing roads. It didn't end there, we were then deployed to Marikina to help clear the debris again so that relief vehicles can pass thru.

I reached out to my instructor, but even after explaining that I was in active service and was called for the Ondoy Disaster to help on the relief operations, that instructor didn't give me a chance for the final exam.

Turns out I still passed that course even without a final exam. The grade was initially incomplete, but later on 2.75 was given to me. After many years I took requested another TOR but the change was surprising, it went down to 3.0 🤣. I think the university lost the updates that was submitted in the grading system late which is supposed to be 2.75 on record but the system somehow after loosing track default to 3.0. I have the receipts from my old TOR to the recent copy both original from the university.

Imagine if I was given the chance to take the final exam, the grade would be much higher, maybe around 2.25 or even 2.0.

I am not really sure if the situation was a result of the culture differences and competition between the Electrical and Electronics Engineering institute and the Computer Science department or it is because I was being singled out after I was identified as a military personnel. Communist subversive materials have been found circulating the Electrical and Electronics Engineering institute although that is not the case in the Computer Science department.

So yes, not all you see in socmed is true. In instagram, a lot of the influencers there motivates people and blast those who are seen as lazy.

But for me I don't believe on that shit. When my body don't want to do workouts, say it feels lazy, I listen to it. Even if at times I don't workout for 3 months. The downgrade of the performance is massive actually, but I know that it's better than increase the probability of incurring a mutation and lead to Cancer by consistently doing regular workouts.

Still in the topic of cancer, I am naïve in its history and thought Cancer is a modern civilization phenomena but then someone in the metaverse corrected me and gave me references where Cancer was recorded in ancient times, specifically in ancient Egypt medical texts.

During those days that I was sick, I had a colleague who developed brain cancer. We were on a night shift engineering work and it's hard to sleep during daytime.

Cancer is not only brought about by inherited genes, but also things that we intake in our body and processes that disrupts our normal body processes.

A good example as what I mentioned is the consistent and long term lack of sleep can lead to brain cancer and this is supported in science.

Another thing is physical exercises. People think that exercise is healthy, indeed it is in aspects of say strengthening muscle groups including the heart, making strong the lungs and bone density, and others. But people forget or they are unaware that exercise also generates free radicals in the body and these free radicals can impart mutation leading to a cancer of some sort.

Even in the SOF community where operators are physically fit because of massive amounts of workouts, they do share a percentage of individuals who developed cancer.

Too much of everything kills 🤣