Close your eyes and open your ears. There are no truly open source AI models. Let that sink in. Now open your eyes, release the weights, and float back to the surface, just like how my router floated in the bathtub when my wife "accidentally" dropped it after discovering why our internet had been running at dial-up speeds for three months.
There are models available for download, sure. You can download Facebook's Llama models right now, but the actual training data is not available. I've attempted to do the same by scraping the whole of the internet to see if I can make sense of it. My kids noticed every time I was home, Netflix was buffering. Soon enough they made the connection and told their mom. She had complained that TikTok was slow on her phone as well, to the point that she had learned to live without social media. I kept my mouth shut. You can imagine the disappointment in her eyes when she learned that her newfound mindfulness and improved mental health were just side effects of my AI ambitions.
Scaling Up in the Shadows
Anyway, until AWS notices, I've downloaded terabytes worth of content on their server. I also am downloading a large file from Facebook. I'm pretty sure it's a collection of ebooks. I think they have all the books in the world in this file. Just yesterday I saw Zuckerberg on CNBC talking about Meta's AI initiatives, and I'm quite certain my download is related to that. Obviously, training an LLM with this data would chew through my entire funds before I can even get past the preprocessing stage. So I made a temporary pivot, a sidestep so to say. I'm gonna use the model to build a state-of-the-art investment tool.
Financial Innovation
Of the data I downloaded, I ran a grep command on the data to extract all pages that include "To the Moon!" I figure this would allow me to train the AI on great investment advice. I'm calling this model TTM (To The Moon). The recent Bitcoin halvening and Ethereum's layer 2 solutions were hot topics in all the data I found, which must be signals of something important.
This proved better than I had expected. The model can ingest all news, charts for the day, and the current time, then spit out a 1 to 10 rank. Softmax for the win! It also spits out the best time of the day to invest. However, I've noticed that the scale 1 to 10 is sometimes to be interpreted upside down—1 being a resounding yes, and 10 a hell no. I'm still working out some kinks, but who has time for extensive testing when the market waits for no one?
The Open Weights Breakthrough
My Pokémon funds are not exhausted yet, but I can't build an LLM on this budget. The previous one that spills words but no sense won't cut it. So this is where Open Weights come to play.
Using the Llama model, I've deleted the last 3 layers and plugged in my TTM. Lo and behold, I created a state-of-the-art model on my very first try. It told me to invest in $NVDA. On my second try, it just said "department of government efficiency." I don't know what that is, but I invested in DOGE. Long story short, I have funds now. Sam Altman would be proud, or possibly sending me a cease and desist letter—either way, I'd consider it validation.
The Path Forward
I'm getting consumed by this project. Now that I'm actually making money, I need to draw a clear plan so I don't deviate from my initial goal. All these rumors about OpenAI's Q* and Anthropic's Claude 3 have me motivated to push harder. While they're securing billions in funding, I'm securing... well, significantly less, but with greater efficiency per dollar!
The cat is out of the bag, but turns out you need millions of dollars to tame that cat. I'm gonna make sure that one day you can run a state-of-the-art model on your calculator. While Jensen Huang keeps announcing new GPUs that cost more than my car, I'll be optimizing for the hardware we already have.
Coming soon: "How I Convinced the SEC That My AI Trading Bot Isn't Actually Me in Disguise" and "Training Language Models on My Kids' Homework: Ethical Considerations and Performance Metrics"
Editor's note: Proxy AI neither endorses nor recommends any investment strategies mentioned in this blog. Ibrahim's financial gains are most likely statistical anomalies or hallucinations. Please consult with a qualified financial advisor before investing in anything, especially cryptocurrency named after memes.
