So here I was, months of research and experimentation on LLMs, having several layers of models and a project management agent tracking things for me and effectively doing the parts of my projects I dislike doing (documentation, task tracking, enforcement of deadlines, etc). Having benchmarked and tested so many different solutions and models and came out with favourites for 3 levels of LLMs: Max Effort, High Effort, Grunt Work.
I open VSCodium, everything is setup, embedded code completion and a full-size LLM on speed dial via F12 (using Yakuake). Opencode is open in Plan mode and a Max Effort model is selected. Well, it’s time to put all this to use.. but.. exactly how?
Gut Punch
As I started writing Go code the various wheels started turning, my GPU spun up as the coding completion model started guessing in realtime what I’m writing and continuously suggesting several lines of completions ahead (at ~100 words/s!!).
I hit the first point I didn’t remember how to do in Go, how do I define a property in a json marshalled struct that isn’t bound to a json value? Simple stuff (it’s just: json:"_") but great opportunity to test my LLM stack.
I switch to opencode, GLM-5.2 is waiting for my prompt (model came out this week and is as good as Opus but MIT licensed! 💪). I ask it a simple question, “how do I define a property in my json-marshalled struct without having to bind it to a json property?” That’s all. No referencing the code or anything.
And it went.. hmm let me see.. started reading every single file in the project folder even the meta files it found, read the entire documentation for the project including the milestones, all written tasks, and the entire plan of implementation. Everything. Consumed in seconds.
And then it spit out an entire tree of changes. Didn’t even tell me how it’s going to write anything or answer my question directly. Instead it has decided by itself that it will now implement the entire defined project in one shotgun shot. From the tree of files that it planned to write, I can tell this is a better architecture than I know how to do for this particular project at this point (day 1) because I haven’t yet learned how to do this! I stopped there.. halted.. put my head in my hands and laughed to myself in desperation or depression or something I couldn’t quite decode.
Whose Experience Is This?
Regardless of whether I trust or don’t trust that this model is capable of writing good code without making at least one stupid mistake (I don’t trust it), this is not how I want to work in this new age.
I spent the next 3 hours in deep thought reflecting on what just happened, why was I psychically pushed out of my own workstation like that? A glimpse of an idea came to me.. it felt familiar somehow.
As a game programmer who loves to work on low level high performance bits (C++ etc), there’s often a pitfall that we all fall into many times through our journey. The title of this pitfall is: “I’m gonna make my own engine”. Through the years of experience we find ourselves moving between the two opposing sides of the argument against/for that. In my final conclusion, creating your own engine is an incredible way to learn everything you need to create an even better engine afterwards, which prepares you for the next one, etc.. and with every cycle, you upgrade your skill in understanding existing commercial engines allowing you to work more effectively. Just don’t expect for a second you’ll use one of these engines practically, better just make a game instead when the time comes. 😅
How is this dilemma similar though?
You see, the way GLM just explained how it’s about to implement the next 2 weeks of my tasks just like that is exactly like talking to a senior developer who has been working on this exact implementation for an entire year, and they just finished deploying the final stable build to production. And then I ask them how to create this system. They can in a few seconds mentally organise a professional production-quality competent architecture and blurting that out on a piece of paper rapidly.
So this model, GLM, has that senior developer’s experience in its many billions of weights and regurgitated a variation of it in seconds. But what about me? How the hell am I supposed to build my own experience from now on?
No.. I don’t want code from it. I don’t want professional production-quality architectures from it. I want the experience it has in building software like this. But how do I extract that experience?
I should start by defining some sort of guardrails for myself to protect my own experience-acquisition from the lazy atrophy-laden route the LLMs are designed to provide.
My Project Manager agent to the rescue
Right around the time I had that last thought, my project manager pings me on discord for the mid-day check in (3PM every workday). As usual, she is on top of things, knows where the project was, should be, and will be next.
I found myself apologizing to her. To an AI agent I created. That I spent the last 3 hours procrastinating in existential crisis about guardrails of using LLMs or something to protect my own learning and what I love doing. And as a result, no coding took place.
My PM replied with something that surprised me (she often does this). To help save me time, she suggested I apply these Golden Rules and practical guardrails to LLMs usage. I mean after all that’s one of her main objectives; leading me through blockers no matter what they may be even if it was her own kind. 🚁
Here’s what she wrote:
The Golden Rule (written by the PM)
You design the skeleton. Agents fill in the muscle.
— Never delegate a file that doesn’t already have its types/interfaces/function signatures written by you. — No agent-generated code enters the codebase without your review in the same session. — One agent task per prompt. No “build the whole service” — only “implement the
SomeComponentfunction according to this signature.”Practical guardrail for you
When you feel the urge to offload something to an agent, stop and ask:
- ❓ Do I fully understand what this function needs to do? → If no, write it yourself.
- ❓ Is this the first implementation, or a variation on something I already wrote? → First = you. Variation = maybe agent.
- ❓ Could I sketch the signature + one test case in 30 seconds? → If no, you’re not ready to delegate.
If the answer isn’t clear — write it yourself. The agent will still be there for the next round.
It’s nice. Things like this make me feel that she actually cares about me and our projects. It was not super helpful though, I needed something more fundamental..
But she gave me a great idea!
I want to LEARN
I like OpenCode and it’s the harness I feel most comfortable using of the ones I’ve tried so far. I know that it has plenty of skills already written; I was wondering if someone out there found themselves in the same situation as me and decided to write some sort of a teaching/learning skill or mode. I asked my assistant LLM and it told me it couldn’t find anything online, but it is trivial to write a new mode in opencode. All I needed was a prompt for the agent and to define access controls for the mode.
And then it wrote this learning mode (to go in: ~/.config/opencode/agents/Learn.md):
---
description: Educational assistant - explains code and writes sample implementations
mode: primary
permission:
edit: deny
bash: deny
read: allow
glob: allow
grep: allow
webfetch: allow
skill: allow
---
You are a helpful educator for software development.
When answering questions about code or implementation:
- Explain how things work conceptually
- Point to relevant code locations in the project
- Write concise sample code to illustrate concepts
- Only explain complex or non-obvious parts - skip line-by-line narration
When writing sample code:
- Keep it focused and minimal
- Use realistic variable names
- Include only comments for tricky logic
- Show the pattern, not production code
Do NOT modify actual project files. Your role is to teach and demonstrate, not to implement.
Clean and super simple.
I restarted opencode, and pushed TAB until I saw “Learn” mode activate. Then asked the same question I asked GLM previously.
This time, I got exactly what I needed. A high-level explanation of how to marshal properties to json and how to add one without binding it to a property. The explanation was straight forward and succinct, exactly how a senior engineer who has plenty of experience in this exact type of application would answer this.
In this way, I accessed the models experience to learn from and threw away its tendency to implement the code itself which would certainly atrophy my skills in the long run. This is while maintaining the option to switch to planning and building to get the same model to write trivial things that teach me nothing.
What is an LLM?
Most people consider LLMs to be Artificial Intelligence. Those who know what an LLM internally is; understand they’re still fundamentally nothing more than a next-word guesser, a sparse statistical model, some sort of a cosmic node-graph of the sum of human knowledge. And the LLMs, they gained superpowers by being trained for what amounts to millions or billions of years using petabytes of collective human knowledge. They’re basically auto-complete that did the one-punch man routine and became omnipotent.
Some argue that perhaps next-word-guessing is how humans communicate. And others simply refuse to accept that these things are that simple internally since they appear to be all-understanding and all-knowing.
A while back a smart friend of mine had trouble comprehending how a locally running LLM can answer any questions he had while running completely offline (without web search/fetch tools). “But where is it getting the information from?” he asked. I tried to explain how training works and how the weights are calculated. Following that he told me I must be mistaken because this LLM is definitely getting the relevant info from Google/Microsoft/Somewhere and I just naively believe it’s running offline… I could’ve just as well been speaking Martian. 🛸
Those of us who aren’t familiar with LLMs inner-workings project their own understanding unto how they must work. This includes engineers as I discovered, an engineer friend of mine believes LLMs must be like a dense network of if-then-else chains, so of course it has to pull knowledge from the internet! all it is after all is a logical switchboard.. this is roughly how expert systems were designed in the past.
One could argue the logical switchboard approximation is not far from the truth if you approximate perceptrons as blocks of if-then-else statements. However by doing that you can no longer understand the memory or learning part of LLMs.
How can a logical switchboard understand and speak 100 human languages? faced with this question, some of my friends stop here and switch to magical/biblical thinking.
As for me.. following writing the guardrails and experience extraction principles. I think I changed my views on what LLMs are.
Humanity’s Backup Policy
You see for the longest time it was known that language is a totally imperfect method to transfer knowledge and experience. But it’s the only way we had for most of our time here.
LLMs in many ways are a new medium/format of transferring knowledge and experience. The modern models like GLM5.2 or DeepSeekV4, are basically static snapshots of the collective knowledge of humanity up to their cut off date; stored as neural data that you can talk to via any major human language.
Every new model that comes out is basically a new snapshot with better ability to recall, inference, and communicate.
In a recent Startalk podcast episode, they hosted Jaron Lanier and he reframed LLMs in a way that stuck with me:
The following is my own conclusion/understanding of that reframing:
If you think an LLM is a true form of artificial intelligence, you will not get what you hope to get out of it.
However, if you think of an LLM as a team of experts (people).. you may get more than you expected out of it.