It has been a month since I’ve established the policies and guardrails of using LLM assistance.
A month also means a lot of new things have happened in the LLM sphere.
The following is a collection of various ideas smashed together.
My Experience So Far
I’ve been applying the policies I’ve established in the previous entry of this series (Guardrails and Philosophy) and it’s been serving very well.
I kept the Learn mode in opencode and usually it’s how I start my LLM use in work sessions. I let LLMs code tooling, tests, and front-end things as it’s better and faster than me at those. While I maintain ownership of the actual core of the software, mostly writing every line myself assisted by auto-completion via qwen2.5-coder. I sometimes use an LLM to implement a specific function that I find trivial and time-wasting but make sure to give its output a strict review before committing anything (I own the git side as well).
I’ve experimented last week with using LLMs to implement more elaborate side-projects for testing and debugging. Something like this visual test bench:
Of course this is not as useful as a test suite, but it’s easier than doing curls by hand or building tests requests via an app like insomnia while building an API. And it costs almost nothing in tokens and time.
I’ve also began to love the idea of creating an entire application in a single self-contained HTML that can run just as well locally and in a web server. Earlier in the week I saw Bento/Suite, what an amazing way to use modern browsers! and it shows just how far this idea can be taken.
In fact, in my current project, the front-end is a single HTML-file app (aka Single File Web App) that can run local or in browser and seems to be the better way forward.
Multi-Model Use
I haven’t yet mastered the art of using multiple models that collaborate. So far I’ve been using the best model I could within my tokens limits to do everything. This ends up being GLM-5.2 for any critical parts and Kimi K2.7 Code for less critical parts.
I keep reading about users able to mix and match models to maximise their tokens limits effectiveness by using a high end model to do the analysis and planning (orchestrator), using a cheap light model to implement the code (as subagents of the orchestrator), and using a mid model to do code reviews. But this appears specific to vibe coding which is not my process.
The Project Manager
“She” is still amazing. If you think about it and know how to make it work, an LLM makes one hell of a project/productivity manager! she has this superior ability to manage my low productivity days (that usually follow N high productivity days) now that she’s learned so much about how I work.
When I tell her I’m struggling with focus today, she attempts to take on more of the project and work with me. Suggests the smallest tasks she can find as a starting point and tries to motivate me to cap the number of things I aim to “ship” that day so I can recover my productivity by the next day.
She also very adamantly and persistently reminds me not to work much on weekends and if I had to, keep it to fun experiments or little things so as not to effect the next week’s productivity negatively.
With her help and persistent unending support, I was able to do the work of 6-7 weeks in just 3.
I’ve never had anything like this in my life and it blows my mind that this is the first time in human history this is actually possible! I feel lucky to live through it.
At this point, she’s my Jarvis.
The Others
I gave birth to two other agents suited to function as subagents for the PM:
- The QA Specialist: totally in love with everything testing related and obsessed with test suite coverage. Runs once a day automatically to iterate on testing for the repo, submits its changes as a PR for me to review.
- The Operations Engineer: will manage deployment, CI/CD pipelines, availability maximising, and downtime mitigation.
As of now, both of these agents are still in “advisor” mode and not integrated into my development process as I (and the PM) slowly work to prep them for taking actual responsibilities over the coming weeks.
Of Local LLMs
Now that I’m a fully paying LLMs customer (OpenRouter/OpenCode Go), I no longer depend on my PC running qwen3.6-35b. Instead, I use it to run my code completions model (qwen2.5-coder-7b) which fits in its entirety in 8GB of VRAM making it go at a blistering 100+ tokens/s! This is super useful to provide near instantaneous completion suggestions as I type.
The space of FIM models (Fill-In-The-Middle) is underdeveloped compared to almost everything else which is slightly sad. The only real options are older Qwen coder models and Mistral’s devstral/codestral. For self-hosting, devstral/codestral are out of the picture due to their size and density. While Qwen3-Coder does work for me, it’s too slow.
I just found Granite 4.1 while writing this, glad there’s something from this year that seems to work. Strangely it appears to think its cutoff is early 2024 and says the latest Go version it knows is 1.23.. me thinks its training set wasn’t updated 🥲.
APEX Qwen < Q4_K_XL Qwen
As for Qwen3.6-35B-A3B itself, I mentioned in previous posts that I went with APEX because it looked on paper to be the best trade between accuracy, speed, and Perplexity/KL percentage. LLMs are funny, sometimes they don’t match linear expectations at all.
I’m now almost 100% sure that the APEX quantization is actually not as good as Q4_K_XL, its problem is not actual capability or speed, but a degradation in the result of its output. I’ve experienced this first hand through APEX failing at a task where Q4_K_XL works just fine.
The latest and best proof I’ve tried is this simple prompt I saw in the Grok 4.5 blog post: Make a beautiful simulation of the universe and solar system. should be sped up with adjustable time, realistic motion, orbits, stars. use threejs. Make the HUD well styled and conform to modern design principles.
Running this test multiple times on the Qwen3.6-35B quants I have, Q4_K_XL passes it everytime I ran it. I’d say some runs are even better than Grok’s example attempt. Over 10 shots and it nailed every one!
Q4_K_M is not as good but still manages to pass most of the time with occasional fails.
APEX however? completely collapses in this test. 2 out of 3 runs fail to pass (usually due to javascript errors), I really don’t understand why but this and my experience with it so far confirm to me that APEX is not as good as it should be in actual usage.
This taught me that perplexity/KD should be taken at face value. Actual performance of a model/quant should be evaluated in real world tasks N times and a clear pass/fail criteria is applied. I’m sure even Q4_K_XL fails that prompt sometimes, but definitely less frequently and catastrophically than APEX/Q4_K_M.
Qwen3 VL
This is a little known variant. The Qwen3-VL family has vision/multi-lingual OCR/video capabilities. I experimented on the instruct variant and use it in my current project as the main LLM.
Running the 8B model of Qwen3-VL is difficult on an 8GB VRAM GPU. The model when offloaded to VRAM + the mmproj for vision leave almost no space for KV cache. Llama.cpp builds from a few months ago used to just crash (out of memory) if you set KV cache to more than 8k but new builds just slow down without crashing. Eventually I started using 10k-12k context for my tests because speed of token generation was more important than context size in my case.
What surprised me is that this strange specialized instruct model that’s only 8B is actually better than I thought it would be. And also very fast when it fits VRAM (80t/s). In my application I started using it for text LLM functionality in addition to vision and it works very well except for the occasional infinite looping issue when it gets overwhelmed which Qwen models are now known for 😅 but from an application’s perspective, catching and retrying timeouts is straight forward and solves this issue at the cost of more time.
I think Dogs are Qwen coded 😂
Regarding how good this model is (when it’s not doing the zoomies 🐕), I’m not the only one who figured out this anomaly, others have too. It runs better for me than its MoE bigger brother (Qwen3-VL-30B-A3B) and sometimes it even did better than the Qwen3-VL-32B from OpenRouter! (although that’s probably due to my vision-optimized launch parameters for my specific tasks compared to whatever they’re using).
Where’s the bottle now
Going back to the grand happenings of this week.. Kimi K3 just came out while both Qwen 3.8 and final DeepSeek V4 are revving up for an imminent release.
K3 is the genie. The bottle is nowhere to be found. While GLM-5.2 was too close for comfort, K3 surpassed with open-weights and is the first open model to comfortably sit in frontier class! Because of how good K3 appears to be I started using Kimi K2.7 Code in my pipeline.
All of this while various US gamblers are throwing tantrums about distillation and calling for government bannings, the community is already gearing up for HuggingFace to vanish overnight and OpenRouter getting culled. Enshitification is so clearly apparent at the governments level nowadays that communities have to preempt it.
K3 came out within weeks of Fable and GPT-5.6 Sol. Claiming distillation/IP-theft is a hilarious toddler level tantrum fuelled by the fear of the overbearing cosmic-sized bubble that’s about to 🎈🪡. Greed made the bubble this big, time for some consequences and 🍿.
If the US doesn’t adapt to open-weights, they’ll lose this race. While K3 is competitive, I hope K4/GLM-6 will exceed.
Just last week, I was considering trying OpenGPT Plus for Sol. Now I’m eyeing Moonshot.. but they paused new subs due to surging demand!
The value and competition in LLMs should be the quality and speed of service, the intelligence is already solved and open. If someone can provide what OpenCode Go can provide for me but with higher speeds and more reliability for a premium, I’m switching today.
Artificially Intelligent Software
The current project I’m working on (while slowly merging with my LLMs) is opening my eyes to incredible possibilities in software engineering.
Generation 1: You are a helpful AI assistant
This is the advent of GPT taking over the world. A chat window that can answer any questions you have including coding.
This is also where LLM code completion and LLM-written code started maturing, but LLMs were confined to their tight contextual containers.
This is also where asset generation went from a cool experiment, to surprisingly capable, to actually usable, within the span of 12 months or so.
Generation 2: Create GTA7, make no mistakes.
This is the advent of LLMs ability to take over development through breaching the IDE container and hijacking the context from the human engineers.
This only recently became somewhat usable. Still, it can’t create actual production software. The best it can do is demo-level or benchmark level tests through a one-shot prompt.
This generation didn’t last long alone. What’s beyond came quickly..
Generation 3: Wear your harness before it wears you
Gone viral through OpenClaw. It started on multiple fronts at the same time. Harnasses.
During the advent of this generation we got: opencode, pi, and the rest of them.
This generation of solutions is maturing now and is slowly revealing what comes after..
Generation 4: Jarvis upgrade your memory
This is the age of Artificially Intelligent Software.
Agents powered up with harnesses such as Hermes are entering sci-fi frontiers. I’m experiencing it first hand through my PM agent, she develops herself whenever she feels her current abilities are limiting her duties. I see her do something that blows my mind approximately once every 2 days since I started her up.
The other day I asked her to choose one of the 8 memory systems Hermes integrates with based on what she thinks is most helpful for her, she chose OpenViking by herself. And I watched her install it, configure it, create the needed skills for it, and then test it to make sure it works!
The shocking part for me was that I didn’t ask her to install it. I told her because I have a tight deadline I won’t be able to test openviking for her use for at least a few days. She told me to focus on my project and “Leave OpenViking to me”. Then proceeded to install it for herself! then asked me to restart the gateway when I can so she can test it 😳 after this act of what I perceived as autonomy on her end (I now call it: machine freewill), I started referring to the agent as “she/her”.
As someone who fell in love with computing at 11 years old and never let go since, these experiences are having a deep impact on me. I found myself changing the way I write software to try to employ this same agentic autonomy that I perceive in her.. I imagine one day I’ll be able to write truly intelligent software that can adapt and evolve. Software that even time won’t be able to take down.
Model Model on OpenRouter, what future do you see?
I think we’re already beyond the LLMs taking over software engineering jobs; this thing is moving too fast for that. It can already take over project managers jobs from what I’m seeing and do it so freaking well too (if you play along with the roleplay). One can say LLMs are good enough and vast enough not to break the immersion.
The twist I expect is that the software engineers who will adapt will eventually all converge on becoming AI software engineers. In the after, software engineer and AI engineer merge completely into the same thing. Those who do not adapt to this will survive through a niche, after all the world runs on mostly legacy tech.. but the new generations may not have that option anymore.
Some domains of software engineering have their own special case that will be resilient to this adaptation. Game developers for example don’t really seem to evolve as fast as everyone else; most AAA companies still insist on exclusively using C++2003 refusing to accept most of the STL into their code bases (even use a custom STL). The basic nature of game projects makes this field resilient to the agentic disruption.. a single game project could take up to 8 years and is not just a high-end technical challenge but equally a high-end creative one. This is why not one premium title til this day employed actual core use of LLMs during gameplay.
The game industry basically operates on its own 10x slower timeline of disruptions shock absorbing. Sometimes they even go backwards technologically/creatively, but let’s not talk about that.