LLMs

  • AI, LLM
  • 8
  • 8
  • finished

Yesterday “human” won Cambridge’s dictionary word of the year award, but if you read between the lines, it’s clear that the real word of the year is “AI”. It took me long enough to put on paper (virtual one) my thoughts of LLMs, but I feel that this is the right moment. Maybe it’s one of the last ones.

I’m fighting back the temptation to title this post “LLMs - the Good, the Bad and the Ugly”, but it’s cliché and I learned from other humans to avoid clichés. Yet this would describe my experience with LLMs. They’re multi-dimensional: sometimes great, other times they make my life miserable. I have problems with thinking of any other tool that have such high happiness factor distribution.

☺

I’m a programmer, one of the old-school ones, or at least that’s what I think about myself. I entered this field because I love creating things. Transferring the sheer, ephemeral thought to the computer screen and creating something out of nothing feels like magic. Black magic sometimes. This is why I still hand-code. I despise “software factories” where you put a feature request in and get something out. I despise creating something without understanding the process or the internals. I tried vibe-coding of some personal tools. I hate it, but I admit it: they were ready in no time. They lack the human spark though. Uncanny valley all the way.

To me, LLMs are tools, not a magic wand. I read the output of my tools and fix it if it’s incorrect. I treat it the same way as text completion in Vim. If I press C-n too many times and go past the desired completion, I admit my mistake, go back and fix it. The same is with LLMs: if they produce bullshit, I admit my mistake and fix it. Sometimes by better prompting, sometimes by rewriting the whole thing by hand.

Sometimes I’m stuck or I have to work in area I’m not familiar with. When I was implementing a filtering language for Autosub, I made the tokenization part, because I have some experience with string parsing (especially with inefficient one), but I didn’t know what to do with produced tokens. I asked LLM to implement this part and it produced a fairly simple recursive descent parser. I studied if for a day, I refactored it twice to make sure that it looks like example on Wikipedia, so if I need a quick theory recap, it’d be easier to grasp. Even if I still don’t know the theory behind this kind of parsers, I know exactly how my parser works and I didn’t feel uneasy releasing it to the public.

Don’t sleep on the “unstucking” part. I don’t hate debugging - it’s the most “puzzle-solving-like” activity that programmers do, but it’s not something that I am always keen to do. Time constraints, fatigue or pure laziness. Sometimes it’s great to have a tool which I can ask about this one misbehaving test that I’m staring at for at least 30 minutes, and be pointed to this long forgotten method with esoteric side effect that I’ve never thought of.

Small and cheap models work great with my attitude towards programming. I hope one day I’ll be able to self-host them, but these days I can only afford to throw only a few petrodollars each month in pay-as-you-go API usage bill. (sidenote: Maybe rather than not being able to afford a pricey subscription, I should say that I don’t want to pay yet another monthly fee.)

This was the good.

☹

The bad and the ugly are complicated.

Too often I feel like a meat proxy. LLMs slowly but steadily become the source of truth and work. The only source.

I asked LLM for security audit of sand, my bwrap wrapper and it produced a list of 10 security issues that could lead to sandbox escaping. I understood less than half of them and deeper investigation of some of them resulted in at least 4 more issues. I had to ask LLM to create reproducer tests, because the prose it produced was incomprehensible. It took some time, but I implemented the fixes. It was fatiguing, miserable process because it felt like I’m doing never-ending orders of the machine. The promise of social and civilization progress was that machines were to help humanity, not the other way around.

Sometimes I speak with other meat proxies. These are my colleagues with whom I could discuss architectural decisions just a few months ago. Now, the most frequent answers to my questions are “can’t you just ask AI?” or “what did AI tell you?”. Even though we’re sitting next to each other in the office, when it comes to technical discussions, I’m alone, stuck with AI. The experience is inhuman, alienating and deeply depressing.

I’m one of the most senior engineers in my department, so I spend a lot of time doing code reviews. I never felt that I’m particularly good at this activity, but my peers never complained and the bugs caused by my omissions were scarce, so maybe it’s just imposter syndrome.

With AI, code reviews are 3x longer. Often they feel like I’m burning tokens of my peers’ quotas: I ask them to change something, they prompt their AI with my comment and post back the results.

Fear of missing something out always forces me to ask AI to double check and it does find more issues, oh boy, it does. Before sending, I read and understand them all, because blind copy-pasting of AI output doesn’t click with me. I sign my name under the review, this is my output, not AI’s. But I feel stupid for not catching this undefined reference or that unchecked optional access. Why am I here if all I’m doing is verifying AI findings?

In return I get AI-generated responses. Other meat proxies don’t care that much. Not always though, this would be too egocentric to say. If you’re a sloperator who still redacts your own responses: thank you. I love you and I feel your pain.

Code changes are bigger than ever before. Sometimes they include hundreds of files and thousands of changed lines. I sit for 10 minutes and don’t even know where to start. Sometimes I give up and tell AI to review this mess, hoping for the best and that I’ll grasp the idea of what’s going on while digging through the endless issues it finds.

At the same time, people spend a lot of tokens to make new tools fo AI. Maybe this is how technological singularity looks like: AI used to produce more AI. Better tools for AI. Easier tools for AI. Faster tools for AI. Tools made of only Markdown files that only AI can use. Every week I attend one or two meetings where people advertise their new AI tool, which underneath is a set of 10+ SKILL.md files. Maybe with a Bash script that copies these files to appropriate directories.

We spend so much time talking AI that it feels there’s no other topic possible. This is one of the reasons why I hesitated writing this post. Endless stream of new model releases keeps the topic hot. And they’re all “grand”. Sonnet, Opus, Sol, Astra, Carbon, Neon. At least Chinese labs keep names of their models more-less sane.

This was the bad.

☹☹

The ugly isn’t technical.

The ugly is constant fear of being deprecated. In a year LLMs become better programmers than I am with my 23+ years of experience. (sidenote: Fuck, I’m old.) They wouldn’t if not millions of developers like me who put thousands of hours of their hard work, worth billions of dollars, on the internet under copyleft licenses, on which LLMs trained and which they legally don’t violate because they produce “derivative” work, not exact 1-to-1 copies (at least not anymore). We were fools to trust that GPL gives us security of keeping our code open. It gives none and now we’re being cut off, one after another. I wonder how many proprietary lib-totally-not-a-curl were created this year.

The ugly is constant fear about future of me and my family. Will my skills still be needed on Monday? Or maybe Anthropic releases a new model on Friday that finally translates 3-word prompts of boss of my boss to valid business and implementation plans? Maybe I should reevaluate my life choices and start a new career in waste management, like the wise Harry King? After all, 10 billion of people will always need to piss and shit somewhere.

The ugly are environmental and economic consequences of training LLMs. Years before the first GPT we were talking about CO₂ emissions, costs of electricity, investments in renewable energy. It all went straight to trash because large AI labs need power plants for their huge-ass data centers, and they need them here and now. Fuck environment. Fuck local communities. I used to spend whole winters playing in the snow when I was a kid. I wonder if my kids will see any snow this winter.

If my computer breaks today, I won’t be able to repair it because of surging memory prices. But maybe the golden era of computing for everyone is ending. Maybe renting of terminals constantly connected to big AI providers is the future? Maybe this is true that 640 KB is enough for anybody - what for do you need all this RAM if all you have to do is constant streaming of JSONL slop? Oh, right. Node.js.

Does it make any sense to become a programmer today? If you want a steady employment without too much stress: no. It makes sense to learn programming, because it’s fun and because learning new things is what makes human species awesome and extraordinary. But as a trade? I won’t advise it to my kids.

Does it make sense to become a knowledge worker? I don’t know, but I’m afraid that sooner or later AI will replace all knowledge workers, except maybe highly regulated fields (medicine, law, finance, security engineers). But regulations might change at the whim of legislators and these are (or will be) in pockets of AI labs.

Where this leaves us? Do we still need to learn new things or educate our kids if AI gives us all the answers and instructions how to apply them? Maybe it’s enough to teach children how to read and type, so they can communicate with AI? Not even that with LLM-based speech-to-text and text-to-speech.

No matter the hallucinations, we don’t talk about them anymore. Sources don’t matter too, because sources that yesterday were laughable, today are cited in white papers. When asked, LLMs will use hallucinations of previous generations of models as a proof for their own reasoning. Mistakes of LLMs are so disguised that we stopped seeing them and started trusting that they don’t make any mistakes at all. We stopped seeking knowledge from the books, encyclopedias, dictionaries or even specialistic, trustworthy web pages. Why, if LLMs know all the answers and will give you them either in the most shallow or the most incomprehensible way? 90% of the time.

AI output blends in so much that we stop seeing it and pointing it out. Posters are the worst. They all look the same: generic, cartoony, with a lot of happy, rounded curves. How much time until all “art” becomes basically the same? Until “claudisms” become parts of language used by humans? It’s already happening. I don’t recall myself saying that something’s “genuine” before 2023. I used this word twice last week and I wasn’t talking about “genuine leather”.