Abstractions are everywhere

We wake up every morning and brush our teeth without even once thinking about what is inside the toothpaste. Then we make breakfast on a gas stove or an induction cooktop without understanding the science behind either of them. We just learnt how to use them. Then we might take a cab to work without thinking about how the car works, or how the app that booked it works. Again, we just learnt how to use it, and we got so used to all of it that we hardly use our conscious brain.

You are reading this on a screen, and most of us don’t know the science behind that either. Do you know how hard it was to make a blue LED? Red LEDs have existed since 1962. A bright blue one took about thirty more years. Isamu Akasaki, Hiroshi Amano and Shuji Nakamura got it working in the early 1990s with gallium nitride, a material most researchers had given up on, and they got the 2014 Nobel Prize in Physics for it [1]. Without blue there is no white LED, and most laptop and TV screens are lit by white LEDs, which are really blue LEDs with a yellow coating on top. How Nakamura did it is my favourite part, and I’ll come back to it at the end. None of this matters while you read. We just know how to use the screen.

Every day we use many tools like these, and I call them abstractions. For us they are more or less black boxes. We learn their inputs and outputs, i.e. how to use them, and we never need to open the box.

In software

When I was in my first year of engineering, I was doing competitive programming all the time. One day I read about the segment tree. I did not understand a single thing about how it worked inside. I only understood what it can do: give it an array, and it answers questions like “what is the sum of elements 3 to 7?”, and handles changes to single elements, each in O(log n) time. I learnt how to use it. A week later, in a contest, I used a segment tree to solve a problem. That was my eureka moment. I understood the power of knowing tools, even without knowing what is inside them. Of course, I did not think much about abstractions back then.

query(l, r)   // sum of a[l..r], O(log n)
update(i, x)  // set a[i] = x,   O(log n)

I understood the segment tree much later. Meanwhile, I was just submitting it without understanding it much. With time I learnt how to tweak it without breaking it, like changing the aggregating function from sum to min or max, and much more, and that eventually made the segment tree feel very natural to me. Till I joined my job, I had no idea that a modern CPU executes instructions out of order. When I was in college, the compiler and the hardware jointly took care of most of this for me. That is abstraction built up over decades, so that developers don’t have to worry about any of it.

for (int x : a) s += x;

.L3: add  edx, DWORD PTR [rax]   ; s += *p
     add  rax, 4                 ; move p to the next int
     cmp  rax, rcx               ; reached the end?
     jne  .L3                    ; if not, go again

What an abstraction is

Abstractions can be defined in many ways. For me, they are generalisations of something that gets used many times, so that not everyone has to think about the underlying details. To use one, you learn three things about it: its inputs, its outputs and its assumptions.

The third one is the one we forget. A segment tree assumes the operation can be grouped any way you like, i.e. that it is associative. Sum, min, max and gcd all work. The median doesn’t, because the medians of two halves don’t give you the median of the whole. Back then I didn’t know that. Once I tried implementing an aggregation that was not associative, and it kept failing. Very early on I also didn’t understand the time complexity of each merge operation, and that made some of my ideas fail with time limit exceeded, because I thought “yeah, this is magic, I can do whatever I want”. C++ has assumptions too. It lets you think of memory as one long row of numbers, where reading any of them costs the same. That isn’t true either. Most of the time it doesn’t matter. Later in this post I’ll show a case where it makes the same code twelve times slower.

LOL

My favourite abstraction isn’t technical at all. It’s “lol”.

It started as “laughing out loud” on online message boards in the 1980s. Today almost nobody who types it is laughing. With one of my close friends, it has grown a whole grammar of its own. A plain “lol” just means okay, or it is something to say when there is nothing to say, usually with a WhatsApp sticker. “Lollll” means something is actually funny. “Damn lollll” means surprised, even shocked, but in a fun way. And “lol tf” is shocking, still funny, but with something sad or dark underneath, like when I bring up how bad the living conditions were in my college hostel.

It fits the definition exactly. The input is three letters, plus how many l’s and what comes next to them. The output is a tone. The assumption is that the other person reads it the same way, and that only holds because we share so much context. Send “lol tf” to a colleague and it may land very differently. The linguist John McWhorter calls words like this pragmatic particles [2]: their job is to manage the conversation, not to carry content, like “you know” in speech. I even type it to AI models, which are definitely not laughing. “That worked lol” means “I can’t believe that was the fix”.

It also shows what happens to a good abstraction over time. The more people used it, the less anyone thought about what was inside, until the inside stopped mattering. Most people who type “lol” never think the words “laughing out loud”. By 2011 it was in the Oxford English Dictionary. And it leaks like any other abstraction when its assumption breaks. There is an old joke about a parent who thought LOL meant “lots of love”, and texted “Grandpa passed away, LOL”. Same input, a different assumption, and a very different output.

If three letters can carry that much, and an LLM reads them correctly, it is not that surprising that plain language now works as a programming language.

A compiler over natural language

Abstractions got much more useful with LLMs. In very loose terms, an LLM is a compiler over natural language. If it were perfect at execution, and it keeps getting closer, we wouldn’t need to think about implementation details at all. We would only need to think about what exactly we want.

Well, it sounds simple, but it isn’t. Even when we wrote all the code by hand, more often than not we found “new” details while writing it, and those details changed our requirements. You start writing a function, realise you never decided what should happen when the input is empty, decide, and carry on. The code was where a lot of our thinking happened. Now the LLM writes the code, so that thinking needs a new place. We still have to go through those cycles of refining the requirements, without being inside the code. And reading every line it writes isn’t practical; it’s too much code.

Probing the black box

The method that works for me is black-box testing. After the LLM implements something, I ask it a list of questions: “Was this case tested?”, “How was this done?”, “What happens if this input is missing?” This way I steer it towards the specific cases that matter, without reading the code myself.

The best way I can describe an LLM is a very fast, very eager intern who badly wants to show you a clean result. Ask them to tidy the house before guests arrive, and they’ll be done in ten minutes, because everything went into one cupboard. The house looks perfect. You only find out when you open the cupboard.

Here is a real cupboard I opened. I asked an AI to search through some big log files, and a few of them, around 900 MB each, kept timing out. A while later it came back with a neat result, and nothing looked wrong. So I asked my usual question.

The fix was just a longer timeout. There was nothing wrong in the code. The code did exactly what the AI had decided, the problem was that it never told me it had decided anything. And this was not the only time. Once, one item in a result had no data at all, and the AI just re-ran that one item and moved on. I asked how many other items had the same problem, and it turned out a lot of them did, because an earlier run had stopped halfway and left them looking finished. So these days, most of my time goes into opening cupboards.

The LLM is a black box for us, and the code it wrote is also a black box. We use the LLM to “probe” the black box it made, to see if it behaves the way we want. So these days we mostly need to think about two things: what the correct behaviour is, and a system that checks that behaviour is met. And you know what, we don’t even have to write that system ourselves. It becomes another black box. So it is just more and more abstractions since LLMs got popular.

Of course we can’t guarantee 100% correct behaviour, because LLMs themselves are not correct all the time. But we can control the probability of failure. Building something now takes very little time, so we can afford several independent paths that check the same behaviour, and each one lowers the chance that a bug gets through.

The important word here is independent. If the same model checks its own work with the same context, it will most likely make the same mistake again. On one bigger project, I had one AI agent doing the work and a separate one reviewing it, and I asked the reviewer to re-run the checks itself instead of reading the worker’s report. Reading the report is like looking at the tidy house; re-running the checks is opening the cupboard. That caught a step the worker had quietly skipped, and a bug in how it was adding up results, and neither of them was in its report. Also, whenever I change code that shouldn’t change any behaviour, I don’t trust anyone’s word on it, not even mine. I replay a recorded day of data through both the old and the new version and check that the outputs match byte by byte. More checks only help when they don’t share the same blind spots. If each check misses a bug with probability p, and some share c of bugs is invisible to all of them, the chance that a bug survives k checks is

You can play with the two numbers below.

Know your car

With AI, you need to get comfortable working with abstractions, and that includes knowing where the errors happen. You don’t need to understand the engineering and maths behind LLMs, but knowing “enough” about them helps. It’s like being an F1 driver. A driver doesn’t study the aerodynamics or the physics of the car in detail, but they know exactly where it is weak. If the downforce is low, they take care in the corners. If the car is good on the brakes, they brake late to overtake.

For LLMs, my list looks like this.

When abstractions leak

So far I have talked about how useful abstractions are, but there is a downside. They are generalisations, and for very niche work they fail. Joel Spolsky wrote about this in 2002 and called it the Law of Leaky Abstractions [3]:

All non-trivial abstractions, to some degree, are leaky.

Remember the assumption that reading any part of memory costs the same? Here is a case where it breaks. The two loops below add up every number in a square grid. The only difference is their order: one goes row by row, the other column by column. Same answer, same amount of work.

// row by row
for (int i = 0; i < n; ++i)
    for (int j = 0; j < n; ++j) s += g[i * n + j];

// column by column
for (int j = 0; j < n; ++j)
    for (int i = 0; i < n; ++i) s += g[i * n + j];

Both do the same number of operations, but after about 10 MB the column-by-column code got a lot slower, up to 12 times. C++ did exactly what you asked for. What happened was that the developer was not aware of memory caching. The CPU fetches memory 64 bytes at a time, which is 16 of these numbers. Row by row uses all 16. Column by column uses one and jumps to the next row, and once the grid gets big, the CPU can’t keep enough of it close by, so it spends most of its time waiting for memory.

That one I could measure. The leaks that hurt more don’t look like performance at all. My favourite is a program that refused to start because of one character. Its config file had a dash in it, but not the plain “-” on your keyboard. It was the long “—” that word processors and chat apps like to put in for you. The program printed its config at startup with a logger that only understood plain ASCII text, so it crashed before doing anything else. “Text is text” is an assumption so deep that nobody thinks of it as one. And guess who loves writing that long dash? LLMs.

Sometimes working around a leak isn’t enough, and you break out of the abstraction completely. This is why you sometimes see people write assembly inside C++: it gets performance the compiler won’t give you. HFT firms bypass the operating system’s network stack for the same reason.

The most famous example is probably Python and NumPy [4]. Python is very easy to write; you never think about how a number is stored or how a loop runs. But under the hood every number is a separate object somewhere in memory, and a loop goes through the interpreter one step at a time. For heavy number crunching this was too slow, so people broke out of Python and wrote the heavy loops in C and Fortran. NumPy packaged that. A NumPy array keeps the numbers next to each other in memory, just how the CPU likes it (remember the grid), and the loop runs in C. On my machine, adding up 10 million numbers took 112 ms with a Python loop and 3 ms with arr.sum(). That is 38 times faster.

And then the break-out itself became the new abstraction. Most people using arr.sum() today have never seen the C code underneath it, and pandas, scikit-learn and PyTorch are all built on the same idea. With AI, writing this kind of code has become very easy. If your Python is slow, you can just ask an LLM to vectorise it, and it mostly gets it right, so now it’s enough to know that you should vectorise, and why. Slowly, this has also become another abstraction. What is still hard is everything around it: knowing what to measure, finding where the time is actually going, and proving that the new version is really faster on your real data, not just on some benchmark.

The cycle

This will keep happening. An abstraction hits its limit, someone breaks out of it, the trick spreads, and someone packages it into a new abstraction. Then it starts again, one level up.

A fair question is: if nobody understands the layer below anymore, who fixes it when it breaks? My answer is that someone always does. Not everyone needs to know what is inside the box, but someone has to.

Remember the blue LED from the start? Shuji Nakamura didn’t get there by using his tools as black boxes. To grow gallium nitride crystals you need a machine called an MOCVD reactor, which flows gases over a very hot surface so that they settle into a crystal, layer by layer. The standard machine didn’t work for gallium nitride. The surface had to be so hot that the gases rose away from it before they could settle. So Nakamura spent about a year modifying the machine himself, often changing it in the morning and growing crystals in the afternoon. He added a second gas flow from above that pushed the first one down onto the surface [7, 8]. He broke out of the abstraction everyone else was using, and that is what made the bright blue LED possible.

Thirty years later, that LED is behind the white light of most laptop and TV screens, and almost none of us know it is there. His break-out became our abstraction. Most of us will only ever use the screen, and that’s fine. That’s what an abstraction is for. But someone will always open the box, and the people who open it are the ones who build the next one.

References

1. The Nobel Prize in Physics 2014, press release. nobelprize.org
2. John McWhorter, “Txtng is killing language. JK!!!”, TED2013, February 2013. ted.com
3. Joel Spolsky, “The Law of Leaky Abstractions”, Joel on Software, 11 November 2002. joelonsoftware.com
4. Charles R. Harris et al., “Array programming with NumPy”, Nature 585, 357–362 (2020). nature.com
5. AtCoder Library documentation, “Lazy Segtree”. atcoder.github.io
6. Jeffrey Dean and Sanjay Ghemawat, “MapReduce: Simplified Data Processing on Large Clusters”, OSDI 2004. research.google
7. Shuji Nakamura, “Background Story of the Invention of Efficient Blue InGaN Light Emitting Diodes”, Nobel Lecture, 8 December 2014. nobelprize.org
8. Shuji Nakamura, Yasuhiro Harada and Masayuki Seno, “Novel metalorganic chemical vapor deposition system for GaN growth”, Applied Physics Letters 58, 2021 (1991).