I've been thinking about automation degrees again lately

Here's an abstract graph that illustrates three possible growth rates for the size of code needed to achieve various automation degrees [^1]. Size of code here (*ELOC*) can be any code size metric, but literal lines of code seems to be a good proxy. Note that there is probably some sort of natural exponential (\(O(\exp n)\)) or combinatorial (\(O(n!)\)) increase in lines of code just for handling the code's own complexity, but I'm ignoring that. So in a sense this is size of code without the natural complexity cost

There's almost certainly no way to know what the lines of code of any program is before actually writing it, let alone know what it is in general (across all possible writings of the same program), but this is just illustrative in the same way big-O notation is

The *A* case (green) has the lines of code growth rate decreasing as automation degree increases like \(O(n \ln n)\), \(\ln n\), \(n^\alpha\) where \(0 < \alpha < 1\), etc. This means its getting easier and easier to add higher order capabilities to the program. This seems to predict that it might be easier to go from automation 1 to automation 2 (eg: a nail-hammering machine, to a machine that can figure out how to make nail-hammering machines) than it is to go from automation 0 to automation 1 (eg: hammering a nail yourself, to a machine that can hammer the nail). That *seems* wrong because there are many automation 1 programs in existence but no automation 2 programs. But we (probably) don't know how to make an automation 2 program, so maybe its easier than expected once you know how

I imagine the *A* case would look like the programmer leveraging the program itself to produce its own higher order capabilities. For example, if you build some generic solver for the program, then you might be able to use that solver within a more general solver, and so on. Sort of like how we use programs to automate the design of new CPU architectures which can then run the same programs on themselves

The *C* case (blue) has lines of code growth rate increasing as automation degree increases like \(O(\exp n)\), \(O(n!)\), etc. When difficulty is a proxy of code size -- which is very frequently the case -- this means it becomes harder and harder to add higher order capabilities to the program as the size of the program's code increases. I would expect this to be the case when edge cases require extra code. Though, generalization is ultimately preferred, so edge cases might not make sense generally

The *B* case (red) is between the other cases and is linear or roughly linear (linear on average, its derivative converges to a non-zero constant, etc)

Note: at some point the program should pass a threshold where it can automate the original programmer entirely (probably at automation 2). After this point the program's lines of code / program size metric must be constant. It isn't necessarily the case that the ELOC function approaches this threshold smoothly, but if you assume it does then the ELOC function is probably some sigmoid or sum of sigmoids

Another note: transformer systems like GPT-4+ are relatively tiny programs, and I think would classify under scenario *A*. Just adding a slightly different architecture and amping up the size of the tensors you're using is enough to get human-nearby programming abilities. With slightly more code for planning, designing, modeling, etc it may be possible to make an automation 2 system using transformers that can program itself zero shot

A third note: the order of the intersection points \(S_{i\, j}\) in reality isn't necessarily in the order shown on the graph

[^1] see [my other post on mastodon](https://qoto.org/@jmacc/111482842687834034) for what are automation degrees