Rusty @RustyStriker@qoto.org

Btw, I believe that #AI has hit a philosophical wall that technology alone cannot move.

When I started working on AI in the mid 2000s, it was still all about expert systems, decision trees that modeled first-order logic, onthologies, and graph exploration to come up with a best strategy to solve a problem.

That generation of AI could already solve impressive problems, but it was limited by the amount of knowledge that humans could put into it to describe all the possible combinations of a complex problem, or all the possible logically valid propositions, or all the grammar rules of a language.

It was a purely reasoning-based AI. Deterministic, reliable, but its utility was constrained by the amount of logical and algorithmic rules that humans could put into it.

Then computing became cheaper and more scalable, data for big corporations became cheaper and scalable as well, and neural networks, largely forgotten for nearly 25 years, got their moment. We suddenly got statistical systems that could figure out patterns and rules from labeled data, without a human explicitly encoding them into a graph. And we really thought that we had solved the problem of AI. But then you get systems that can recognize a human and a stop sign individually, but don't know what to do when they are together - because it was never trained to deal with such an unusual combination, or even told what the real meaning of a stop sign is.

Deep learning trashed away decades of reasoning-based expert systems to focus on empirical models trained through statistical pattern matching, but in doing so it created parrots that can talk about anything without even understanding what they're talking about.

It's again the long-lived clash between rationalism and empirism. In spite of the technology, these problems have been around at least since the times of Plato. Do we through deduction (we learn the basic building blocks of reality, and then we learn how to logically connect them together in increasingly complex structures), or do we learn through experience (by observing and replicating things again and again, measuring the feedback, and gradually converging towards a local optimum that statistically minimizes the odds of error)?

Well, it turns out that we may need both, but we can't make such a big theoretical leap in understanding how machines (and even humans) learn while the whole field is in the hands of a handful of companies mostly interested in doing small iterative improvements over their existing imperfect models, with little to no incentives to take big risks required to really push the industry forward.

Best terminal W3 browser? #AskFedi