Show newer

@kzimmermann

These are opinions that please the machine spirit.

@extrn

^The topology section of the thesis I linked to.

@extrn

It was an informal discussion in our research group.

But the need for topology came in because it was a debate on the flexibility of real number tensors versus what programs can do in practice. So I boiled both of them down to sets and am looking at their behavoir.

The modularity issue in deep learning is just the connected property. But it is pretty easy to introduce a noncontinuous function. In fact, this is common.

knowledge.uchicago.edu/record/

The issue is that connectedness is a valuable property to use, most of the time. So deeper networks are better. More data is better.

Program synthesis is also aided by the connected property, but relies on it less. So, not useful for modularity per se, but it can probably bridge larger data dead spots, or rough patches in data.

Tropical Algebra and Algebraic Topology of Deep Neural Networks

We present a theoretical and empirical study of feedforward neural networks using tropical algebra and topological data analysis. We show how examining neural networks through the lens of these two disciplines yields insights into their operation and efficacy. This work is divided into two parts: \emph{Topology of Deep Neural Networks} and \emph{Tropical Geometry of Deep Neural Networks}. Each part is respectively a self-contained analysis of deep neural networks from the perspectives of algebraic topology and of tropical algebra. There is a noteworthy connection between the two parts: One of our conclusions from the first part is that it is important to bound the topological complexity of decision boundaries; the work in the second part, among other things, provides such a bound in terms of the number of linear regions. The first part of this thesis is joint work with Liwen Zhang and Lek-Heng Lim and has appeared as a paper in ICLM conference. The second part is joint work with Andrey Zhitnikov and Lek-Heng Lim and has been submitted for publication.

knowledge.uchicago.edu

Seeing how a set theory can be used through topology to talk about geometric notions such as connectedness of points, is fascinating.

It also solves a current debate about artificial neuron-based versus sbolic logical reasoning. No idea why topology is optional at my school.

@lucifargundam

Sure. They are different. Human minds can understand the vitali set. Machine learning's best tools are only able to approximate borel measurable sets, despite taking the name of "universal approximators".

cognitivemedium.com/magic_pape

Humans are cool.

I should be reading about automated reasoning logics. But cognitive stuff is good too.

frontiersin.org/articles/10.33

Superior pattern processing is the essence of the evolved human brain

Humans have long pondered the nature of their mind/brain and, particularly why its capacities for reasoning, communication and abstract thought are far superior to other species, including closely related anthropoids. This article considers superior pattern processing (SPP) as the fundamental basis of most, if not all, unique features of the human brain including intelligence, language, imagination, invention, and the belief in imaginary entities such as ghosts and gods. SPP involves the electrochemical, neuronal network-based, encoding, integration, and transfer to other individuals of perceived or mentally-fabricated patterns. During human evolution, pattern processing capabilities became increasingly sophisticated as the result of expansion of the cerebral cortex, particularly the prefrontal cortex and regions involved in processing of images. Specific patterns, real or imagined, are reinforced by emotional experiences, indoctrination and even psychedelic drugs. Impaired or dysregulated SPP is fundamental to cognitive and psychiatric disorders. A broader understanding of SPP mechanisms, and their roles in normal and abnormal function of the human brain, may enable the development of interventions that reduce irrational decisions and destructive behaviors.

www.frontiersin.org

@oblate @cope @bun

Too bad they do not offer that course anymore, too abstract for a CS freshman these days.

@cope @bun

Sometimes computer scientists are scientists, or mathematicians. But they are always technologists.

@oblate @lupyuen

Nobody wants to pay for formal methods programmers. So they will be fixing bugs and having their stuff stolen for the rest of eternity.

My boss gave me 133 or so pages of math to read before Friday morning. I need to find something in this kind of logic to improve upon before then. Sleep should be happening now. :blobfoxcofe_w_:

What’s Decidable About Program Verification Modulo Axioms?
arxiv.org/pdf/1910.10889.pdf

Decidable Synthesis of Programs with Uninterpreted Functions
arxiv.org/pdf/1910.09744.pdf

Decidable Verification of Uninterpreted Programs
arxiv.org/pdf/1811.00192.pdf

@gentooman

> You will comply, even as the technology moves on from the point that such rituals make sense.

Classic Java culture of programmers.

@gentooman

Yeah. Rust is basically just Java, for people too young to remember how over-marketed Java was.

Thread: /g/88569667

>C++ is authoritarian
Not even remotely you can code the way you like and use any feature you like whether it's from C++98 or C++20

>Rust is pretty liberal.
Lol not even remotely the complier is forcing you to code in a specific style

@johnabs

It is more advanced. Yes. Very, more so than regular programs in a lot of ways.

It is a concept that comes out of the formal methods community. A more descriptive title is more like "automated program synthesis via mathematical logic and deep learning".

Show older
Qoto Mastodon

QOTO: Question Others to Teach Ourselves
An inclusive, Academic Freedom, instance
All cultures welcome.
Hate speech and harassment strictly forbidden.