It’s clear to me that Hopfield networks have had a big impact on how we think about memory and the hippocampus (memories as attractor states; pattern completion). Are there any good explicit discussions of this: the impact of Hopfield nets on the memory field?
@cogneurophys @charanranganath
@PessoaBrain
@tyrell_turing
@NicoleCRust @cogneurophys @PessoaBrain @tyrell_turing I know Ila Fiete talks about the capacity limitations of Hopfield networks. See this from Chaudhuri & Fiete https://www.nature.com/articles/nn.4237
@NicoleCRust @cogneurophys @PessoaBrain @tyrell_turing Also, Randy O'Reilly has argued that the capacity limits in Hopfield networks encapsulate why Hebbian learning is computationally suboptimal relative to error-driven learning. I can't find a good paper but this touches on some relevant ideas http://psych.colorado.edu/~oreilly/papers/OReilly98_6princip_tics.pdf
@charanranganath @NicoleCRust @cogneurophys @PessoaBrain
Indeed! It's also worth noting that more recent evidence suggests that the hippocampus does not work like a Hopfield network (e.g. behavioural timescale #plasticity in CA1 https://www.science.org/doi/full/10.1126/science.aan3846, and existing motifs in CA3 https://onlinelibrary.wiley.com/doi/abs/10.1002/hipo.23034).
Still lots to uncover, but the idea that the #hippocampus is just a #Hebbian #autoassociative #recurrent network is surely false.
@tyrell_turing @charanranganath @cogneurophys @PessoaBrain
Thanks Charan! Q for all:
I regard these as stepping stones in models of vision: Perceptron > Fukushima's Neocognitron > LeCun's ANNs > AlexNet > ....
Analogous statements to what Charan is saying about Hopfield nets for memory could be made for (eg) The Neocognitron as a model for vision - it didn't quite work right, but a lot was there.
Do you all think about Hopfield nets in an analogous way for memory? Or is that still TBD?
@tyrell_turing @charanranganath @cogneurophys @PessoaBrain It strikes me that what @marcwhoward and @tyrell_turing have offered up suggests that it's at least TBD and unlikely to be analogous. But I'm not sure here.
@NicoleCRust @tyrell_turing @charanranganath @cogneurophys @PessoaBrain @marcwhoward@mstdn.social There’s the famous saying that all models are wrong but some are useful. I have yet to see a model that captures all forms of memory and their properties (learning rate, extinction, second order associations, pattern completions, ...) In other words so far so good, but still lots of runway for discovery.
#neuroscience #memory
