New computational model of #HippocampalReplay:
A recurrent network model of planning explains hippocampal replay and human behavior

By Jensen et al.
It looks super interesting! But let me tell you: this is not what hippocampal replay does in rodents… replay is not for planning.🤷‍♂️

#Hippocampus #Replay

A recurrent network model of planning explains hippocampal replay and human behavior

When interacting with complex environments, humans can rapidly adapt their behavior to changes in task or context. To facilitate this adaptation, we often spend substantial periods of time contemplating possible futures before acting. For such planning to be rational, the benefits of planning to future behavior must at least compensate for the time spent thinking. Here we capture these features of human behavior by developing a neural network model where not only actions, but also planning, are controlled by prefrontal cortex. This model consists of a meta-reinforcement learning agent augmented with the ability to plan by sampling imagined action sequences drawn from its own policy, which we refer to as 'rollouts'. Our results demonstrate that this agent learns to plan when planning is beneficial, explaining the empirical variability in human thinking times. Additionally, the patterns of policy rollouts employed by the artificial agent closely resemble patterns of rodent hippocampal replays recently recorded in a spatial navigation task, in terms of both their spatial statistics and their relationship to subsequent behavior. Our work provides a new theory of how the brain could implement planning through prefrontal-hippocampal interactions, where hippocampal replays are triggered by - and in turn adaptively affect - prefrontal dynamics. ### Competing Interest Statement The authors have declared no competing interest.

www.biorxiv.org

@elduvelle Thanks for posting! I’m curious: Could you elaborate why replay in rodents is not for planning?

@lnnrtwttkhn yes! There is no replay at the time of planning in rodents. Well, in rats at least. All replay is linked to consuming reward - some form of consolidation, but very likenot planning. There is maybe 1 exception that I can think of which is the shock zone study and the data is not super convincing…

Also, I am currently analyzing data from an experiment where we specifically address this, and it confirms it… sad, but it is what it is :)

@elduvelle Thanks for your response! It seems to me that the definitions of „replay“ and „planning“ are crucial. How would you specify those terms? Also what’s your interpretation of results showing forward replay during awake SWRs at decision points or during shorts pauses (e.g., Ambrose et al., 2016; Davidson et al., 2009; Karlsson & Frank, 2009; Pfeiffer & Foster, 2013; Singer et al., 2013)? Which cognitive process do these neural events reflect, if not planning? Thanks!

@beneuroscience @lnnrtwttkhn Indeed - I was actually going to point out to the discussion of @ak_gillespie’s paper for a list of the arguments against a (direct) role of hippocampal replay in planning :) and also more generally for the definitions of terms.
All the studies you (@lnnrtwttkhn ) mention are in continuous tasks alternating between two or more reward locations. Replay happening at those could reflect consolidation of the path that led to the current reward, or even to a previous reward, instead of planning the future path. The Pfeiffer & Foster paper (and Xu et 2019) are the only ones that go a bit beyond that. but when you actually dissociate the planning location from the goal location, as in my current experiment*, you see practically no replay at the planning location.

(*hoping to be able to tell more about this in the coming months!)

@elduvelle @lnnrtwttkhn @ak_gillespie El do you really mean there is no replay at the planning location (but still SWR?) or that the replay seen there does not predict future choice?

@beneuroscience @lnnrtwttkhn @ak_gillespie

No or very few swr and no or very little replay, even without considering the content, and even when rate / time spent pausing is considered…

@elduvelle @lnnrtwttkhn @ak_gillespie Wow very interesting, look forward to the upcoming story! On a related point, do you believe theta sweeps during VTE reflect planning?

Sign in to participate in the conversation
Qoto Mastodon

QOTO: Question Others to Teach Ourselves
An inclusive, Academic Freedom, instance
All cultures welcome.
Hate speech and harassment strictly forbidden.