@freemo @arh @purple the depression is eternal but my current line of thinking is basically

- there is some kind of novelty prediction mechanic that is looking at the codebook of known states and it activates the dopamine on exposure to previously unrecognized states

? this implements the concept of boredom; an actor seeing the same shit repeatedly doesn't get rewarded but then seeing a new thing does which makes them want to see more of an area instead of sailing in circles (actual problem OpenAI had)
? probably also treadmill of hedonism; but haven't been thinking of that

- there is a set of hardcoded reward and punishment conditions; those were set by evolution
- lovheim cube of emotion is just a state switcher; i'm not fully sure what the point is, but it exists and glial cells like it and for whatever reason we're capable of becoming sad (even though sad/anger/etc seem to be suboptimal)

- the whole brain does not learn at once

? most of the brain is hard coded for specific tasks. prefrontal cortex seems to be there to veto bad action plans, otherwise action plans seem to just sort of erupt randomly as impulses
? no long term memory without working hippocamopus
? it's the top of the neocortex
? memories mutate over time
? neocortex is a particular circuit of parts repeated; kind of looks like these individual units are lifted up to the hippocampus to be reprogrammed. the whole system does not learn at once (a la google) but some limited subset that is capable of holding 7+/-2 engrams while it programs the circuits, non-heirarchially, but then shoves them back down in to the folds, where they are capable of referencing other nearby pieces.

i'm not sure what an agent actually DOES though. a self directed system can't have a score it optimizes for. i can only assume two things

- it's trying to maximize rewards subject to the emotional state (i guess this is why they are there; horny and combat modes alter reward and punishment response) which is in turn protected from abuse by the novelty recognizer (to prevent the agent from just masturbating 24/7 until it dies because that's an easy source of reward)
- the hippocampus otherwise has no goal; it's just building predictors.

now deepqn from google (the original deepmind paper) does something like this. they collect an experience bank and then "re-play" it to adjust the Q learning stuff, but they hide all the Q learning stuff inside the giant neural network black box. so it's somewhat possible to record snapshots of state for some number of captures and then we have a system that tries to optimize prediction from one state to the next states in sequence using the entire stack (of only ~6 layers) of circuits but it's only allowed to modify the 7+/-2 circuits that are currently in the buffer.

GMDH provides a suggestion how to do this; it basically drops out some of the data points for a training/validation set and then it does a process which runs training in some ways but then compares the trained blobs automatically to validation and it stops when the error rates are about similar.

GMDH kinda makes deep networks, in some instances, but its most basic form is just running "learning" systems and then stopping when training and validation approach an identical error rate (stop overfitting.)

so we have a reward/punishment system (reinforcement learning) but it needs a novelty detector to stop loops and then the memory system ... seems to just build pattern recognizers whos goals are to just predict whatever it's seen before.

so we have HMMs, GMMs and cortical models from Numenta which are pretty good at accomplishing basic pattern recognition. the big trouble is stacking them (not fully sure how HMMs do it; could be something to do with intermediary states but we can't directly define those.)

it seems like all the parts to pull it off