All Notes

One thing I initially missed about TBT is, when I heard it’s a “sensorimotor” framework, I started thinking about robots, but it really is talking more generally about how “locomotion”/movement is crucial to learning and intelligence. Current AI agrees with… Read more

DNNs create features and combines them to recreate the input data during training. Both the features and the percentage of which feature adds up to which input is stored in the same parameters. Therefore every new training data has the… Read more

I’ve been conversing with AI to understand TBP and Monty. We’ve been having extensive discussions and I’ve been rewatching the youtube videos to make sure they start making sense and AI is not making stuff up. And yes there’s been… Read more

Yes i think they are, and not because they carry more information, but a 2D pixel representation of reality is higher fidelity and allows for better world modeling Textual tokens we use today are an idealized compression of text embedded… Read more

These all highlight the deficiencies of the current dominant architecture with respect to how it doesnt have the “predictive” state that connects “feature models” relevant to a particular input that needs to be fine tuned or that the input is… Read more

Current AI architecture does something akin to this too, but not deliberately, only as a side effect and very inefficiently. It has several layers of attention+deep neural nets (DNN) where DNN builds mini models of the features it finds in… Read more

This video talks about Regularization techniques () that reduce overfitting which is a common issue with training neural nets. Do they look familiar? “Data Augmentation” looks a lot like Monty exploring the same object from various vantage points and in… Read more

This video does a great job of explaining how the current AI is “intelligent”. Intelligence has several aspects and the current AI does embody some of them to a certain extent. For example, it is able to extract features of… Read more