All Notes

This video talks about Regularization techniques () that reduce overfitting which is a common issue with training neural nets. Do they look familiar? “Data Augmentation” looks a lot like Monty exploring the same object from various vantage points and in… Read more

This video does a great job of explaining how the current AI is “intelligent”. Intelligence has several aspects and the current AI does embody some of them to a certain extent. For example, it is able to extract features of… Read more

Back in the 50s they conceptualized the artificial neuron which led to neural networks which lead to deep neural nets and the latest architecture is the transformer (attention algo + deep neural nets) Mind you, this is all gobbledygook.. all… Read more

You’re absolutely right! 😄 Well actually no! The issue is, there are two kinds of memory in current AI architecture. There is the memory in the model and the there is the memory in the context window This is not… Read more

Maybe we can do it all with the transformer architecture and even reach AGI but it’ll be ridiculously inefficient, and will require a data center the size of Jupiter. And even if we try to make the architecture more efficient,… Read more

The main question I have in my mind is, can we build an LLM like solution using TBP‘s brain based architecture, hopefully more efficiently and with even better skills Ironically AI is helping me understand Monty and put together a… Read more

I’ve been spending time digging into Monty. There are quite a few comments and documentation which helps a lot and LLMs also proved useful in understanding how things work, why things are the way they are. TBP team has been… Read more

Tonight I’m poking around in frameworks/models/evidence_sdr_matching.py One of the challenges of building an AI system is tokenization and building embedding vectors. TBP uses SDRs (Sparse Distributed Representation) as embedding vectors. Unlike LLMs, this is a bit array (as opposed to… Read more

Dependency management is python’s Achilles’ heel. Monty uses conda, which is on the heavier side of things. I was curious if it would be possible to use something much simpler like pip+venv. ChatGPT says tbp.monty needs conda primarily due to… Read more