Finally we have a sensor, a thalamus, a column and a voting module. It is all neuroscience legit although we did keep some things simple at the moment.
The sensor is retina-like, with several patches, partially overlapping. Each patch takes in raw input (character based text), looks for features and generates an SDR.
Thalamus is a relay at the moment, but has the hooks to apply attention when we get to that.
The column takes feature SDR, generates pose SDR internally based on “saccades”, updates its sequence memory, and predicts the next feature SDR(s). Also generates an object SDR as its best guess.
Voting mechanism takes in object SDRs from all columns, and forms a consensus with an agreement score, and feeds it back into the columns
It is a single layer, 2-column system at the moment. It needs more testing and tweaking before it can be scaled.
This is all great but chatgpt slapped me with the reality that this is far from a useful LLM-like chatbot! It is a memory system that needs:
1) A query to SDR(s) converter
2) And a module to answer the query in English
Noone is working on a brain-based solution that can do this at the moment, although it seems more like an engineering problem rather than “we have no idea how to build this”.
It suggested I add a transformer to generate sentences for the bits the system returns, which is obviously not acceptable in the long run, but is a good short term solution to solidify the system, build layers of columns and perfect other aspects of the learning mechanism.
It would be amazing if it could work with a relatively small transformer I can easily run on my laptop!