Neuromorphic AI Intelligent Text Recognition

DARPA has shown interest in creating artificially intelligent text reading systems recently. The air force research laboratory has been developing neuromorphic programs that are inspired by how the brain functions. They are creating software that can fill in missing portions of sentences (PDF) based on the context of the previous words.

Modern pattern recognition technology can perform accurately at its job when images are complete and easily observable. However when there is only a partial text image, a computer's accuracy pales in comparison to the human brain. The human brain is able to fill in details based on contextual relationships of surrounding words. Researchers have now been able to get a computer to make up missing details in sentences. They have mimicked aspects of the mind that are involved with visual processing in order to carry out this task. With a sufficiently advanced system this might allow a computer to understand text or speech that is garbled or partially missing.

There are a variety of different "levels" of the mind that people are trying to emulate. I've mention about the Blue Brain project that is attempting to replicate neurons and synaptic connections in software. That project may become even more detailed than that as time goes forward. There are also somewhat less detailed simulations that may not go into that much depth and use more simplified neuron configurations. At another end, there are people who are attempting behavioral modeling. Instead of emulating brain cells/synapses, they are basically looking at the overall modularity of the brain and how certain areas function together in order to enable specific states of cognition to exist.

The air force has been focusing on simulations that are in between the more detailed Blue Brain and the less complicated behavioral models. This level of detail is at the cortical column level. They mention there are approximately 10^8 minicolumns in the neocortex. There are about 100 brain cells in each minicolumn and simulating connectivity between them is easier than trying to account for individual neuronal connections. Each of the 32 axons from a single minicolumn extend to 32 other minicolumns. So in the neocortex, they estimated that there are 10^11 connections at this level compared to 10^14 connections at the neuron scale. They are currently modeling brain areas like the lateral geniculate nucleus (LGN) and the primary visual cortex (V1). As time goes forward they may be able to make replicas of the functioning of other brain regions that are important for processing information.

They have investigated several different models. One is a bayesian model of invariant pattern recognition (see left picture). A representation of the visual cortex has already been developed using this bayesian framework. That brain region is important for processing what a person sees. This design can allow for a neuromorphic AI to be able to identify items.


The bayesian model has previously been tested using the images on the left. The model had an approximately 50% recognition rate of these 32 by 32 pixel arrays. The company Numenta Inc. has been developing this software independently of the military. Numenta has been continuously refining the technology for more sophisticated visual identification under more varied conditions.

The scientists also researched a network of attractors. They discuss the "Ersatz Brain Project". This project is an effort to replicate aspects of how the mind functions by using nested networks of fixed point attractors. This is more of an algorithmic method of mimicking parts of the brain. They specifically picked the brain state in a box (BSB) algorithm. They wanted to figure out how these models could scale to the full neocortical scale and copy the functioning of minicolumns.

They also investigated a spiking neuron columnar model. They performed a literature review on a variety of neuronal software imitations including Hodgkin/Huxley (HH), Morris-Lecar, and Izhikevich. They built software to emulate aggregate neuronal function with a higher level of detail than some of other parts of this project. Most of the other things involve duplicating brain modularity at a level above that of the neuron.

Finally, the paper discusses a confabulation program that has been developed. Confabulation means that the program can make up completions to partially blank sentences. The researchers trained the sentence completion program by feeding it a lot of text. The intelligent text recognition would perform differently depending on what author it had read. For instance when it was trained using only material written by Shakespeare, the confabulated (made up) sentences resembled ones produced by that author. The "Original" sentence is shown below. That is followed by the "Starter" words "Go to" that are fed to the computer. The "Completion" is the made up sentence that is based on the two starter words.

Original: “Go to the forge with it then shape it I would not have things cool.”

Starter: “Go to”

Completion: “Go to me at your convenient leisure and you shall know how I speed and the conclusion shall be “

The scientists have combined a few of these discrete models into one cohesive program. They have developed a hybrid BSB/neuronal model and a hybrid BSB/confabulation model. They have also investigated about the potential of scaling up the hierarchical bayesian model and the fixed point attractor network models to an entire brain.

It appears that they are getting some interesting results thus far. The hybrid BSB/confabulation model was able to recognize character fonts using a 16 by 16 pixel array. They fed the program sentences with characters missing (see picture left). When 20% of the characters/words were missing, the program was 99% perfect in identifying the correct missing letters. This success rate was due to the program being trained on reading material.

I've been skeptical myself of being able to copy the functioning of the mind in software. However a lot of this work is more inspired by how the brain works, instead of imitating it exactly. I think this research on neuromorphic AI looks promising. If scientists can meld aspects of what the human brain does best with what computers do best, I think we will see a lot of interesting things come out of it.