ALEXANDER ASILADOR
Tue Apr 26, 2016
The aim of this project is to capture the spatial learning mechanism for Rattus norvegicus. To model this, I have chosen to use a training paradigm that has been well studied in the rat (Morris 1984), this is the Morris water maze. The purpose of the Morris water maze is to identify the ability of the rat to perform spatial learning in a variety of tasks. Spatial learning will be modeled as Reinforcement learning and implemented by a neural network. The bot was designed to have two sensors to represent eyes. These eyes are capable of detecting the distance of the landmarks. Each eye is capable of detecting 180 degrees, but have a 76 degree overlap in front of the bot to closely resemble the visual system. As a consequence, the bot cannot see 360 degrees about it. The resulting 'blind spots' cause the sensors to read 'zero' and still contribute to the input of the system. In this simulation, the bot (rat model) was randomly placed in the maze. The location of the landmarks and platform were fixed to allow learning in the training phase. The training phase consisted of 15 trials at 6000 ticks. The culmination of each trial will reset the animal After training period, a testing phase would be 1 trial under the same conditions.
The hypothesis entering this project was that spatial learning, detailed by the standards of the Morris water maze in rodents, can be simplified into a 3-layer 10 neuron neural network model.
To determine success, we will compare training timing between simulation and real data by measuring the difference between the two. If difference is less than standard deviation of training of n experiments, then the neural network model is a success
The initial simulations revealed a large issue with wall avoidance and was added to the tests. Simulations were run for the following scenarios:
| Number of Layers | Number of Neurons | Wall Avoidance |
| 3 - layer neural network | 10 neurons | no |
| 3 - layer neural network | 15 neurons | no |
| 4 - layer neural network | 50 neurons | no |
| 4 - layer neural network | 50 neurons | yes |
Q values |
During the training phase, the bot would get stuck to a wall and causing a bias towards the BACKWARD action. The result of the testing phase produced a bot that constantly travelled backwards.
Similarly to the 3-layer neural network, the bot had difficulty escaping from the wall upon collision. A hand-coded wall avoidance measure was included in this simulation (parameters included below) These results show that the agent is capable of spatial learning using a four-layer 50 neuron network. Separate training sessions reveal that the network sometimes fails to converge to the platform within the 15 trials.
| Controller | Fitness mean (std dev) |
|---|
The trade-off in mobility and number of free parameters (i.e. action outputs) were apparent in this simplified model. The actions were chosen by trial-and-error. One of the major obstacles observed in this simulation was the large impact that the blind spot had to the neural network training phase. Occasionally, the bot would get stuck on the wall or choose a region that is far away from the target are.
The simulation was hand-coded to include a way for the bot to escape from a wall collision event. This was accomplished by changing the heading direction 180 degrees at the collision of a wall. Ideally, hand-coding would not be needed but was needed as a method to determine whether spatial learning can be compared to a well-studied model in cognitive neuroscience. It is difficult to determine which extra parameter would improve of the model, but fewer neurons in a 4-layer network should be considered as one of the adjusted parameters.
This project was driven by the utility of combining behavioral biology, cognition, and sensory physiology.
Morris, R. 1984. "Developments of a water-maze procedure for studying spatial learning in the rat." J Neurosci Methods. 11:47-60