\chapter{Review of Current Technology}


	\hsp{8mm} 
	Currently there are two major tools being utilized by researchers
	to model neural activity. These two are NEURON\cite{nn:tnse} 
	and GENESIS\cite{nn:tbogernmwtgenesis},
	and their corresponding parallel implementations. These tools
	are used primarily to model single cells or small networks of
	cells in an extremely detailed manner. These tools are excellent
	for neuroscientists who wish to test or model the behavior
	of specific compartments within a cell in response to a given
	input. In subsection 2.1, we give a more detailed account of
	NEURON and GENESIS.

	In terms of spike encoding, current technology in this area spans
	a very broad area, ranging from neuromorphic analog VLSI design
	to theoretical models of memory-less state machines. In subsection
	2.2, we will cover a high level account of these areas. 

	\section{Current Biological Neural Network Modeling Tools}

	\hsp{8mm}
	Other simulators currently in use today consist of NEURON
	and GENESIS. These simulations first began with a sequential
	implementation and have recently developed parallel versions. Each
	of these simulators models the constituent cells in the network
	in very intricate detail, that is so coupled that the manuals
	on these simulators say you can only compute small realistic
	simulations. Both NEURON and GENESIS calculate the Hodgkin-Huxley
	equations \cite{mnm:mcacd}(which will be discussed in a later section) at each
	step in the simulation run\cite{nn:tnse,nn:tbogernmwtgenesis}.

	Furthermore, the cells are modeled with precise anatomical
	accuracy, in that the neuron contains on the order of thousands
	of dendrites, each with its own highly detailed dynamics being
	performed\cite{nn:tnse,nn:tbogernmwtgenesis}. However, 
	in each simulator, synaptic dynamics
	are virtually ignored. While GENESIS models only a loose Hebbian
	and anti-Hebbian synaptic weight modification 
	algorithm\cite{syn:aafmnrpbopapsst},
	NEURON does neither synaptic learning nor Redistribution of
	Synaptic Efficacy\cite{syn:opfadogabaiasitn}. GENESIS does, 
	however, allow the users to
	write their own script for synaptic modification and incorporate
	that into the simulation\cite{nn:tbogernmwtgenesis}.

	Because of this fine detail of activity within the single neuron,
	the overall network of cells - in both NEURON and GENESIS - are
	very sparsely connected\cite{nn:tbogernmwtgenesis,nn:enromwn}. 
	Thus, the parallel implementations
	of these simulations utilize a coarse grain parallelism approach,
	in which one multi-compartment cell is modeled on one processor,
	such an example was recently published in \cite{nn:alsmotccupg}, 
	where a single
	Purkenje cell was allocated to a single processor on a Cray
	T3E. In GENESIS, furthermore, the user must specify in their input
	file how the cells are to be distributed among the different
	nodes that are to run in parallel\cite{nn:tbogernmwtgenesis}. 
	Each simulator utilizes
	Parallel Virtual Machine (PVM) \cite{pvm:book} as it's parallel platform.  

	In addition, the modeling language in which NEURON is written is a
	proprietary and interpretive language\cite{nn:enromwn}. To modify the existing
	NEURON code, or run complex simulations based on an input file,
	one must learn the language of NEURON first, making salability
	more difficult\cite{nn:enromwn}.

	\section{Current Spike Encoding Systems}

	\hsp{8mm}
	Spike encoding of a particular message is a very important way
	to send information as it allows a multi-dimensional analysis
	of data. The encoding of information comes from the combination
	of several factors including temporal, phase, and inter-spike
	interval variability\cite{pnn:eiina}. In temporal encoding, the information
	is encoded in the rate of spikes per some time interval. The
	reigning hypothesis states that the stronger the signal, or the
	greater the significance of the signal, the more spikes there will
	be within a given time interval. This has been shown to be true
	in regards to the motor cortex, in which a stimulus to a muscle
	will result in greater spikes in response to greater stimulus
	\cite{pnn:eiina}. 

	In phase encoding, the placement of the spikes within some
	interval and their overall oscillatory pattern is used to encode
	information \cite{pnn:eiina}. In addition, the inter-spike interval is used to
	encode a third dimension of data. In this case, it is the varying
	time intervals between successive spikes that encodes data. All
	three of these combined allows for a three dimensional encoding
	of data, which is currently in use in satelite
	communication~\cite{datacom}. 

	In other areas of spike encoding, researchers have developed
	models to represent learning in a neural network. In 
	\cite{nn:liannmirturws}, the
	authors present their findings from running a simple simulation
	using the IQR421 software modeling tool. Their primary focus
	is real-time processing, and as such, small networks are
	employed. A Fast Fourier Transform (FFT) is performed on the
	audio input and the excitatory neurons in the network receives
	an additional input from a noise source firing at 10Hz with a
	Poisson distribution. 

	In the network, the populations of excitatory neurons project
	one-to-one onto interneurons. They employed Hebbian learning
	principles to train the network on audio input. Their results
	shows that they could process this network in real-time. 

	Other researchers, such as \cite{pnn:apccifns}, are creating 
	biologically accurate
	cortical models within analog VLSI chips. What they are calling
	a Silicon Cortex (SCX) is a hybrid analog-digital electronic
	system fabricated with CMOS VLSI technology. They used visual
	and auditory signals as input because of the analog properties
	inherent in these  waveforms. 

	Their hardware design was done using an analog VLSI (aVLSI) where
	the creative algorithm consisted of making as many neurons and
	synapses in hardware while conserving the costly  real estate of
	the microchip. The neuronal and synaptic design was conducted
	using resistors to emulate current flow across membranes, and
	capacitors to store state values. 

	The communication paradigm had to be optimized for space by
	handling inter-chip communication with an asynchronous digital
	multiplexing Address-Event Representation (AER) array. The
	inter-neuronal communications were tested using one of two
	systems: hardwired and arbitered. In the hardwired case, the
	neurons were connected directly to the address encoder, where each
	spike activates the encoder to send the action potential signal
	to the destination synapse. This proved to be very high-speed
	and simple, but collision of incoming spikes was a problem since
	this produced erroneous destination addresses. 

	This resource contention of the address encoder lead to the second
	design, the arbitrated communication paradigm. In this system,
	an arbitrator is interposed between the sending neuron and the
	address encoder, in which collided spikes are either queued
	or discarded. This implementation gave the highest throughput
	but also gave some timing uncertainty because of queuing or
	discarding of inputs to the arbitor. 

	The communication itself is handled via an address loo up within
	an object called the Local Address Event Bus (LAEB). The
	purpose of this is to take the spike event (which is merely a
	broadcast of the senders address), look up the sender's address
	in the table, and send this spike notification to the destination
	synapses corresponding to this sending neuron. Because of hardware
	limitations, the current maximum number of neurons on any given
	chip is $10^4$.

	The communication within the SCX is asynchronous, relying on
	the random firing pattern of silicon neurons. Because this is
	implemented in hardware, and most communication is local to the
	chip, processing is in real time. However, in a Poisson spike
	train, timings of spikes will become out of phase, and results
	and biological accuracy are sacrificed. 

	In more theoretical models of biological neural networks, 
	\cite{lsm:cinmanabop}
	presents a new approach to computational neural networks. This
	theory is a hybrid model of finite state machines (in which
	neuronal activity is broken into discrete time slices and
	synchronized accordingly), and integrate-and-fire biological
	neurons (in which neural activity is asynchronously timed and all
	neurons store "data" for the current moment only). The authors
	call their new approach a "Liquid State Machine" (LSM)  in
	which neurons are the liquid and the mathematical principles
	form the state functions. 

	Their goal in producing this LSM is to model neural activity in
	real time and with low memory usage. In order to achieve this,
	they propose a "memory-less"  model where the current state
	of the liquid represents the present and past inputs to the
	liquid. Additionally, these liquids are not given multiple time
	constants for relaxation and all interaction is local.

	The authors categorize two main factors important to the
	model. The first is the separation property (SP), which is
	similar in design to the inter-spike interval encoding of other
	systems. The second factor is the approximation category, AP,
	which governs the transference of internal states to target
	outputs. It is with both of these factors that the mathematics
	of the LSM create the memory-less system.

	In their results, they show a liquid consisting of 135 neurons,
	randomly connected with a relative probability, and an input
	of randomly generated spike trains injected into 30\% of the
	cells. They used perceptron-like learning rules to train the
	synapses. Their network classified the two Poission spike trains
	that were input. Their hope is that this theoretical approach can
	serve as a basis for novel neuromorphic engineering techniques.
