\chapter{System Design}\label{sec:sd}

	
		\hsp{8mm} 
		In this chapter we will cover the computational design of
		the neocortical simulator. The first section consists of
		a biological background of neocortical mechanics. This
		biological discussion is necessary for understanding
		of the design decisions that were made in order to
		effectively model the cortex. In the computational
		section, we will go into detail of how each object is
		classified, and what functions the biological objects
		perform. For a more detailed account of neural processing,
		please see \cite{neuro:neurobook}, 
		an excellent text on the subject.

			
	\section{Biological Topology}

		\hsp{8mm} 
		The neocortex is organized into structural and functional
		units, each one containing other elements or performing
		specific functions. The anatomical constructs are the
		columns and layers, which contain such physiological
		constructs as neurons, synapses, and channels. Within
		the brain, the columns are highly localized areas of
		neural activity, with each column containing multiple
		layers. Each layer contains various quantities of
		heterogeneous clusters of neurons. Each neuron is
		comprised of multiple dendrites, one soma, one axon,
		multiple synapses and multiple channels. The synapses are
		the connecting points between two distinct neurons and
		their strength is modulated during neuronal activity. The
		channels on the other hand are a part of the membrane
		wall and are active regardless neural communication.

		Connectivity among neurons within the brain is best
		described as being highly connected within layers, less so
		within a column, and much less connectivity across column
		boundaries. Neurons communicate with each other through
		binary events called action potentials, or spikes. No
		healthy neuron is an island, and it is the relationships
		between neurons in which all activity occurs. 
		Figure~\ref{fig:viscort} 
		shows a set of columns in the visual cortex.

	\begin{figure}
	\epsfxsize=4in
	\centerline{\epsffile{chap03/VisColumn.ps}}
	\caption{A Photograph of columnar activity in visual cortex\cite{neuro:neurobook}} \label{fig:viscort}
        \end{figure}


  	   \subsection{Neurons}
		\hsp{8mm} 
		Physiologically, individual neurons can be grouped
		into particular  categories based upon their membrane
		properties, channel dynamics, and the types of synapses
		they employ in connecting to other neurons. For example,
		an excitatory, Pyramidal  cell has a reversal potential
		of 0 (zero) milliVolts, and connects with other cells
		using a type of synapse that encourages the receiving
		cell to become active (excitatory synapse). On the other
		hand, an interneuron has a reversal potential of -80
		milliVolts, and connects with other cells using a type of
		synapse that dampens the activity of the receiving cell
		(inhibitory synapse).

		The channels that each cells utilize also differ
		significantly. For example, many interneurons contain
		channels that suppress cell activity, while the excitatory
		cells contain only mildly suppressing channels. 

		A neuron itself is comprised of multiple compartments,
		such as basal and apical dendrites, a soma, an axon
		hillock, and an axon. The dendrites form the input
		structure to the center of the neuron, the soma. The
		axon hillock is the location on the soma where an action
		potential, or spike, is generated, and the axon is the
		compartment that carries the spike out of the neuron to
		other neurons that are connected to this one. 
		Figure~\ref{fig:pyrneuron}
		shows a photo of a pyramidal neuron. 

	\begin{figure}
	\epsfxsize=4in
	\centerline{\epsffile{chap03/Neuron.ps}}
	\caption{A Photograph of a Pyrramidal neuron\cite{neuro:neurobook}} \label{fig:pyrneuron}
        \end{figure}

		The channels line the cell membrane and influence
		cell behavior, these will be discussed in sub-section
		3.1.4. The synapses form the connections between two
		distinct cells and can attach at any point within
		a neuron, from the furthermost dendrite to the soma
		directly, as well as to the axon hillock itself. These
		will be discussed in subsection X. 

		In the soma, all incoming currents are aggregated and a
		decision is calculated as to whether or not threshold
		has been reached. If it has, then a resulting action
		potential is to be fired. The action potential is a
		spike shape of very short duration (on the order of
		seven to ten microseconds), which ranges in amplitude
		from the cell threshold (typically -40mV) to 40mV. The
		spike shape is considered a binary event because it
		is of such short duration as compared to the waveform
		output of the synapse, which is on the order to one to
		two hundred microseconds. 

		Once a cell fires an action potential, this signal is
		then output through the axon to all of the other neurons
		with which this cell connects. This spike shape triggers
		the synapses on the connecting (post-synaptic cells) to
		release their neurotransmitters. This chemical	response
		results in a post-synaptic current (PSC) waveform
		being delivered onto the receiving (post-synaptic)
		compartment's membrane. The PSC is aggregated into
		the post-synaptic compartment's total current, and
		potentially contributes to a new action potential being
		generated. This begins anew the cycle of a binary event
		(spiking action potential) triggering an analog event
		(the PSC), which triggers a binary event, and so on. This
		process of aggregation and action potential generation
		is called "integrate-and-fire"\cite{bio:boc} and it is through these
		steps that cells within a given network communicate with
		one another. For those wishing to learn more about this
		process of integrate-and-fire, please see \cite{bio:monm} 
		and/or \cite{bio:boc}.

	   \subsection{Channels}

		\hsp{8mm} 
		The channels on the membrane of the cell contribute to
		propagating the Post Synaptic Current (PSC) from the
		synapses by opening/closing themselves to allow/block
		the entrance of certain ions. This ion flow changes the
		membrane potential of the compartment, and input to the
		soma, contributing or inhibiting the generation of an
		action potential. All channels to be discussed here are
		potassium ($K^{+}$) channels and most contain activating (m)
		and inactivating (h) particles
		\cite{mnm:mcacd,chan:kcrofpidohpn}.  
		Figure~\ref{fig:biochan} shows
		a photograph of a channel.


	\begin{figure}
	\epsfxsize=4in
	\centerline{\epsffile{chap03/BioChannel.ps}}
	\caption{A Photograph of a channel, left shows side view, right shows
		top-down view \cite{chan:csottdofspc}} \label{fig:biochan}
        \end{figure}

		The M-Channel, of the $K^{+}$ channel family, has only an
		activation particle. This channel becomes active when an
		incoming spike shape is received by the parent cell. Its
		general behavior is that of inhibiting its parent cell
		from reaching threshold, resulting in a suppression of the
		output spike shape. Since this channel does not contain an
		inactivation particle (h), it reaches a steady state of
		activity gradually, relative to the time constant. This
		can be seen in \cite{syn:opfadogabaiasitn} where the 
		output spike train becomes
		a regular pattern (constant inter-spike interval). 

		The A-Channel, also of the $K^{+}$ channel family, helps
		the cell to wash out background noise. It does this by
		opening more as the cell approaches threshold, and then
		shutting off once threshold has been reached. Unlike
		the M-Channel, this one contains an both and activation
		(m) and an inactivation (h) particle, which allows its
		dynamics to turn off after a certain time period. This
		activity can be seen in \cite{syn:opfadogabaiasitn} 
		where there resulting spike
		train of the parent cell has shows delayed behavior. 

		The AHP-Channel, the After Hyper Polarization Channel,
		is a voltage independent, Calcium ($Ca^{2+}$) dependent
		channel. Like the other channels of the $K^{+}$ family, this
		channel confers a generally suppressing behavior onto its
		host cell. the name is derived from the functionality
		of the channel. The hyper-polaraizing process occurs
		when the membrane voltage makes a drastic jump from a
		very high value (+40mV) to a value that is between the
		resting potentials of the host cell and the channel. This
		is the negative dip in the spike shape waveform. 

		With each outgoing action potential, $Ca^{2+}$ enters the cell,
		forcing the channel to open more, which hyper-polarizes
		the cell's membrane voltage. This channel slows the
		spiking rate of its host cell down as the resulting
		action potentials progress, until the cell reaches a
		steady state of spiking. The effect of the AHP channel
		can be seen in \cite{syn:opfadogabaiasitn} where the 
		channel accommodates the
		cell spiking (gradually increasing inter-spike intervals
		until a steady state is reached).

 	   \subsection{Synapses}

		\hsp{8mm} 
		The synapses are the contextual filters whose
		effectiveness is modified based on the timing of the input
		to each cell. They are also responsible for converting
		the binary action potential input signal into a resulting
		analog post-synaptic current (PSC) via neurotransmitter
		release. The synapses are classified as either excitatory
		or inhibitory. The excitatory PSC is a positive waveform,
		and contributes positively to the post-synaptic cell
		reaching threshold and firing an action potential. The
		inhibitory PSC is a negative waveform, and suppresses
		the post-synaptic cell's ability to reach threshold,
		thereby inhibiting a resulting spike.

		The effectiveness of each synapse is directly
		determined based on the amount of input it receives
		from the pre-synaptic cell, and on the amount of
		signals outgoing from the post-synaptic cell. There
		are two major algorithms that determine the amount of
		neurotransmitters released at each pre-synaptic signal,
		and these are Redistribution of Synaptic Efficacy and
		Hebbian Learning. The amount of neurotransmitters ready
		to be released is quantified by a percent availability
		term called Utilization of Synaptic Efficacy, or USE
		(hereafter called \verb+U_mean+). When a synapse releases
		more neurotransmitters, its signal is amplified;
		and conversely, when a synapse releases less
		neurotransmitters, its signal is dampened. 

		The Redistribution of Synaptic Efficacy algorithm states
		that a synapse has a finite amount of neurotransmitters to
		be released, and that amount is decreased or increased
		depending on the depression and facilitation time
		constants, and upon the initial value of \verb+U_mean+. For
		example, in a train of incoming spike shapes, the
		synapse will release less neurotransmitters in response
		to each spike if the following conditions hold: The
		depression time constant (tau) is greater than the
		facilitation tau, and the \verb+U_mean+ is large enough to
		accommodate significant decay. Conversely, the amount
		of neurotransmitters released with each spike will be
		increased if the facilitation tau is higher than the
		depression tau, and the baseline \verb+U_mean+ value is low
		enough to have room to room to grow. 

		The Hebbian learning algorithm is another set of
		principles that determine the effectiveness of a given
		synapse \cite{syn:tncbnpndonrp}. Unlike the RSE, 
		this algorithm is dependent
		on both pre- and post-synaptic spike timing. This
		algorithm is described as follows: There is a given
		window in which positive and/or negative learning can
		take place, and at the center of this moving window
		is the resulting, post-synaptic spike of the cell. The
		position of the pre-synaptic spike within this learning
		window determines the updating of the \verb+U_mean+ value. 

		If the pre-synaptic spike occurs within the positive
		learning window (before the post-synaptic spike), then
		the \verb+U_mean+ value is increased, thus strengthening the
		connection between these two cells. Conversely, if the
		pre-synaptic spike occurs within the negative learning
		window (after the post-synaptic spike), the \verb+U_mean+ value
		is decreased, thus weakening the connection between these
		two cells. Additionally, if the pre- and post-synaptic
		spike occur at the same time, or the post-synaptic spike
		occurs outside either the positive or negative window,
		the \verb+U_mean+ value remains unchanged. 

		Through Hebbian learning the synapse will strengthen
		a connection in which the pre-synaptic cell directly
		contributed to a post-synaptic spike, and will weaken a
		connection in which the pre-synaptic cell made a negative
		contribution to the post-synaptic action potential. 

	\section{Computational Topology}

		\hsp{8mm} 
		The computational topology is closely modeled after
		the biological principles mentioned in section XX. In
		this simulator, the user specifies a biological and/or
		administrative template for each segment of the model,
		and the program create the amount of elements needed,
		as well as creating their inter-relationship
		\cite{des:tadslfwccp}. 

		For example, the brain template specifies the duration and
		frequency of the simulation (administrative) and which
		columns to include and their inter-column connectivity
		(biological). Other objects, such as the spike shape and
		post synaptic conductance (PSG) waveform are modeled
		through static templates. The choice for making some
		processes into templates was done to enhance performance
		on a very large-scale networks, thus the trade off
		between network size and computational complexity. 

		\hsp{8mm} 
		The choice for an object-oriented design for this
		simulator was made because the biological brain is
		segmented into distinct but interrelated parts. The
		object-oriented paradigm allows the simulator to model
		objects generically, changing their behavior through
		the input parameters without affecting the underlying
		object functionality. Operation and reporting is based on
		parameters specified in a text input file. In this way,
		a user can rapidly model multiple brain regions merely
		by changing input parameters. 

		The object-oriented design also allows the user to scale
		the network size more easily. By changing only the input
		file, this simulator can model very large numbers of
		cells and various connections strengths, which affects
		the amount of synapse objects and communication. The
		design also allows for a modeling of very large numbers
		of channels and external stimulus all within a given
		network. 

		This system design enables object modularity, in
		which one object implementation can be swapped out for
		another because functionality is encapsulated within
		that object. For example, we have employed different
		communication paradigms by swapping out the MessageBus
		object with another MessageBus object that implements
		communication differently. 

		On an implementation level, there are several basic
		designs for the objects within the simulator. Some
		are containers for other objects and maintainers of
		administrative data; others are both containers and state
		machines; and others are simply state machines, and do
		not contain any other objects. Secondarily, there are
		other objects that perform specific functions important
		to the computational side, but not directly affecting
		the biological system. Of the objects described here,
		the state machines are very important to the simulation
		as they perform very specific biological tasks that
		directly impact the behavior of the system, and from
		which biological data is garnered. 

		In the following subsections, we will describe the
		object-oriented design of the simulator. Each primary
		functional object will be illustrated along with a
		discussion of its relevance to the system as a whole,
		and its relationship to other objects within the model.

	   \subsection{Brain Object}
		The brain object is both a container of other objects and
		a repository for administrative data. It is responsible
		for ensuring that each owned object is visited and
		updated. It contains the column, layer, cell, stimulus,
		report, and the synapse helper objects. 

		At each time step in the simulation the brain visits the
		following objects: Each stimulus object (to send data to
		the cells); visits each report object (to collect data
		on the cells); visits the MessageBus object (to ensure
		communication between the cells on any node within the
		cluster); and visits each cell/compartment object (to
		process the input and change the state of the overall
		simulator). 

		On the administrative side, the brain is responsible for
		such duties as calculating the computational duration of
		the simulator (in ticks per second), and keeping track
		of the passage of simulation time. The brain object also
		holds the location of the signal trap, allowing the user
		to exit gracefully from the program at any point in time,
		without losing output report data. 
		Figure~\ref{fig:oobrain} shows an
		object-oriented layout of the brain object.

	\begin{figure}
	\epsfxsize=4in
	\centerline{\epsffile{chap03/BrainObject.ps}}
	\caption{A Representation of the Brain.} \label{fig:oobrain}
        \end{figure}

	   \subsection{Cell Object}

		The cell is another container object; it holds all of
		the compartment objects for the particular neuron being
		modeled. The cell object is responsible for ensuring
		that the messages delivered to it are passed on to the
		appropriate compartment, whether that message originated
		outside the cell or within the cell (from a sibling
		compartment). The cell is also responsible for ensuring
		that all of its constituent compartments are visited, to
		allow compartmental processing of their data. Of these
		contained compartments, only the soma is the minimum
		required to mimic a point to point model. 
		Figure~\ref{fig:oocell}
		shows an object-oriented model of a neuron.

	\begin{figure}
	\epsfxsize=4in
	\centerline{\epsffile{chap03/CellObject.ps}}
	\caption{A Representation of a Neuron.} \label{fig:oocell}
        \end{figure}

	   \subsection{Compartment Object}

		The compartment object is both a container and a state
		machine. The compartment object contains the synapse,
		channel, and spike shape objects. It is a state machine
		in that its membrane voltage is calculated and updated
		at each time step.

		It is a generic object whose functionality is based
		upon user specified input parameters. For example,
		only one compartment object is used to model all of
		the compartments within the cell - specifically, the
		basal dendrites, the apical dendrites, an axon and a
		soma. Everything is based on the variables that were
		specified in the user input file. This includes whether
		or not this particular compartment will fire an action
		potential, and if so, which spike shape it will utilize,
		and to which cells it will send this binary spike event.

		The compartment object houses functionality that
		makes up the heart of the cortical simulator. All
		the integrate-and-fire routines exist here, all action
		potentials originate here, and all synapses and channels
		are updated here. The compartment changes state at each
		timestep regardless of external input, because it's
		membrane voltage degrades by a pre-set amount towards
		resting potential. 

	   \subsection{Synapse Object}
		The synapse object is owned by the compartment object
		and is functionally a state machine. Its state change
		is triggered by an input from its host cell. This input
		can take the form of either a pre-synaptic (incoming)
		spike shape, or a post-synaptic (outgoing) spike shape. 

		This object incorporates all of the Redistribution of
		Synaptic Efficacy equations \cite{syn:opfadogabaiasitn}, 
		as well as all of the
		Hebbian learning algorithms \cite{syn:aafmnrpbopapsst}. 
		The PSG waveform is
		modeled as a template that is input by the user. It's
		amplitude is modulated by a scaling factor, and is
		typically on the order of 100 microseconds in length. The
		decision to model the PSG waveform as a template was
		made in order to cut down on computationally expensive
		calculations that would consume process time in a very
		large network. In this way certain biomechanics were
		traded for performance and scalability. 

	   \subsection{Channel Object}

		The channel object is also owned by the compartment object
		and is also primarily a state machine. The channel is a
		generic object that is designed to model  any type of
		channel. It's state change takes place at every time
		step in the simulation, unlike the synapse object,
		whose state change is triggered by a specific event. 

		The channel equations in use are modeled after channels
		discussed in \cite{mnm:mcacd}, which utilize the activation and
		inactivation particles, voltage and tau values to
		calculate channel current. This current is output to
		the host cell at each timestep in the simulation. The
		channels currently being modeled include the following:
		Non-Inactivating Muscarinic Potassium Current (M-Channel);
		Delayed, Rectifying Potassium Current (A Channel);
		Non-Inactivating Calcium-Dependent Potassium Current
		(AHP Channel). 

		Each channel, when visited by the compartment, is given
		the compartment's membrane voltage at that timestep,
		and in the case of the AHP channel, the Calcium level of
		the compartment. In the process of the channel current
		calculations, certain parameters are saved for use in
		the next state. 

	   \subsection{Stimulus Object}

		The stimulus object is of the category of objects that
		perform specific functional tasks but are not necessarily
		state machines. This object is used to mimic signals that
		originate from external courses, such as a voltage or
		current clamp, as well as ongoing activity from neurons
		in other areas of the brain (not currently being modeled
		explicitly).The stimulus object is able to inject into
		its recipient cells either a voltage or a current that
		is calculated at pre-determined time interval. 

		The stimulus object is designed to model the following
		types: a file based audio input (normalized from decibels
		to frequency, and then converted into stochastic
		current stimuli)\cite{math:numrecipbook}; 
		a linear (ramp) function; a pulsed
		(staircase or constant) function; and a sinusoid waveform
		function. Superposition of multiple stimuli allows the
		user to mimic background noise in a community of cells,
		or to enhance synaptic learning. 

		The destination of these stimuli is determined
		by another object called the StimulusInject, whose
		lifetime is limited to the initialization phase of the
		simulation.  The StimulusInject allows for a probabilistic
		determination of receiving cells. This design allows
		us to input the same stimulus into multiple groups of
		cells without having to create another stimulus object,
		this conserving memory and enhancing performance. 

	   \subsection{Report Object}

		The report object is another object that performs
		a functional task but does not change state. It
		handles the data processing of what to report on, which
		cell/compartment to receive this report, at what frequency
		(in time steps, not seconds), and at what points in
		simulation time the report is to be generated. Currently,
		the user is able to report on a range of information about
		the compartment, including: voltage; stimulus current;
		synaptic current; channel current; leak current; net
		current; and current from adjacent compartments. The
		parameters of what to report on, which cells to receive
		this report request and what output file name are all
		specified in the user input file.

		The report object communicates with the cells via the
		MessageBus object. The report requests are generated
		in the report object when it is visited by the brain
		object during the main simulation loop and sent to the
		MessageBus. The MessageBus in turn delivers the report
		request to the specified cells/compartments. Finally,
		the reported data is handed off to the FileIO object
		for processing in a separate thread. 

	   \subsection{FileIO and PostProcessing Objects}
		The FileIO object handles the file input/output within a
		separate thread from the computation thread. It outputs
		the data from a cell/compartment to a given temporary
		file (located on the local hard drive of the node in
		the cluster). This temporary file is later processed
		by the PostProcessing object, which formats it into a
		more user friendly display. The PostProcessing object
		deletes the temporary file from the local hard drive,
		and a final, user friendly output data file is sent to
		the head node for viewing by the user.

	   \subsection{Message Object}
		The Message object is a collection of data that is
		used in communication between all objects within the
		simulator. All of the necessary messages are instantiated
		at program inception, and pointers to this pool are
		passed around by various objects. The implementation of
		this will be discussed in section XX.

	   \subsection{MessageBus Object}

		One of the most important functional objects is the
		MessageBus object \cite{des:ssoacsm}. Its detailed 
		model will be discussed
		in section XX, however, it is responsible for all of
		the message passing between cells, stimulus and cells,
		reports and cells, and cells and FileIO. It encapsulates
		all of the Message Passing Interface (MPI)
		\cite{mpi:ref,mpi:book,mpi:pacheco} calls, and
		is responsible for all inter-nodal communication. It
		is utilized at all stages of the simulator, from
		initialization to simulation end. It is also responsible
		for synchronizing the globally distributed set of brains,
		so as to prevent race conditions, deadlock and starvation
		among the nodes on the cluster. 
