     \documentclass{amsart}
     \usepackage{amssymb,amsfonts,latexsym,amsmath}
%     \usepackage[dvips]{graphics}
\usepackage{graphicx}   
  \begin{document}
                              
     \title{Chapter 10
     Case Study: Neural Integrators}
     \begin{abstract} This chapter provides some
     applications of the present study on threshold
     transformations and dynamical systems of neural
     networks. In particlar, neural ingtegrators, which keep
     track of the position of eye movement, are described in
     our framework. 
     \end{abstract}
     
     \maketitle
     
     \section*{10.1 Introduction} 
     
     In dynamical systems of neural networks, as in any
     other dynamical systems, there are two kinds of
     stability, one structural another orbital. In
     discrete-time and finite-state models, the structural
     stability is ensured, if the generating function is
     expressed by a self-dual Boolean form. In the previous
     chapters, almost all results are expressed in Boolean
     functions, so that there is no problem about the
     structural stability.
     
          These chapters is primarilly concerned with the
     orbital stability. The orbital stability is expressed
     in terms of attractors. First, the enhanced Arimoto
     theorem (Theorem 5.5.2 of Chapter 5.5) asserts that
     there exists a first order system on $\mathbf{Q}^n$
     having a unique cycle of any length $k \leq 2^n$. This
     is an existence theorem for attractors. Starting at any
     initial state, the state will eventually be on the
     unique cycle. However, I thought this dynamical system
     was too strong. Often it is more desirable that the
     eventual states depend on initial states. Still, it is
     essential that the eventual states are orbitally
     stable. Therefore, the remaining chapters have been all
     devoted to existence of attractors that their basins
     for attraction are proper subsets of the whole
     state-space. Particularly, Chapter 6 and 7 are
     concerned with autonomous systems with spatial
     summation only, which are called first-order. Chapter 8
     is concerned with autonomous systems with spatial
     summation and temporal summation over two time points,
     which are called second-order. Chapter 9 is concerned
     with non-autonomous first-order systems, which receive
     an input sequence from their outside. These chapters
     clearly defined what attractors are and proved the
     existence of such attractors having particular cycle
     structures.
      
          Now, at more concrete level, "Does there exist a
     mechanism that sets and resets an initial state in real
     nervous systems?  If it does, then how?"  If a system
     is not evolving, that is, if the efficacies of its
     synaptic connections are not plastic but fixed, then
     only the input to the system from its outside can alter
     the initial state.  But how?
     
          Concerning this question, I encountered models of
     saccadic eye movements. It seems that a huge quantity
     of documents on such models have been published, but
     Van Gisbergen, Robinson, and Gielen (1981) have been a
     basic source. Their models like most others have, as
     their main components, neural integrators, which keep
     eye position, and burst neurons, which send pulses to
     the neural integrators.
     
          To my above question, it seems Seung et al
     (2000) seems most relevant. A common denominator of
     their research and mine is attractors in neural
     networks.  They characterize a neural integrator as a
     dynamical system having multiple attractors. Further,
     as the integrator receives pulses from burst neurons,
     new attractors of different firing rates are activated. 
     The integrator is a special case of a general memory
     model in which a piece of data is retrieved by input. 
     Therfore, my above general question had a clear answer.
     That is, input from burst neurons is responsible for
     resetting initial states of a dynamical system.
      
          However, the model of Seung et al (2000) is
     continuous as currently conventional dynamical systems
     of neural networks that are based on physiological
     studies. The continuous models may be better than
     discrete model in corporating physiological data.
     However, the original models cannot deal with
     population dynamics. Therefore, for example, the state
     space must be drastically simplified to the space of
     firing rates or its equivalent. In general, what are
     stable or attractive in a neural network are not the
     firing rates but a synchronic and diachronic specific
     firing patterns.  For example, the firing rates of the
     temporal firing patterns 10101010.. and 110011001100..
     are both 1/2, but one may be an attractor and the other
     may be not. For the firing patterns of a pair of
     neurons
     $$
     \begin{array}{lll}
        10101010101010.....& &10101010101010....\\
        01010101010101.....& &10101010101010....,
     \end{array}
     $$
     one may be an attractor and the other may be not. The
     simplified continuous models cannot distinguish these
     patterns.
     
          In contrast, our discrete dynamical systems based
     on the classical McCulloch and Pitts model do
     distinguish these firing patterns. In the same discrete
     framework, my project here is first to construct an
     autonomous NN(neural network) having multiple
     attractors of tonic firing patterns with different
     firing rates. Then the second part of the project is to
     construct a non-autonomous NN having two input
     connections, one excitatory and another inhibitory by
     determining appropriate efficacies for the connections.
     The third part is to determine a necessary burst
     pattern that resets the initial state in a push-pull
     manner when the pulses of the pattern are input to the
     excitatory and inhibitory connections.  That means a
     stable state changes from one tonic attractor to
     another tonic attractor. In summary, the project is to
     construct a discrete model of neural integrators. 
     Generation of such a burst pattern will be also
     addressed in section 10.7.
     
     
     \section*{10.2 Autonomous NN having multiple tonic
     attractors}
     
     Let $\mathbf{Q} = \{ 0,1\}$ be the minimal Boolean
     algebra with the binary operation $\cdot$ and $\vee$
     and the unary operation $\neg$.  For any positive
     integer $n$, $\mathbf{Q}^n$ is a Boolean algebra and
     also a metric space defined by the Hamming distance. 
     Let $x(t)$ be the state of a neuron at time $t$, where
     $t = 1,2,3,...,$ and $x(t) \in \mathbf{Q}$.  The
     sequence $x$ is called a \emph{tonic} sequence of
     firing rate $0 \leq i/m \leq 1$, where $m$ is a
     positive integer and $m$ and $i$ are relatively prime,
     if $x$ is a periodic sequence of period $m$, the number
     of 1s in $x(1), ...,x(m)$ is i, and 1s are the most
     uniformly distributed.  The last sentence is not
     rigorous, but practically will be clear.  For example,
     $1011010110.....$ is tonic with firing rate 3/5.
     $$
               101010010100101010010100101010010100...
     $$
     is tonic, but
     $$
               101010100100101010100100101010100100...
     $$
     is not tonic, although their firing rates are both
     5/12.  For a sequence $x$ in $\mathbf{Q}^{n}$, if every
     component sequence of $x$ is a tonic sequence of firing
     rate $i/m$, then $x$ is called a tonic sequence of
     firing rate $i/m$.
     
       A large number $m$ is necessary, if virtually
     continuous firing rates are required.  However, if an
     eye position is determined by an $(m+1)$-adic number
     corresponding to a set of firing rates of several
     integrators, then $m$ in each integrator can be
     small.
        
          For the first step for constructing a schematic
     neural integrator, it suffices to construct an
     autonomous network such that for each firing rate of 0,
     1/4, 1/2 and 3/4 there exists an attractor consisting
     of one or more tonic sequences of the firing rate. 
     These tonic sequences are 
     $$
     \begin{array}{l}
               00000000000000000000...\\
               10001000100010001000...\\
               10101010101010101010...\\
               11101110111011101110...\\
               11111111111111111111...
     \end{array}
     $$
     
     Here, a sequence obtained by shifting another is also
     tonic.  These tonic sequences may be expressed by just
     their first 4 terms, namely, $0000, 1101, 1011, 0101$
     and the like to be distinguished from each other.  We
     call them also \emph{tonic}.
        
          A simple method will be to construct a 4th-order
     NN of dimension 3.  In this case, 3 neurons are
     involved.  Starting from the initial states
     $$
     \begin{array}{l}
               x(1)x(2)x(3)x(4),\\
               y(1)y(2)y(3)y(4),\\
               z(1)z(2)z(3)z(4),
     \end{array}
     \eqno (10.2.1)
     $$
     the subsequent states are successively determined by a
     threshold function $H: \mathbf{Q}^{12} \to
     \mathbf{Q}^3$ and
     $$
     \begin{array}{ll}
          (x(i),y(i),z(i)) =& H(x(i-4),y(i-4),z(i-4),\\
          & \quad x(i-3),y(i-3),z(i-3),\\
          & \quad x(i-2),y(i-2),z(i-2),\\
          & \quad x(i-1),y(i-1),z(i-1)).
     \end{array}
     $$
     
     This fourth-order NN is equivalent to a first-order NN
     of dimension 12.  For this conversion we rename the
     above matrix of variables
     $x(1),y(1),z(1),...,x(4),y(4),(z4)$ to the
     12-dimensional vector by the correspondence, 
     $$
     \begin{array}{llll}
               1,& 4,& 7,& 10\\
               2,& 5,& 8,& 11\\
               3,& 6,& 9,& 12. 
     \end{array}
     $$
     The converted first-order NN is generated by the
     transformation $F$ of $\mathbf{Q}^{12}$ defined by
     $$
     \begin{array}{l}
          F: (v_1,....,v_{12}) \mapsto  (w_1,....,w_{12}),\\
          w_i = v_{i+3} \; \mbox{for $i = 1, ..., 9$,}\\
          (w_{10},w_{11},w_{12}) = H(v_1,v_2,...,v_{12}).
     \end{array}
     \eqno (10.2.2)
     $$
     In general, $F$ is defined by its component function
     $F_i = p_i F, i = 1,...,12$, but in the present case,
     only $F_{10} (= H_{10})$, $F_{11} (= H_{11})$, and
     $F_{12} (= H_{12})$ are yet to be determined.
     
          Let $\rho $ denote the cyclic permutation
     $(1,2,...,12)$.  Starting at (10.2.1), $H$ creates a
     tonic sequence such that 
     $$
          x(i+4) = x(i), y(i+4) = y(i),\; \mbox{and}\;
     z(i+4) = z(i)
     $$
     for every $i$, if and only if 
     $$
               Fv = \rho ^{-3}v
     $$                       
     for every $v =  \rho ^{-3j}u$ for every $j$, where
     $$
                u = (u_{1},u_2,..., u_{12}) =
     (x(1),y(1),...., z(4)).
     $$
     For example, 
     \begin{eqnarray*}
               x&=& 1110111011101110....\\
               y&=& 0111011101110111....\\
               z&=& 1101110111011101....,
     \end{eqnarray*}
     if and only if
     $$
          101.111.110.011 \mapsto 111.110.011.101      
     \mapsto 110.011.101.111 \mapsto 011.101.111.110 \mapsto 
     ....
     $$
     by $F$, where . is inserted every 3 components of a
     point in $\mathbf{Q}^{12}$.
     
          Let's call $v$ in $\mathbf{Q}^{12}$ tonic, if
     $(v_1,v_4,v_7,v_{10})$, $(v_2,v_5, v_8,v_{11})$, and
     $(v_3,v_6, v_9,v_{12})$ are all tonic with the same
     firing rate.  Let's determine an $F$ that satisfies a
     stronger condition 
     $$
               Fv = \rho ^{-3}v'
     \eqno (10.2.3)
     $$
     for any $v$ in the 1-neigborhood of any tonic $v'$ in
     $\mathbf{Q}^{12}$.  In (10.2.3), a sufficient condition
     for attractiveness of any tonic sequence is combined
     with the necessary and sufficient condition that any
     tonic sequence is a limit orbit.
      
          We assume $F$ is self-dual.  Then, $F_i$ is in
     turn defined by $f_i =  p_i \cdot \neg F_i$, its
     [\,]-representation.  Then, to determine $f_i$ means to
     determine the set of points $v$ such that $v_i = 1$ and
     $(Fv)_i = 0$.
       
          To determine $f_{10}$, we assume $v_{10} = 1$ and
     obtain some necessary conditions for $F_{10}v = 0$, $v$
     being in the 1-neighborhood of a tonic point, and
     (10.2.3) being satisfied.
      
          First, suppose $((Fv)_1,(Fv)_4,(Fv)_7,(Fv)_{10})
     =(1,1,1,0)$.  Then $v$ must be in the 1-neighborhood of
     a tonic point of firing rate 3/4.  Therefore, if $v_1 =
     0$, then $(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$ must
     contain at least one 0.  If $v_1 = 1$, then
     $(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$ must contain
     at least two 0s.
     
          Second, suppose $((Fv)_1,(Fv)_4,(Fv)_7,(Fv)_{10})
     =(1,0,1,0)$.  Then $v$ must be in the 1-neighborhood of
     a tonic point of firing rate 1/2.  Therefore, if $v_1 =
     0$, then $(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$ must
     contain at least three 0s.  If  $v_1 = 1$, then
     $(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$ must contain
     at least four 0s.
     
          Third, suppose $((Fv)_1,(Fv)_4,(Fv)_7,(Fv)_{10})
     =(0,0,1,0)$, which implies $v$ is in the1-neigborhood
     of a tonic point of firing rate 1/4.  Therefore, if
     $v_1 = 0$, then
     $(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$ must contain
     at least five 0s.  If $v_1 = 1$, then
     $(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$ contains at
     least six 0s.
     
          The case where $((Fv)_1,(Fv)_4,(Fv)_7,(Fv)_{10}) =
     (0,1,1,0)$ is out of our consideration, since
     $(0,1,1,0)$ is not part of a tonic point of firing rate
     $i/4$ (It can be part of a tonic sequence of firing
     rate 2/3).
     
          Combining the above conditions, we obtain the
     following necessary conditions.  If $(v_1,v_4,v_7) = 
     (0,1,1)$, then $F_{10}v = 0$ implies that
     $(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$ contains at
     least one 0s.  If $(v_1,v_4,v_7) =(1,1,1)$, then
     $F_{10}v = 0$ implies that
     $(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$ contains at
     least two 0s. If  $(v_1,v_4,v_7)  = (0,1,0)$, then
     $F_{10}v = 0$ implies that
     $(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$ contains at
     least three 0s.  In any other cases, $F_{10}v = 0$
     implies that $(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$
     contains at least four 0s.
     
          A function $F_{10}$ that satisfies the above
     conditions is, for example,
     $$ 
               f_{10} = p_{10} \cdot  (
          \neg p_1 \cdot p_4 \cdot p_7 \cdot S_1
          \vee p_4 \cdot p_7 \cdot S_2 
               \vee \neg p_1 \cdot S_3
               \vee S_4),
     \eqno (10.2.4)
     $$
     where $S_i$ is the disjunction of all conjunctions of
     $i$ functions selected from
     $$
     \{ \neg p_2, \neg p_3, \neg p_5, \neg p_6, \neg p_8,
     \neg p_9, \neg p_{11}, \neg p_{12}\} .
     $$
      
          We construct $F$ as a symmetric transformation in
     the sense $F \tau = \tau F$ for permutations $\tau =
     (1,2)(4,5)(7,8)(10,11)$, $(1,3)(4,6)(7,9)(10,12)$, and 
     $(2,3)(5,6)(8,9)(11,12)$.
     Therefore, $f_{11}$ and $f_{12}$ are immediately
     obtained from $f_{10}$.
     
          Unfortunately, $f_10$ is not a threshold function,
     so that the function $H$ must be realized by a
     composition of several neural circuits.  However, from
     the above construction, it will be easily proved that
     each of all 36 tonic cycles is an attractor.  Computer
     simulation also shows that there is no other cycle. 
     Therefore, leaving another construction for later time,
     I will use the autonomous NN generated by $F$ for the
     next stage in the to construct an integrator.    
     
     
     \section*{10.3 Non-autonomous NNs having burst input}
     
     We assume that pulses input from a burst neuron resets
     initial states, but what happens when it finished its
     job and becomes silent?  The synaptic connection for
     the burst neuron always exists.  This situation is
     totally different from the case where no input exists. 
     If the connection is excitatory, then a prolonged input
     of the resting potential produces inhibitory effects
     and takes the new state back to the original attractor
     and even further activates another attractor in the
     opposite direction.  Therefore, there must exist some
     other input that neutralizes the inverse effects during
     the time when the burst neuron is silent.  Fortunately,
     in a neural integrator for saccadic eye movements,
     there exist two kinds of synaptic connections in each
     integrator neuron for burst input, one inhibitory and
     the other excitatory.  When burst neurons connected two
     these synapses are both silent, the effects of resting
     potentials of both neurons cancel each other out.  This
     situation is the same as the case where no input
     exists, that is, an autonomous network   In short, eye
     position is kept steady not only by attractiveness of a
     tonic firing and but also by pairs of excitatory and
     inhibitory connections for burst neurons.
         
          Now let the vector sequences input from burst
     neurons to the inhibitory synapses of three neurons be
     $b$, and the vector sequence input from burst neurons
     to the inhibitory synapses of three neurons be $c$. 
     Both $b$ and $c$ are indexed as
     $$
     \begin{array}{lllllll}
               1,& 4,& 7,& 10,& 13,& 16,&...\\
               2,& 5,& 8,& 11,& 14,& 17,&...\\
               3,& 6,& 9,& 12,& 15,& 18,&....,
     \end{array}
     $$
     so that $b_1$ is the state input to the first neuron at
     time 1, $b_8$ is the state input to the second neuron
     at time 3, etc.
     
          To accept burst input from these sequences, we add
     $\mathbf{Q}^{24}$.  The weights of the synaptic
     efficacies of the input connections relative to the
     efficacies between the integrator neurons depend on the
     dimension and burst input and need some feedback from
     the next stage. In the present case, I make the
     relative weights equal to each other.
     
          Then, by modifying the transformation $F$ defined
     by (10.2.2) and (10.2.4), we obtain the function $G:
     \mathbf{Q}^{36} \to \mathbf{Q}^{12}$ defined by
     $$
               G:  (v_1,....,v_{36})  \mapsto 
     (w_1,....,w_{12}),
               w_i = v_{i+3}\; \mbox{for $i = 1, ..., 9$}. 
     $$
     $w_{i}$ for $i = 10, 11, 12$ are determined by $g_{i}$,
     where $g_{i} =  p_i \cdot \neg (p_i G)$, defined by
     $$
               g_{10} = p_{10} \cdot (
                    \neg p_1 \cdot p_4 \cdot p_7 \cdot S_5 
                    \vee p_4 \cdot p_7 \cdot S_6
                    \vee \neg p_1 \cdot S_7
                    \vee S_8,
     \eqno (10.3.1)
     $$
     where $S_i$ is the disjunction of all conjunctions of
     $i$ functions selected from 
     $$
               \{ \neg p_2, \neg p_3, \neg p_5, \neg p_6,
     \neg p_8, \neg p_9, \neg p_{11}, \neg p_{12}, 
                p_{13}, p_{16}, p_19, p_{22}, \neg p_{25},
     \neg p_{28}, \neg p_{31}, \neg p_{34}\} .
     $$
     $G$ is symmetric in the sense $G \tau = \tau G$ for
     permutations
     \begin{eqnarray*}
      \tau &=&
     (1,2)(4,5)(7,8)(10,11)(13,14)(16,17)(19,20)(22,23)(25,2
     6)(28,29)(31,32)(34,35)\;\\
      &\mbox{and}&\\
     \tau &=&
     (1,3)(4,6)(7,9)(10,12)(13,15)(16,18)(19,21)(22,24)(25,2
     7)(28,30)(31,33)(34,36).
     \end{eqnarray*}
          Therefore, $g_{11}$ and $g_{12}$ are immediately
     obtained from $g_{10}$.  $G$ is also self-dual, so that 
     $\neg p_i \cdot F_i = \bar{\neg}g_i$.
     
          The non-autonomous dynamical system $\varphi :
     \mathbf{Q}^{12} \times \mathbf{Z}_+ \to
     \mathbf{Q}^{12}$ generated by $G$ and the input
     sequence $b$ and $c$ is defined by
     \begin{eqnarray*}
               \varphi (v, 0) &=& v,\\
               \varphi (v, t) &=& G( \varphi (v, t-1),
     b_{1+3(t-1)},...,b_{12+3(t-1)}, c_{1+3(t-1)},...,
     c_{12+3(t-1)}).
     \end{eqnarray*}\\
     
     \section*{10.4  Determination of burst patterns} 
     
     Determination of burst patterns was unexpectedly hard. 
     I tried a lot of failed patterns before I hit the
     jackpot.
     
          First I tried a single pulse being input to three
     neurons with the same timing.  In this case, (10.3.1)
     was different in that more weight should be put on the
     burst input and the equation (10.3.1) should be more
     complex.  For example, in one of several variants of
     (10.3.1), in order that the tonic firing patterns were
     changed by a pattern of inhibitory burst neurons, from
     1111 to 1110, 1110 to 1010, 1010 to 1000, 1000 to 0000,
     the pattern 1110 was not only changed to 1010, but also
     further changed to 1000.
          Then, I tried some examples where one neuron
     receives a burst pulse after another neuron, so that,
     for example,
     $$
     \begin{array}{lllll}
          1111& &  & &           011101\\
          1111& \mapsto & ...& \mapsto& 101110\\
          1111&   &    &   &  110111.
     \end{array}
     $$
     Also, I changed the burst pattern from just one pulse
     000100000.. to 0001010000.. and the like. 
     Things improved.  Still, the best example I obtained
     failed in one case of timing between the burst pattern
     and one particular tonic sequence.
       
          We must consider at least two factors.
     1.  A desired change in each tonic pattern must occur
     at specific time points of the pattern.  For example,
     for the change from 1110 to 1010, the change must occur
     at the second time point. If this time point is missed,
     then another one period of 4 time points is required
     for the change. In the present model temporal summation
     is over 4 time points. Therefore, there is only one
     chance, if the change is made by one pulse of burst
     input. On the other hand, from the pattern 1111 to the
     pattern 1110, there are 4 chances in one period.  For
     one kind of change there is too few chances and for
     another kind of change too many chances.
     
          Attractiveness of these tonic patterns during the
     time when both inhibitory and excitatory neurons are
     silent is made by cooperation by the three neurons. 
     This cooperative nature is alive during the transient
     period of change from one pattern to another by burst
     input.  Therefore, a change in one neuron influences
     another neuron.  In particular, if two neurons are
     changed from one pattern to another pattern, the third
     neuron will be changed without any pulse of the burst
     neuron input thereto.
     
     My first trials where a single pulse is simultaneously
     input to the three neurons have no problem for the
     above factor 2, but were severely affected by the above
     factor 1.  My attempts to diversify the phases of
     patterns and increasing the number of burst pulses
     relieved the factor 1 but brought some effects of the
     factor 2. Therefore, a neuron receives burst pulses
     later than another should receive fewer burst pulses.
       
          After several trials in this way, I finally got an
     example where one burst pattern can change each tonic
     pattern to another tonic pattern one step lower, when
     starting from
     $$
     \begin{array}{l}
               111111111...\\
               111111111...\\
               111111111...
     \end{array}
     $$
     The burst pattern is:
     $$
     \begin{array}{l}
               101010000000...\\
               000010100000...\\
               000000000000...
     \end{array}
     $$
          Here, the first line is the burst input to neuron
     1, the second is the burst input to neuron 2, and the
     third is the burst input to neuron 3.  The dynamical
     system is self-dual, so that starting from
     $$
     \begin{array}{l}
               000000000\\  
               000000000\\
               000000000,
     \end{array}
     $$
     with excitatory burst input of the same pulse pattern,
     the firing of the three neuron reaches
     $$
     \begin{array}{l}
               111111111...  
               111111111...
               111111111...
     \end{array}
     $$
     through 4 changes, each change occurring whenever the
     pulse pattern is input.

          I have not checked all cases where excitatory
     burst input is followed by inhibitory burst input and
     vice versa. But it seems the push-pull property is also
     satisfied.  
     
     
     \section*{10.5  Circular threshold constructions}
     
     Now we address the pending problem that the function
     $H$ in the transformation $F$ in Section 10.2 is not a
     threshold function.  It seems impossible to modify the
     function $f_i$ to a threshold function such that the
     modified F has tonic attractors of firing rates $i/4, i
     = 0,1,2,3,4$.   Therefore, we try to construct a
     totally new network.
  
          First, let us consider only the following types of
     tonic sequences of four neurons.
     $$
     \begin{array}{lllll}
     00000...& 100010...&01010...& 011101...& 11111...\\
     00000...& 010001...&10101...& 101110...&11111...\\
     00000...&001000...&01010...&110111...&11111...\\
     00000...& 000100...&10101...& 111010...& 11111...
     \end{array}
     $$
     Then, we can distinguish these tonic sequences of
     firing rates $i/4$ from each other in any of their
     synchronic cross sections.
     
     For example, the first and third cross sections are
     respectively
     $$
     \begin{array}{lllll}
          0&   1&   0&   0&   1\\
          0&   0&   1&   1&   1\\
          0&   0&   0&   1&   1\\
          0&   0&   1&   1&   1
     \end{array}
     $$
     and
     $$
     \begin{array}{lllll}
               0&   0&   0&   1&   1\\
               0&   0&   1&   1&   1\\
               0&   1&   0&   0&   1\\
               0&   0&   1&   1&   1
     \end{array}
     $$
     That is, these tonic sequences can be distinguished by
     their cross sections, which are elements in
     $\mathbf{Q}^4$.  We call the firing rate in each cross
     section a synchronic firing rate in contrast to a
     diachronic firing rate, which is the normal definition
     of firing rate.  In the present case, each synchronic
     firing rate of a tonic sequence is equal to its
     diachronic firing rate.  However the minimum distance
     between the cross sections of two different sequences
     is 1.  Therefore, in order to separate the cross
     sections at least by distance 3 and make each tonic
     sequence an attractor in a dynamical system, we
     consider the state space of $\mathbf{Q}^{12}$.  The
     tonic sequences now become
     $$
     \begin{array}{lllll}
     00000...& 100010...&01010...& 011101...&11111...\\
     00000...& 010001...&10101...& 101110...&11111...\\
     00000...& 001000...&01010...& 110111...&11111...\\
     00000...& 000100...&10101...& 111011...&11111...\\
     00000...& 100010...&01010...& 011101...&11111...\\
     00000...& 010001...&10101...& 101110...&11111...\\
     00000...& 001000...&01010...& 110111...&11111...\\
     00000...& 000100...&10101...& 111011...&11111...\\
     00000...& 100010...&01010...& 011101...&11111...\\
     00000...& 010001...&10101...& 101110...&11111...\\
     00000...& 001000...&01010...& 110111...&11111...\\
     00000...& 000100...&10101...& 111011...&11111...
     \end{array}
     $$
     The first and the last sequences are constant-term
     sequences in $\mathbf{Q}^{12}$.  The second and fourth
     are cyclic sequences of period 4, and the third is the
     cyclic sequence of period 2.  Also the component
     sequence is the right shift of the immediately above
     component sequence. 

          We now try to construct a first-order NN of
     dimension 12, in which the above 5 tonic cyclic
     sequences are attractors.  After several attempts, I
     obtained the following self-dual circular threshold
     transformation $F$ of $\mathbf{Q}^{12}$ defined by $F =
     \langle f_1\rangle $,
     $$ 
          f_1 = p_1 \cdot S_2\{ \neg p_4, \neg  p_8, \neg 
               p_{12}\} .
     \eqno (10.5.1)
     $$
     
     This threshold translation is described in Examle
     7.2.5 of Chapter 7 for general dimension.  In fact,          
     according to Chapter 7.2, this NN has only 6 cycles
     and all of them are strong attractors. 
     Five of them correpond to those illustrated above, and
     the other correspond to the non-tonic cyclic sequence
     of period 4,
     $$
     \begin{array}{l}
               11001100...\\
               01100110...\\
               00110011...\\
               10011001...\\
               11001100...\\
               01100110...\\
               00110011...\\
               10011001...\\
               11001100...\\
               01100110...\\
               00110011...\\
               10011001...
     \end{array}
     $$
          The next problem is how to resolve this unwelcome
     non-tonic attractor.  However, it seems again
     impossible to modify the circular threshold
     transformation to another having only the five tonic
     attractors.  For non-threshold constructions, see
     Section 10.8.  we use this autonomous NN to construct a
     neural integrator.  An advantage is that we can easily
     generalize the dimension of this transformation. 
     However, there will be more and more non-tonic
     attractors as the dimension becomes higher.
     
          A non-autonomous network having burst input for an
     integrator can now be defined by the mapping $G$: 
     $\mathbf{Q}^{36} \to \mathbf{Q}^{12}$, $G = \langle
     g_1\rangle $,
     $$
          g_1  =  p_1 \cdot (S_2\{\neg p_4, \neg  p_8, 
               \neg p_{12}\} \cdot S_1\{ p_{13}, \neg
     p_{25}\} \vee S_2\{ p_{13}, \neg p_{25}\} ),
     \eqno (10.5.2)
     $$
     where the indices 13 to 24 respectively represent the
     inhibitory burst input to neuron 1 to 12, and the
     indices 25 to 36 respectively represent the excitatory
     burst input to neuron 1 to 12.  See Chapter 9.3 for the
     notation $\langle g_1\rangle $ (Modify the definition
     for one input 13 to 24 into two inputs 13 to 24 and 25
     to 36). 
     
     
     \section*{10.6  Functions of burst generators}
     
     The functions of a generator of burst sequences to be
     input to a neural integrator defined by (10.5.2) should
     be very sophisticated and complex. 
          First, the generator supplies pulses so that
     non-tonic attractors should be avoided.  Moreover, the
     generator supplies pulses with exact timing.  For
     example, in order to reduce a firing rate from 3/4 to
     1/2, at a time when the state (cross section) is
     $$
               111011101110,
     \eqno (10.6.1)
     $$
     
     and the excitatory burst generator is silent, the
     inhibitory burst generator supplies a pulse to at least
     two of neurons 3, 7, 11.   If pulses are input to
     neurons 3, 7, and 11, then the next state is
     $$
               0101001010111.
     $$
     By the attractiveness, the next state will be a tonic
     cross section of firing rate 1/2, if no more pulses are
     supplied.  Therefore, a tonic sequence of firing rate
     1/2 is obtained.  However, if the inhibitory burst
     generator supplies pulses to neurons 1 and 5 at the
     state (10.6.1), then there is no change in the firing
     rate in the next cross section, just a circular shift. 
     If the inhibitory burst pulses are supplied to neurons
     2 and 6, then the next cross section is
     $$
               001100110111.
     $$
     By the attractiveness, the next state will be a
     non-tonic cross section of firing rate 1/2.  Therefore,
     a non-tonic sequence will succeed, when both burst
     generators are silent.
          According to Van Gisbergen, Robinson, and Gielen
     (1981), the heart of the local feedback hypothesis is
     that the output $E'$ of the integrator is relayed back
     to burst generators.  The burst generators also receive
     a command signal $E_d$ from higher centers.  "Burst
     cell is driven by a signal proportional to motor error,
     which is the difference between where the eye is ($E'$)
     and where it should be ($E_d$)" (p. 419).
      
          For our integrator network defined by (10.5.2),
     the matter is not only of firing rate but also of
     precise timing.  Therefore, designing a neural circuit
     of the burst generators and the local feedback system
     is another combinatorial problem to be studied in the
     next stage.\\
     
     
     \section*{10.7  Generator-integrator feedback system}
     
     Now, we try to construct a burst generator and a local
     feedback system for the burst generator and integrator. 
     Van Gisbergen, Robinson, and Gielen (1981) contains a
     proposed model, which is not a neural circuit in a
     strict sense but rather a conceptual block diagram for
     the feedback system.  The proposed model simulates
     experimental data in terms of firing rate.  But what is
     a neural circuit that receives two signals and outputs
     a signal having the firing rate of the difference
     between the firing rates of two input signals.  To
     construct such a circuit will require resolving the
     same kind of difficulties as the generator-integrator
     feedback system does, such as timing between two
     signals and limitations of threshold functions.  To
     describe the generator-integrator feedback system in
     terms of the "difference" of two signals is as 
     tautological as to say that an integrator gives a
     signal whose firing rate is the sum of the firing rates
     of two signals.  In our following model, it is the
     burst generator itself that produces a kind of
     difference as burst signals.
     
          Here our purpose is not to simulate a real nervous
     system for saccadic eye movements.  Instead, we are
     concerned with an idealized abstract system in contrast
     to statistical analysis and parameter physics that
     simulate experimental data, where firing rate
     represents a crude abstraction of reality.  Therefore,
     we try to construct neural circuits of a feedback
     system, as a basis for a concrete system, in terms of
     pulse signals and Boolean functions in place of firing
     rates.  Still, this approach deals with intrinsic
     properties of neural circuits, such as timing and
     threshold functions.  We do not take into account the
     time required for transmission of signals between the
     burst generator and the integrator, but slight
     modification will easily incorporate the delays.
     
          When we describe a system in terms of 0-1 pulse
     signals and not in terms of firing rate, we can define
     the difference of two signals, that is, two sequences x
     and $y$ in $\mathbf{Q}^n$ by a pair of $h$ and $c$
     defined by,
     $$
     \begin{array}{lll}
               h(t+1) &=& x(t) \cdot \neg y(t),\\
               c(t+1) &=& y(t) \cdot \neg x(t).
     \end{array}
     \eqno (10.7.1)
     $$
     For example, consider the following three sequences in
     $\mathbf{Q}^4$.  Note that the firing rate of the first
     is 3/4. The second and the third sequences differ only
     in their phases, and their firing rate is 1/4. 
     $$
     \begin{array}{lll}
          101110111011&  100010001000&  010001000100\\
          110111011101&  010001000100&  001000100010\\
          111011101110&  001000100010&  000100010001\\
          011101110111&  000100010001&  100010001000
     \end{array}
     $$
     For the first two sequences, $h$ and $c$ are
     respectively
     $$
     \begin{array}{ll}
               001100110011&       000000000000\\
               100110011001&       000000000000\\
               110011001100&       000000000000\\
               011001100110&       000000000000
     \end{array}
     $$
     Therefore, $h$ is a sequence having a firing rate 1/2
     that is the difference of the firing rates of the first
     and the second.  However, for the first and the third
     sequences, $h$ and $c$ are respectively
     $$
     \begin{array}{ll}
               101110111011&       010001000100\\
               110111011101&       001000100010\\
               111011101110&       000100010001\\
               011101110111&       100010001000
     \end{array}
     $$
     Here, the firing rate of $h$ is 3/4 and the firing rate
     of $c$ is 1/4.  Therefore, we have not obtained a
     sequence of firing rate 1/2.  Therefore, we have faced
     another timing problem to produce a signal having the
     difference firing rate of two signals.  In order to
     avoid a complex timing problem, it is better not to
     seek such a signal but to use the pair $(h,c)$ for the
     difference of two signals.

          Now assume that the above sequence $x$ is an
     integrator signal and y is a command signal.  Further
     assume that h is the inhibitory burst signal, and c is
     the excitatory burst signal.  Then we can construct the
     following feedback system. 
     
     \begin{figure}[h]
     \centering
       \scalebox{.5}{\includegraphics{fback.eps}}
     \caption{\label{Fig. 1}}
\end{figure}
     
     where, besides (10.7.1),
     $$
     \begin{array}{lll}
               x_{t+1} &=& G(x(t), h(t), c(t)),\quad G =
     \langle g_1\rangle ,\\ 
               g_1  &=&  p_1 \cdot (\neg p_4 \cdot (p_5 \vee
     p_9) \vee p_5 \cdot \neg p_9).
     \end{array}
     \eqno (10.7.2)
     $$
     (10.7.2) is a simplified function of (10.5.2) obtained
     by removing attractiveness.  If we consider the state
     space $\mathbf{Q}^{12}$ in place of $\mathbf{Q}^4$, we
     can use (10.5.2) as it is.
          Two problems should be addressed now.  First, for
     an integrator to change the firing rate 3 /4 of $x$ to
     1/4, it must receives $h(t)$ and $c(t)$ that may,
     depending on the timing between $x$ and the command
     signal as described above, be simultaneously active at
     some time $t$ from the burst generator.   This does not
     pose any serious mathematical problems.  It also agrees
     with reality, since Van Gisbergen, Robinson, and Gielen
     (1981, p.418) says, "... during any saccade, ... burst
     neurons on both sides of the brain stem often are
     active simultaneously."
          
          The second problem was already discussed in the
     last section.  The function $G$  is very sensitive to
     the timing between the inputs $x$ and $(h, c)$.  In
     fact, the above feedback system defined by (10.7.1) and
     (10.7.2) does not work.  There are two time lags
     between the measuring of the difference and the
     realization of changing the synchronic firing rate of
     $x$.  At the same time period, if all components of $h$
     and $c$ are all 0, then the cross section of $x$ will
     round-shifted by 2. Therefore, (10.7.1) should be
     modified to
     $$ 
     \begin{array}{lll}
          h(t+1) &=&  \rho^2 (x(t) \cdot \neg y(t)),\\
          c(t+1) &=&  \rho^ 2 (y(t) \cdot \neg x(t)),
     \end{array}
     \eqno (10.7.3)
     $$
     where $\rho$ is the cyclic permutation (1,2,3,4). 
     Clearly $h$ and $c$ are threshold functions.  Computer
     simulation expects that this feedback system will work. 
     For example, for the first and third sequences in the
     beginning of this section respectively as $x(t)$ and
     $y(t)$, we obtain 
     $$
     x(t): \begin{array}{l}
               10000100\\
               11100010\\
               11010001\\
               01001000
     \end{array}
     $$
     $$
     y(t): \begin{array}{l}
               01000100\\
               00100010\\
               00010001\\
               10001000
     \end{array}
     $$
     $$
     h(t): \begin{array}{l}
               01100000\\
               00100000\\
               01000000\\
               01100000
     \end{array}
     $$
     $$
     c(t): \begin{array}{l}
               00000000\\ 
               01000000\\
               00100000\\
               00000000
     \end{array}
     $$
     
     Next, we prove a theorem that validates the
     generator-integrator feedback system constructed in
     above.  The state spaces are now $\mathbf{Q}^m$, and
     $\rho$  is the cyclic permutation $(1,2,...,m)$.  Let
     $y$ be a tonic sequence in $\mathbf{Q}^m$.  If $y(t+1)
     = \rho y(t)$ for every $t$, we call $x$ a
     \emph{circular tonic sequence}.
     
          $G$ is a circular function: $\mathbf{Q}^{3m} \to
     \mathbf{Q}^m$, that is,  $\rho G(u,v,w)   = G(\rho u,
     \rho v, \rho w)$ for $u, v, w \in \mathbf{Q}^m$, 
     defined by $g_1 = p_1 \cdot \neg (p_1 G)$, that is, $G
     = \langle g_1\rangle$, 
     $$
          g_1  =  p_1 \cdot
           ( \neg p_m \cdot (p_{m+1} \vee \neg p_{2m+1}) 
          \vee p_{m+1}\cdot \neg p_{2m+1}).
     \eqno (10.7.4)
     $$
     The command signal $y(t)$ is a given circular tonic
     sequence in $\mathbf{Q}^m$, and the integrator signal
     $x(t)$ is a sequence in $\mathbf{Q}^m$ defined by 
     $$
               x_{t+1} = G(x(t), h(t), c(t)).
     \eqno (10.7.5)
     $$
     The burst signals $h(t)$ and $c(t)$ are sequences in
     $\mathbf{Q}^m$ defined by 
     $$
     \begin{array}{lll}
          h(t+1) &=&  \rho ^2 (x(t) \cdot \neg y(t)),\\
          c(t+1) &=&  \rho ^2 (y(t) \cdot \neg x(t))
     \end{array}
     \eqno (10.7.6)
     $$
     In the following, $M = \{ 1,...,m \}$ is regarded as
     the residue class ring with m as the zero element.  For
     example, $1 - 2 = m - 1$.  Further, $o = (00,..,0) \in
     \mathbf{Q}^m$. 
     
     \textbf{Theorem 10.7.1}  Let $h(0) = c(0) = o$.  Then,
     $x(2) = y(2)$, and $x(3) = y(3)$.
     
     \begin{proof}   Let $h(1)_k = 1$ and $c(1)_k = 0$. 
     Clearly $x(2)_k = 0$.  On the other hand, we have
     $x(0)_{k-2} = 1$ and $y(0)_{k-2} = 0$, since $h(1)_{k}
     = 1$.  Therefore, $y(2)_k = y(0)_{k-2} =  0$. 
     Therefore, $x(2)_k = y(2)_k$.  By self-duality, if
     $h(1)_k = 0$ and $c(1)_k = 1$, then $x(2)_k =
     y(2)_k$.
     
          Next, let $h(1)_k = c(1)_k = 0$.  If $x(1)_k = 1$
     and $x(1)_{k-1} = 0$ then $x(2)_k = 0$.  If $x(1)_k =
     1$ and $x(1)_{k-1} = 1$, then $x(2)_k = 1$.  Therefore,
     by self-duality, $x(2)_k = x(1)_{k-1}$.  On the other
     hand, $x(0)_{k-2} = y_{k-2}$, since $h(1)_k = c(1)_k =
     0$.  Therefore, $x(1)_{k-1} = x(0)_{k-2}$, since $h(0)
     = c(0) = o$.  Therefore, $x(2)_k = x(0)_{k-2}$. 
     Clearly $y(2)_k = y(0)_{k-2}$.  Therefore, $x(2)_k =
     y(2)_k$.    Therefore, $x(2) = y(2)$.
     
          Further, $x(1) = \rho x(0)$ and $y(1) = \rho
     y(0)$.  Therefore,
     \begin{eqnarray*}
          h(2) &=&  \rho^2 (x(1) \cdot \neg y(1)) = \rho^3
     (x(0) \cdot \neg y(0)) = \rho h(1).\\
          c(2) &=&  \rho^ 2 (y(1) \cdot \neg x(1)) = \rho^3
     (y(0) \cdot \neg x(0)) = \rho c(1) .
     \end{eqnarray*}
          Let $h(2)_k = 1$.  Then $h(1)_{k-1} = 1$, so that
     $x(2)_{k-1} = 0$.  Therefore, if $x(2)_k = 1$, then
     $x(3)_k = 0 = x(2)_{k-1}$. If $x(2)_k = 0$, then
     $x(3)_k = 0 = x(2)_{k-1}$.  If $c(2)_k = 1$, then
     $x(3)_k = 1 = x(2)_{k-1}$ by self-duality.  Next, let
     $h(2)_k = c(2)_k = 0$.  Then clearly 
     $x(3)_k = x(2)_{k-1}$.  Therefore,
      $x(3) = \rho x(2) = \rho y(2) = y(3)$.
     \end{proof}
     
     The above Theorem 10.7.1 shows that if $h(0) = c(0) =
     o$, then, $x(2) = y(2)$ and $x(3) = y(3)$, so that
     $h(3) = c(3) = o$, so that $x(4) =  y(4),...$ 
     Therefore, if $h(0) = c(0) = o$, then $x(t) = y(t)$ for
     every $t \geq 2$, and $h(t) = c(t) = o$ for every $t
     \geq 3$.   Therefore, if $y$ changes to another
     circular tonic sequence at some time $t' \geq 3$, then
     $x(t) = y(t)$ for every $t \geq t'+2$, since $h(t') =
     c(t') = o$.  It is well known that omni-pause neurons
     make burst generators silent.  The input from the omni
     pause neurons to the burst generator is one feature
     that is absent from the present generator-integrator
     feedback model.
     


     \end{document}
     