\documentclass{amsart}
\usepackage{amssymb,amsfonts,latexsym,amsmath}
\usepackage{graphics}
\begin{document}
\title{Chapter 10
Case Study: Neural Integrators}
\begin{abstract} This chapter provides some
applications of the present study on threshold
transformations and dynamical systems of neural
networks. In particlar, neural ingtegrators, which keep
track of the position of eye movement, are described in
our framework.
\end{abstract}
\maketitle
\section*{10.1 Introduction}
In dynamical systems of neural networks, as in any
other dynamical systems, there are two kinds of
stability, one structural another orbital. In
discrete-time and finite-state models, the structural
stability is ensured, if the generating function is
expressed by a self-dual Boolean form. In the previous
chapters, almost all results are expressed in Boolean
functions, so that there is no problem about the
structural stability.\\
These chapters is primarilly concerned with the
orbital stability. The orbital stability is expressed
in terms of attractors. First, the enhanced Arimoto
theorem (Theorem 5.5.2 of Chapter 5.5) asserts that
there exists a first order system on $\mathbf{Q}^n$
having a unique cycle of any length $k \leq 2^n$. This
is an existence theorem for attractors. Starting at any
initial state, the state will eventually be on the
unique cycle. However, I thought this dynamical system
was too strong. Often it is more desirable that the
eventual states depend on initial states. Still, it is
essential that the eventual states are orbitally
stable. Therefore, the remaining chapters have been all
devoted to existence of attractors that their basins
for attraction are proper subsets of the whole
state-space. Particularly, Chapter 6 and 7 are
concerned with autonomous systems with spatial
summation only, which are called first-order. Chapter 8
is concerned with autonomous systems with spatial
summation and temporal summation over two time points,
which are called second-order. Chapter 9 is concerned
with non-autonomous first-order systems, which receive
an input sequence from their outside. These chapters
clearly defined what attractors are and proved the
existence of such attractors having particular cycle
structures.\\
Now, at more concrete level, "Does there exist a
mechanism that sets and resets an initial state in real
nervous systems? If it does, then how?" If a system
is not evolving, that is, if the efficacies of its
synaptic connections are not plastic but fixed, then
only the input to the system from its outside can alter
the initial state. But how?\\
Concerning this question, I encountered models of
saccadic eye movements. It seems that a huge quantity
of documents on such models have been published, but
Van Gisbergen, Robinson, and Gielen (1981) have been a
basic source. Their models like most others have, as
their main components, neural integrators, which keep
eye position, and burst neurons, which send pulses to
the neural integrators.\\
To my above question, it seems Seung et al
(2000) seems most relevant. A common denominator of
their research and mine is attractors in neural
networks. They characterize a neural integrator as a
dynamical system having multiple attractors. Further,
as the integrator receives pulses from burst neurons,
new attractors of different firing rates are activated.
The integrator is a special case of a general memory
model in which a piece of data is retrieved by input.
Therfore, my above general question had a clear answer.
That is, input from burst neurons is responsible for
resetting initial states of a dynamical system.\\
However, the model of Seung et al (2000) is
continuous as currently conventional dynamical systems
of neural networks that are based on physiological
studies. The continuous models may be better than
discrete model in corporating physiological data.
However, the original models cannot deal with
population dynamics. Therefore, for example, the state
space must be drastically simplified to the space of
firing rates or its equivalent. In general, what are
stable or attractive in a neural network are not the
firing rates but a synchronic and diachronic specific
firing patterns. For example, the firing rates of the
temporal firing patterns 10101010.. and 110011001100..
are both 1/2, but one may be an attractor and the other
may be not. For the firing patterns of a pair of
neurons
$$
\begin{array}{lll}
10101010101010.....& &10101010101010....\\
01010101010101.....& &10101010101010....,
\end{array}
$$
one may be an attractor and the other may be not. The
simplified continuous models cannot distinguish these
patterns.\\
In contrast, our discrete dynamical systems based
on the classical McCulloch and Pitts model do
distinguish these firing patterns. In the same discrete
framework, my project here is first to construct an
autonomous NN(neural network) having multiple
attractors of tonic firing patterns with different
firing rates. Then the second part of the project is to
construct a non-autonomous NN having two input
connections, one excitatory and another inhibitory by
determining appropriate efficacies for the connections.
The third part is to determine a necessary burst
pattern that resets the initial state in a push-pull
manner when the pulses of the pattern are input to the
excitatory and inhibitory connections. That means a
stable state changes from one tonic attractor to
another tonic attractor. In summary, the project is to
construct a discrete model of neural integrators.
Generation of such a burst pattern will be also
addressed in section 10.7.
\section*{10.2 Autonomous NN having multiple tonic
attractors}
Let $\mathbf{Q} = \{ 0,1\}$ be the minimal Boolean
algebra with the binary operation $\cdot$ and $\vee$
and the unary operation $\neg$. For any positive
integer $n$, $\mathbf{Q}^n$ is a Boolean algebra and
also a metric space defined by the Hamming distance.
Let $x(t)$ be the state of a neuron at time $t$, where
$t = 1,2,3,...,$ and $x(t) \in \mathbf{Q}$. The
sequence $x$ is called a \emph{tonic} sequence of
firing rate $0 \leq i/m \leq 1$, where $m$ is a
positive integer and $m$ and $i$ are relatively prime,
if $x$ is a periodic sequence of period $m$, the number
of 1s in $x(1), ...,x(m)$ is i, and 1s are the most
uniformly distributed. The last sentence is not
rigorous, but practically will be clear. For example,
$1011010110.....$ is tonic with firing rate 3/5.
$$
101010010100101010010100101010010100...
$$
is tonic, but
$$
101010100100101010100100101010100100...
$$
is not tonic, although their firing rates are both
5/12. For a sequence $x$ in $\mathbf{Q}^{n}$, if every
component sequence of $x$ is a tonic sequence of firing
rate $i/m$, then $x$ is called a tonic sequence of
firing rate $i/m$.\\
A large number $m$ is necessary, if virtually
continuous firing rates are required. However, if an
eye position is determined by an $(m+1)$-adic number
corresponding to a set of firing rates of several
integrators, then $m$ in each integrator can be
small.\\
For the first step for constructing a schematic
neural integrator, it suffices to construct an
autonomous network such that for each firing rate of 0,
1/4, 1/2 and 3/4 there exists an attractor consisting
of one or more tonic sequences of the firing rate.
These tonic sequences are
$$
\begin{array}{l}
00000000000000000000...\\
10001000100010001000...\\
10101010101010101010...\\
11101110111011101110...\\
11111111111111111111...
\end{array}
$$
Here, a sequence obtained by shifting another is also
tonic. These tonic sequences may be expressed by just
their first 4 terms, namely, $0000, 1101, 1011, 0101$
and the like to be distinguished from each other. We
call them also \emph{tonic}.\\
A simple method will be to construct a 4th-order
NN of dimension 3. In this case, 3 neurons are
involved. Starting from the initial states
$$
\begin{array}{l}
x(1)x(2)x(3)x(4),\\
y(1)y(2)y(3)y(4),\\
z(1)z(2)z(3)z(4),
\end{array}
\eqno (10.2.1)
$$
the subsequent states are successively determined by a
threshold function $H: \mathbf{Q}^{12} \to
\mathbf{Q}^3$ and
$$
\begin{array}{ll}
(x(i),y(i),z(i)) =& H(x(i-4),y(i-4),z(i-4),\\
& \quad x(i-3),y(i-3),z(i-3),\\
& \quad x(i-2),y(i-2),z(i-2),\\
& \quad x(i-1),y(i-1),z(i-1)).
\end{array}
$$
This fourth-order NN is equivalent to a first-order NN
of dimension 12. For this conversion we rename the
above matrix of variables
$x(1),y(1),z(1),...,x(4),y(4),(z4)$ to the
12-dimensional vector by the correspondence,
$$
\begin{array}{llll}
1,& 4,& 7,& 10\\
2,& 5,& 8,& 11\\
3,& 6,& 9,& 12.
\end{array}
$$
The converted first-order NN is generated by the
transformation $F$ of $\mathbf{Q}^{12}$ defined by
$$
\begin{array}{l}
F: (v_1,....,v_{12}) \mapsto (w_1,....,w_{12}),\\
w_i = v_{i+3} \; \mbox{for $i = 1, ..., 9$,}\\
(w_{10},w_{11},w_{12}) = H(v_1,v_2,...,v_{12}).
\end{array}
\eqno (10.2.2)
$$
In general, $F$ is defined by its component function
$F_i = p_i F, i = 1,...,12$, but in the present case,
only $F_{10} (= H_{10})$, $F_{11} (= H_{11})$, and
$F_{12} (= H_{12})$ are yet to be determined.\\
Let $\rho $ denote the cyclic permutation
$(1,2,...,12)$. Starting at (10.2.1), $H$ creates a
tonic sequence such that
$$
x(i+4) = x(i), y(i+4) = y(i),\; \mbox{and}\;
z(i+4) = z(i)
$$
for every $i$, if and only if
$$
Fv = \rho ^{-3}v
$$
for every $v = \rho ^{-3j}u$ for every $j$, where
$$
u = (u_{1},u_2,..., u_{12}) =
(x(1),y(1),...., z(4)).
$$
For example,
\begin{eqnarray*}
x&=& 1110111011101110....\\
y&=& 0111011101110111....\\
z&=& 1101110111011101....,
\end{eqnarray*}
if and only if
$$
101.111.110.011 \mapsto 111.110.011.101
\mapsto 110.011.101.111 \mapsto 011.101.111.110 \mapsto
....
$$
by $F$, where . is inserted every 3 components of a
point in $\mathbf{Q}^{12}$.\\
Let's call $v$ in $\mathbf{Q}^{12}$ tonic, if
$(v_1,v_4,v_7,v_{10})$, $(v_2,v_5, v_8,v_{11})$, and
$(v_3,v_6, v_9,v_{12})$ are all tonic with the same
firing rate. Let's determine an $F$ that satisfies a
stronger condition
$$
Fv = \rho ^{-3}v'
\eqno (10.2.3)
$$
for any $v$ in the 1-neigborhood of any tonic $v'$ in
$\mathbf{Q}^{12}$. In (10.2.3), a sufficient condition
for attractiveness of any tonic sequence is combined
with the necessary and sufficient condition that any
tonic sequence is a limit orbit.\\
We assume $F$ is self-dual. Then, $F_i$ is in
turn defined by $f_i = p_i \cdot \neg F_i$, its
[\,]-representation. Then, to determine $f_i$ means to
determine the set of points $v$ such that $v_i = 1$ and
$(Fv)_i = 0$.\\
To determine $f_{10}$, we assume $v_{10} = 1$ and
obtain some necessary conditions for $F_{10}v = 0$, $v$
being in the 1-neighborhood of a tonic point, and
(10.2.3) being satisfied.\\
First, suppose $((Fv)_1,(Fv)_4,(Fv)_7,(Fv)_{10})
=(1,1,1,0)$. Then $v$ must be in the 1-neighborhood of
a tonic point of firing rate 3/4. Therefore, if $v_1 =
0$, then $(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$ must
contain at least one 0. If $v_1 = 1$, then
$(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$ must contain
at least two 0s.\\
Second, suppose $((Fv)_1,(Fv)_4,(Fv)_7,(Fv)_{10})
=(1,0,1,0)$. Then $v$ must be in the 1-neighborhood of
a tonic point of firing rate 1/2. Therefore, if $v_1 =
0$, then $(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$ must
contain at least three 0s. If $v_1 = 1$, then
$(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$ must contain
at least four 0s.\\
Third, suppose $((Fv)_1,(Fv)_4,(Fv)_7,(Fv)_{10})
=(0,0,1,0)$, which implies $v$ is in the1-neigborhood
of a tonic point of firing rate 1/4. Therefore, if
$v_1 = 0$, then
$(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$ must contain
at least five 0s. If $v_1 = 1$, then
$(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$ contains at
least six 0s.\\
The case where $((Fv)_1,(Fv)_4,(Fv)_7,(Fv)_{10}) =
(0,1,1,0)$ is out of our consideration, since
$(0,1,1,0)$ is not part of a tonic point of firing rate
$i/4$ (It can be part of a tonic sequence of firing
rate 2/3).\\
Combining the above conditions, we obtain the
following necessary conditions. If $(v_1,v_4,v_7) =
(0,1,1)$, then $F_{10}v = 0$ implies that
$(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$ contains at
least one 0s. If $(v_1,v_4,v_7) =(1,1,1)$, then
$F_{10}v = 0$ implies that
$(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$ contains at
least two 0s. If $(v_1,v_4,v_7) = (0,1,0)$, then
$F_{10}v = 0$ implies that
$(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$ contains at
least three 0s. In any other cases, $F_{10}v = 0$
implies that $(v_2,v_3,v_5,v_6,v_8,v_9,v_{11},v_{12})$
contains at least four 0s.\\
A function $F_{10}$ that satisfies the above
conditions is, for example,
$$
f_{10} = p_{10} \cdot (
\neg p_1 \cdot p_4 \cdot p_7 \cdot S_1
\vee p_4 \cdot p_7 \cdot S_2
\vee \neg p_1 \cdot S_3
\vee S_4),
\eqno (10.2.4)
$$
where $S_i$ is the disjunction of all conjunctions of
$i$ functions selected from
$$
\{ \neg p_2, \neg p_3, \neg p_5, \neg p_6, \neg p_8,
\neg p_9, \neg p_{11}, \neg p_{12}\} .
$$
We construct $F$ as a symmetric transformation in
the sense $F \tau = \tau F$ for permutations $\tau =
(1,2)(4,5)(7,8)(10,11)$, $(1,3)(4,6)(7,9)(10,12)$, and
$(2,3)(5,6)(8,9)(11,12)$.
Therefore, $f_{11}$ and $f_{12}$ are immediately
obtained from $f_{10}$.\\
Unfortunately, $f_10$ is not a threshold function,
so that the function $H$ must be realized by a
composition of several neural circuits. However, from
the above construction, it will be easily proved that
each of all 36 tonic cycles is an attractor. Computer
simulation also shows that there is no other cycle.
Therefore, leaving another construction for later time,
I will use the autonomous NN generated by $F$ for the
next stage in the to construct an integrator.
\section*{10.3 Non-autonomous NNs having burst input}
We assume that pulses input from a burst neuron resets
initial states, but what happens when it finished its
job and becomes silent? The synaptic connection for
the burst neuron always exists. This situation is
totally different from the case where no input exists.
If the connection is excitatory, then a prolonged input
of the resting potential produces inhibitory effects
and takes the new state back to the original attractor
and even further activates another attractor in the
opposite direction. Therefore, there must exist some
other input that neutralizes the inverse effects during
the time when the burst neuron is silent. Fortunately,
in a neural integrator for saccadic eye movements,
there exist two kinds of synaptic connections in each
integrator neuron for burst input, one inhibitory and
the other excitatory. When burst neurons connected two
these synapses are both silent, the effects of resting
potentials of both neurons cancel each other out. This
situation is the same as the case where no input
exists, that is, an autonomous network In short, eye
position is kept steady not only by attractiveness of a
tonic firing and but also by pairs of excitatory and
inhibitory connections for burst neurons.\\
Now let the vector sequences input from burst
neurons to the inhibitory synapses of three neurons be
$b$, and the vector sequence input from burst neurons
to the inhibitory synapses of three neurons be $c$.
Both $b$ and $c$ are indexed as
$$
\begin{array}{lllllll}
1,& 4,& 7,& 10,& 13,& 16,&...\\
2,& 5,& 8,& 11,& 14,& 17,&...\\
3,& 6,& 9,& 12,& 15,& 18,&....,
\end{array}
$$
so that $b_1$ is the state input to the first neuron at
time 1, $b_8$ is the state input to the second neuron
at time 3, etc.\\
To accept burst input from these sequences, we add
$\mathbf{Q}^{24}$. The weights of the synaptic
efficacies of the input connections relative to the
efficacies between the integrator neurons depend on the
dimension and burst input and need some feedback from
the next stage. In the present case, I make the
relative weights equal to each other.\\
Then, by modifying the transformation $F$ defined
by (10.2.2) and (10.2.4), we obtain the function $G:
\mathbf{Q}^{36} \to \mathbf{Q}^{12}$ defined by
$$
G: (v_1,....,v_{36}) \mapsto
(w_1,....,w_{12}),
w_i = v_{i+3}\; \mbox{for $i = 1, ..., 9$}.
$$
$w_{i}$ for $i = 10, 11, 12$ are determined by $g_{i}$,
where $g_{i} = p_i \cdot \neg (p_i G)$, defined by
$$
g_{10} = p_{10} \cdot (
\neg p_1 \cdot p_4 \cdot p_7 \cdot S_5
\vee p_4 \cdot p_7 \cdot S_6
\vee \neg p_1 \cdot S_7
\vee S_8,
\eqno (10.3.1)
$$
where $S_i$ is the disjunction of all conjunctions of
$i$ functions selected from
$$
\{ \neg p_2, \neg p_3, \neg p_5, \neg p_6,
\neg p_8, \neg p_9, \neg p_{11}, \neg p_{12},
p_{13}, p_{16}, p_19, p_{22}, \neg p_{25},
\neg p_{28}, \neg p_{31}, \neg p_{34}\} .
$$
$G$ is symmetric in the sense $G \tau = \tau G$ for
permutations
\begin{eqnarray*}
\tau &=&
(1,2)(4,5)(7,8)(10,11)(13,14)(16,17)(19,20)(22,23)(25,2
6)(28,29)(31,32)(34,35)\;\\
&\mbox{and}&\\
\tau &=&
(1,3)(4,6)(7,9)(10,12)(13,15)(16,18)(19,21)(22,24)(25,2
7)(28,30)(31,33)(34,36).
\end{eqnarray*}
Therefore, $g_{11}$ and $g_{12}$ are immediately
obtained from $g_{10}$. $G$ is also self-dual, so that
$\neg p_i \cdot F_i = \bar{\neg}g_i$.\\
The non-autonomous dynamical system $\varphi :
\mathbf{Q}^{12} \times \mathbf{Z}_+ \to
\mathbf{Q}^{12}$ generated by $G$ and the input
sequence $b$ and $c$ is defined by
\begin{eqnarray*}
\varphi (v, 0) &=& v,\\
\varphi (v, t) &=& G( \varphi (v, t-1),
b_{1+3(t-1)},...,b_{12+3(t-1)}, c_{1+3(t-1)},...,
c_{12+3(t-1)}).
\end{eqnarray*}
\section*{10.4 Determination of burst patterns}
Determination of burst patterns was unexpectedly hard.
I tried a lot of failed patterns before I hit the
jackpot.\\
First I tried a single pulse being input to three
neurons with the same timing. In this case, (10.3.1)
was different in that more weight should be put on the
burst input and the equation (10.3.1) should be more
complex. For example, in one of several variants of
(10.3.1), in order that the tonic firing patterns were
changed by a pattern of inhibitory burst neurons, from
1111 to 1110, 1110 to 1010, 1010 to 1000, 1000 to 0000,
the pattern 1110 was not only changed to 1010, but also
further changed to 1000.
Then, I tried some examples where one neuron
receives a burst pulse after another neuron, so that,
for example,
$$
\begin{array}{lllll}
1111& & & & 011101\\
1111& \mapsto & ...& \mapsto& 101110\\
1111& & & & 110111.
\end{array}
$$
Also, I changed the burst pattern from just one pulse
000100000.. to 0001010000.. and the like.
Things improved. Still, the best example I obtained
failed in one case of timing between the burst pattern
and one particular tonic sequence.\\
We must consider at least two factors.
1. A desired change in each tonic pattern must occur
at specific time points of the pattern. For example,
for the change from 1110 to 1010, the change must occur
at the second time point. If this time point is missed,
then another one period of 4 time points is required
for the change. In the present model temporal summation
is over 4 time points. Therefore, there is only one
chance, if the change is made by one pulse of burst
input. On the other hand, from the pattern 1111 to the
pattern 1110, there are 4 chances in one period. For
one kind of change there is too few chances and for
another kind of change too many chances.\\
Attractiveness of these tonic patterns during the
time when both inhibitory and excitatory neurons are
silent is made by cooperation by the three neurons.
This cooperative nature is alive during the transient
period of change from one pattern to another by burst
input. Therefore, a change in one neuron influences
another neuron. In particular, if two neurons are
changed from one pattern to another pattern, the third
neuron will be changed without any pulse of the burst
neuron input thereto.\\
My first trials where a single pulse is simultaneously
input to the three neurons have no problem for the
above factor 2, but were severely affected by the above
factor 1. My attempts to diversify the phases of
patterns and increasing the number of burst pulses
relieved the factor 1 but brought some effects of the
factor 2. Therefore, a neuron receives burst pulses
later than another should receive fewer burst pulses.\\
After several trials in this way, I finally got an
example where one burst pattern can change each tonic
pattern to another tonic pattern one step lower, when
starting from
$$
\begin{array}{l}
111111111...\\
111111111...\\
111111111...
\end{array}
$$
The burst pattern is:
$$
\begin{array}{l}
101010000000...\\
000010100000...\\
000000000000...
\end{array}
$$
Here, the first line is the burst input to neuron
1, the second is the burst input to neuron 2, and the
third is the burst input to neuron 3. The dynamical
system is self-dual, so that starting from
$$
\begin{array}{l}
000000000\\
000000000\\
000000000,
\end{array}
$$
with excitatory burst input of the same pulse pattern,
the firing of the three neuron reaches
$$
\begin{array}{l}
111111111...
111111111...
111111111...
\end{array}
$$
through 4 changes, each change occurring whenever the
pulse pattern is input.
I have not checked all cases where excitatory
burst input is followed by inhibitory burst input and
vice versa. But it seems the push-pull property is also
satisfied.
\section*{10.5 Circular threshold constructions}
Now we address the pending problem that the function
$H$ in the transformation $F$ in Section 10.2 is not a
threshold function. It seems impossible to modify the
function $f_i$ to a threshold function such that the
modified F has tonic attractors of firing rates $i/4, i
= 0,1,2,3,4$. Therefore, we try to construct a
totally new network.
First, let us consider only the following types of
tonic sequences of four neurons.
$$
\begin{array}{lllll}
00000...& 100010...&01010...& 011101...& 11111...\\
00000...& 010001...&10101...& 101110...&11111...\\
00000...&001000...&01010...&110111...&11111...\\
00000...& 000100...&10101...& 111010...& 11111...
\end{array}
$$
Then, we can distinguish these tonic sequences of
firing rates $i/4$ from each other in any of their
synchronic cross sections.\\
For example, the first and third cross sections are
respectively
$$
\begin{array}{lllll}
0& 1& 0& 0& 1\\
0& 0& 1& 1& 1\\
0& 0& 0& 1& 1\\
0& 0& 1& 1& 1
\end{array}
$$
and
$$
\begin{array}{lllll}
0& 0& 0& 1& 1\\
0& 0& 1& 1& 1\\
0& 1& 0& 0& 1\\
0& 0& 1& 1& 1
\end{array}
$$
That is, these tonic sequences can be distinguished by
their cross sections, which are elements in
$\mathbf{Q}^4$. We call the firing rate in each cross
section a synchronic firing rate in contrast to a
diachronic firing rate, which is the normal definition
of firing rate. In the present case, each synchronic
firing rate of a tonic sequence is equal to its
diachronic firing rate. However the minimum distance
between the cross sections of two different sequences
is 1. Therefore, in order to separate the cross
sections at least by distance 3 and make each tonic
sequence an attractor in a dynamical system, we
consider the state space of $\mathbf{Q}^{12}$. The
tonic sequences now become
$$
\begin{array}{lllll}
00000...& 100010...&01010...& 011101...&11111...\\
00000...& 010001...&10101...& 101110...&11111...\\
00000...& 001000...&01010...& 110111...&11111...\\
00000...& 000100...&10101...& 111011...&11111...\\
00000...& 100010...&01010...& 011101...&11111...\\
00000...& 010001...&10101...& 101110...&11111...\\
00000...& 001000...&01010...& 110111...&11111...\\
00000...& 000100...&10101...& 111011...&11111...\\
00000...& 100010...&01010...& 011101...&11111...\\
00000...& 010001...&10101...& 101110...&11111...\\
00000...& 001000...&01010...& 110111...&11111...\\
00000...& 000100...&10101...& 111011...&11111...
\end{array}
$$
The first and the last sequences are constant-term
sequences in $\mathbf{Q}^{12}$. The second and fourth
are cyclic sequences of period 4, and the third is the
cyclic sequence of period 2. Also the component
sequence is the right shift of the immediately above
component sequence.
We now try to construct a first-order NN of
dimension 12, in which the above 5 tonic cyclic
sequences are attractors. After several attempts, I
obtained the following self-dual circular threshold
transformation $F$ of $\mathbf{Q}^{12}$ defined by $F =
\langle f_1\rangle $,
$$
f_1 = p_1 \cdot S_2\{ \neg p_4, \neg p_8, \neg
p_{12}\} .
\eqno (10.5.1)
$$
This threshold translation is described in Examle
7.2.5 of Chapter 7 for general dimension. In fact,
according to Chapter 7.2, this NN has only 6 cycles
and all of them are strong attractors.
Five of them correpond to those illustrated above, and
the other correspond to the non-tonic cyclic sequence
of period 4,
$$
\begin{array}{l}
11001100...\\
01100110...\\
00110011...\\
10011001...\\
11001100...\\
01100110...\\
00110011...\\
10011001...\\
11001100...\\
01100110...\\
00110011...\\
10011001...
\end{array}
$$
The next problem is how to resolve this unwelcome
non-tonic attractor. However, it seems again
impossible to modify the circular threshold
transformation to another having only the five tonic
attractors. For non-threshold constructions, see
Section 10.8. we use this autonomous NN to construct a
neural integrator. An advantage is that we can easily
generalize the dimension of this transformation.
However, there will be more and more non-tonic
attractors as the dimension becomes higher.\\
A non-autonomous network having burst input for an
integrator can now be defined by the mapping $G$:
$\mathbf{Q}^{36} \to \mathbf{Q}^{12}$, $G = \langle
g_1\rangle $,
$$
g_1 = p_1 \cdot (S_2\{\neg p_4, \neg p_8,
\neg p_{12}\} \cdot S_1\{ p_{13}, \neg
p_{25}\} \vee S_2\{ p_{13}, \neg p_{25}\} ),
\eqno (10.5.2)
$$
where the indices 13 to 24 respectively represent the
inhibitory burst input to neuron 1 to 12, and the
indices 25 to 36 respectively represent the excitatory
burst input to neuron 1 to 12. See Chapter 9.3 for the
notation $\langle g_1\rangle $ (Modify the definition
for one input 13 to 24 into two inputs 13 to 24 and 25
to 36).
\section*{10.6 Functions of burst generators}
The functions of a generator of burst sequences to be
input to a neural integrator defined by (10.5.2) should
be very sophisticated and complex.
First, the generator supplies pulses so that
non-tonic attractors should be avoided. Moreover, the
generator supplies pulses with exact timing. For
example, in order to reduce a firing rate from 3/4 to
1/2, at a time when the state (cross section) is
$$
111011101110,
\eqno (10.6.1)
$$
and the excitatory burst generator is silent, the
inhibitory burst generator supplies a pulse to at least
two of neurons 3, 7, 11. If pulses are input to
neurons 3, 7, and 11, then the next state is
$$
0101001010111.
$$
By the attractiveness, the next state will be a tonic
cross section of firing rate 1/2, if no more pulses are
supplied. Therefore, a tonic sequence of firing rate
1/2 is obtained. However, if the inhibitory burst
generator supplies pulses to neurons 1 and 5 at the
state (10.6.1), then there is no change in the firing
rate in the next cross section, just a circular shift.
If the inhibitory burst pulses are supplied to neurons
2 and 6, then the next cross section is
$$
001100110111.
$$
By the attractiveness, the next state will be a
non-tonic cross section of firing rate 1/2. Therefore,
a non-tonic sequence will succeed, when both burst
generators are silent.
According to Van Gisbergen, Robinson, and Gielen
(1981), the heart of the local feedback hypothesis is
that the output $E'$ of the integrator is relayed back
to burst generators. The burst generators also receive
a command signal $E_d$ from higher centers. "Burst
cell is driven by a signal proportional to motor error,
which is the difference between where the eye is ($E'$)
and where it should be ($E_d$)" (p. 419).\\
For our integrator network defined by (10.5.2),
the matter is not only of firing rate but also of
precise timing. Therefore, designing a neural circuit
of the burst generators and the local feedback system
is another combinatorial problem to be studied in the
next stage.
\section*{10.7 Generator-integrator feedback system}
Now, we try to construct a burst generator and a local
feedback system for the burst generator and integrator.
Van Gisbergen, Robinson, and Gielen (1981) contains a
proposed model, which is not a neural circuit in a
strict sense but rather a conceptual block diagram for
the feedback system. The proposed model simulates
experimental data in terms of firing rate. But what is
a neural circuit that receives two signals and outputs
a signal having the firing rate of the difference
between the firing rates of two input signals. To
construct such a circuit will require resolving the
same kind of difficulties as the generator-integrator
feedback system does, such as timing between two
signals and limitations of threshold functions. To
describe the generator-integrator feedback system in
terms of the "difference" of two signals is as
tautological as to say that an integrator gives a
signal whose firing rate is the sum of the firing rates
of two signals. In our following model, it is the
burst generator itself that produces a kind of
difference as burst signals.\\
Here our purpose is not to simulate a real nervous
system for saccadic eye movements. Instead, we are
concerned with an idealized abstract system in contrast
to statistical analysis and parameter physics that
simulate experimental data, where firing rate
represents a crude abstraction of reality. Therefore,
we try to construct neural circuits of a feedback
system, as a basis for a concrete system, in terms of
pulse signals and Boolean functions in place of firing
rates. Still, this approach deals with intrinsic
properties of neural circuits, such as timing and
threshold functions. We do not take into account the
time required for transmission of signals between the
burst generator and the integrator, but slight
modification will easily incorporate the delays.\\
When we describe a system in terms of 0-1 pulse
signals and not in terms of firing rate, we can define
the difference of two signals, that is, two sequences x
and $y$ in $\mathbf{Q}^n$ by a pair of $h$ and $c$
defined by,
$$
\begin{array}{lll}
h(t+1) &=& x(t) \cdot \neg y(t),\\
c(t+1) &=& y(t) \cdot \neg x(t).
\end{array}
\eqno (10.7.1)
$$
For example, consider the following three sequences in
$\mathbf{Q}^4$. Note that the firing rate of the first
is 3/4. The second and the third sequences differ only
in their phases, and their firing rate is 1/4.
$$
\begin{array}{lll}
101110111011& 100010001000& 010001000100\\
110111011101& 010001000100& 001000100010\\
111011101110& 001000100010& 000100010001\\
011101110111& 000100010001& 100010001000
\end{array}
$$
For the first two sequences, $h$ and $c$ are
respectively
$$
\begin{array}{ll}
001100110011& 000000000000\\
100110011001& 000000000000\\
110011001100& 000000000000\\
011001100110& 000000000000
\end{array}
$$
Therefore, $h$ is a sequence having a firing rate 1/2
that is the difference of the firing rates of the first
and the second. However, for the first and the third
sequences, $h$ and $c$ are respectively
$$
\begin{array}{ll}
101110111011& 010001000100\\
110111011101& 001000100010\\
111011101110& 000100010001\\
011101110111& 100010001000
\end{array}
$$
Here, the firing rate of $h$ is 3/4 and the firing rate
of $c$ is 1/4. Therefore, we have not obtained a
sequence of firing rate 1/2. Therefore, we have faced
another timing problem to produce a signal having the
difference firing rate of two signals. In order to
avoid a complex timing problem, it is better not to
seek such a signal but to use the pair $(h,c)$ for the
difference of two signals.
Now assume that the above sequence $x$ is an
integrator signal and y is a command signal. Further
assume that h is the inhibitory burst signal, and c is
the excitatory burst signal. Then we can construct the
following feedback system.
{\begin{figure}
{\centering \hspace{1cm}\includegraphics{fback.jpg}\hspace{1cm}
\par}
\caption{\label{Fig. 1}}
\end{figure}\par}
where, besides (10.7.1),
$$
\begin{array}{lll}
x_{t+1} &=& G(x(t), h(t), c(t)),\quad G =
\langle g_1\rangle ,\\
g_1 &=& p_1 \cdot (\neg p_4 \cdot (p_5 \vee
p_9) \vee p_5 \cdot \neg p_9).
\end{array}
\eqno (10.7.2)
$$
(10.7.2) is a simplified function of (10.5.2) obtained
by removing attractiveness. If we consider the state
space $\mathbf{Q}^{12}$ in place of $\mathbf{Q}^4$, we
can use (10.5.2) as it is.
Two problems should be addressed now. First, for
an integrator to change the firing rate 3 /4 of $x$ to
1/4, it must receives $h(t)$ and $c(t)$ that may,
depending on the timing between $x$ and the command
signal as described above, be simultaneously active at
some time $t$ from the burst generator. This does not
pose any serious mathematical problems. It also agrees
with reality, since Van Gisbergen, Robinson, and Gielen
(1981, p.418) says, "... during any saccade, ... burst
neurons on both sides of the brain stem often are
active simultaneously."\\
The second problem was already discussed in the
last section. The function $G$ is very sensitive to
the timing between the inputs $x$ and $(h, c)$. In
fact, the above feedback system defined by (10.7.1) and
(10.7.2) does not work. There are two time lags
between the measuring of the difference and the
realization of changing the synchronic firing rate of
$x$. At the same time period, if all components of $h$
and $c$ are all 0, then the cross section of $x$ will
round-shifted by 2. Therefore, (10.7.1) should be
modified to
$$
\begin{array}{lll}
h(t+1) &=& \rho^2 (x(t) \cdot \neg y(t)),\\
c(t+1) &=& \rho^ 2 (y(t) \cdot \neg x(t)),
\end{array}
\eqno (10.7.3)
$$
where $\rho$ is the cyclic permutation (1,2,3,4).
Clearly $h$ and $c$ are threshold functions. Computer
simulation expects that this feedback system will work.
For example, for the first and third sequences in the
beginning of this section respectively as $x(t)$ and
$y(t)$, we obtain
$$
x(t): \begin{array}{l}
10000100\\
11100010\\
11010001\\
01001000
\end{array}
$$
$$
y(t): \begin{array}{l}
01000100\\
00100010\\
00010001\\
10001000
\end{array}
$$
$$
h(t): \begin{array}{l}
01100000\\
00100000\\
01000000\\
01100000
\end{array}
$$
$$
c(t): \begin{array}{l}
00000000\\
01000000\\
00100000\\
00000000
\end{array}
$$
Next, we prove a theorem that validates the
generator-integrator feedback system constructed in
above. The state spaces are now $\mathbf{Q}^m$, and
$\rho$ is the cyclic permutation $(1,2,...,m)$. Let
$y$ be a tonic sequence in $\mathbf{Q}^m$. If $y(t+1)
= \rho y(t)$ for every $t$, we call $x$ a
\emph{circular tonic sequence}.
$G$ is a circular function: $\mathbf{Q}^{3m} \to
\mathbf{Q}^m$, that is, $\rho G(u,v,w) = G(\rho u,
\rho v, \rho w)$ for $u, v, w \in \mathbf{Q}^m$,
defined by $g_1 = p_1 \cdot \neg (p_1 G)$, that is, $G
= \langle g_1\rangle$,
$$
g_1 = p_1 \cdot
( \neg p_m \cdot (p_{m+1} \vee \neg p_{2m+1})
\vee p_{m+1}\cdot \neg p_{2m+1}).
\eqno (10.7.4)
$$
The command signal $y(t)$ is a given circular tonic
sequence in $\mathbf{Q}^m$, and the integrator signal
$x(t)$ is a sequence in $\mathbf{Q}^m$ defined by
$$
x_{t+1} = G(x(t), h(t), c(t)).
\eqno (10.7.5)
$$
The burst signals $h(t)$ and $c(t)$ are sequences in
$\mathbf{Q}^m$ defined by
$$
\begin{array}{lll}
h(t+1) &=& \rho ^2 (x(t) \cdot \neg y(t)),\\
c(t+1) &=& \rho ^2 (y(t) \cdot \neg x(t))
\end{array}
\eqno (10.7.6)
$$
In the following, $M = \{ 1,...,m \}$ is regarded as
the residue class ring with m as the zero element. For
example, $1 - 2 = m - 1$. Further, $o = (00,..,0) \in
\mathbf{Q}^m$.\\
\textbf{Theorem 10.7.1} Let $h(0) = c(0) = o$. Then,
$x(2) = y(2)$, and $x(3) = y(3)$.
\begin{proof} Let $h(1)_k = 1$ and $c(1)_k = 0$.
Clearly $x(2)_k = 0$. On the other hand, we have
$x(0)_{k-2} = 1$ and $y(0)_{k-2} = 0$, since $h(1)_{k}
= 1$. Therefore, $y(2)_k = y(0)_{k-2} = 0$.
Therefore, $x(2)_k = y(2)_k$. By self-duality, if
$h(1)_k = 0$ and $c(1)_k = 1$, then $x(2)_k =
y(2)_k$.
Next, let $h(1)_k = c(1)_k = 0$. If $x(1)_k = 1$
and $x(1)_{k-1} = 0$ then $x(2)_k = 0$. If $x(1)_k =
1$ and $x(1)_{k-1} = 1$, then $x(2)_k = 1$. Therefore,
by self-duality, $x(2)_k = x(1)_{k-1}$. On the other
hand, $x(0)_{k-2} = y_{k-2}$, since $h(1)_k = c(1)_k =
0$. Therefore, $x(1)_{k-1} = x(0)_{k-2}$, since $h(0)
= c(0) = o$. Therefore, $x(2)_k = x(0)_{k-2}$.
Clearly $y(2)_k = y(0)_{k-2}$. Therefore, $x(2)_k =
y(2)_k$. Therefore, $x(2) = y(2)$.
Further, $x(1) = \rho x(0)$ and $y(1) = \rho
y(0)$. Therefore,
\begin{eqnarray*}
h(2) &=& \rho^2 (x(1) \cdot \neg y(1)) = \rho^3
(x(0) \cdot \neg y(0)) = \rho h(1).\\
c(2) &=& \rho^ 2 (y(1) \cdot \neg x(1)) = \rho^3
(y(0) \cdot \neg x(0)) = \rho c(1) .
\end{eqnarray*}
Let $h(2)_k = 1$. Then $h(1)_{k-1} = 1$, so that
$x(2)_{k-1} = 0$. Therefore, if $x(2)_k = 1$, then
$x(3)_k = 0 = x(2)_{k-1}$. If $x(2)_k = 0$, then
$x(3)_k = 0 = x(2)_{k-1}$. If $c(2)_k = 1$, then
$x(3)_k = 1 = x(2)_{k-1}$ by self-duality. Next, let
$h(2)_k = c(2)_k = 0$. Then clearly
$x(3)_k = x(2)_{k-1}$. Therefore,
$x(3) = \rho x(2) = \rho y(2) = y(3)$.
\end{proof}
The above Theorem 10.7.1 shows that if $h(0) = c(0) =
o$, then, $x(2) = y(2)$ and $x(3) = y(3)$, so that
$h(3) = c(3) = o$, so that $x(4) = y(4),...$
Therefore, if $h(0) = c(0) = o$, then $x(t) = y(t)$ for
every $t \geq 2$, and $h(t) = c(t) = o$ for every $t
\geq 3$. Therefore, if $y$ changes to another
circular tonic sequence at some time $t' \geq 3$, then
$x(t) = y(t)$ for every $t \geq t'+2$, since $h(t') =
c(t') = o$. It is well known that omni-pause neurons
make burst generators silent. The input from the omni
pause neurons to the burst generator is one feature
that is absent from the present generator-integrator
feedback model.
\end{document}