\documentclass{amsart}
\usepackage{amssymb,amsfonts,latexsym,amsmath}

\begin{document}

\title{Chapter 9 Attractors in Non-autonomous Neural Networks}
\begin{abstract} Non-autonomous primitive dynamical neural
networks
(PDNNs) that incorporate a spontaneous firing rate of $1/2$ per
unit time are constructed.  The concept of attractors is
redefined so that they should be invariant under shifts in
arrival of the periodic input sequence.  If asymptotic
properties for an attractor depend on both initial states and
input, then the attractor is called bi-dependent.  Then we
show how non-autonomous PDNNs described in Chapter 6 are
modified by input in corresponding non-autonomous PDNNs, so
that a non-attractive limit cycle becomes a bi-dependent
attractive cyclic orbit, a non-unique attractor becomes
unique, and an attractor consisting of more than one cycles
becomes an attractive cyclic orbit.    
\end{abstract}
\maketitle


\section*{9.1  Introduction} 
In the last two chapters, we introduced McCulloch-Pitts networks
of self-oscillatory neurons with
spontaneous firing rates $1/2$ and $1/3$ respectively, and showed
the existence of attractors.  In
the first
order neural networks described in Chapter 6, $\mathbf{N} =
\{1,2,...,n\}$
represents neurons, and the state
space
$\{ -1,1\} ^{\mathbf{N}}$ of the PDNN (primitive dynamical neural
network)
 is a finite metric space with the integer-valued Hamming
distance
$d_{H}$.  The PDNN of spontaneous firing rate $1/2$ is a finite
state dynamical system (FSDS)
on the state space $\{ -1,1\} ^{\mathbf{N}}$ generated by the
threshold
transformation $F$ of $\{ -1,1\} ^{\mathbf{N}}$
defined by
$$
        Fx = \mathrm{Sgn}(Ex), \quad x(t+1) = F(x(t)).
$$

    This model is autonomous, that is, stable periodic firing
patterns that are represented by
attractors are
completely determined by the efficacy matrices of synaptic
connections and the initial states of the
neurons
at time $t = 0$.  However,  the dynamics of any biological system
depends on information that
changes at
every time and that is input from the outside of the system, from
neurons of other nervous systems
and/or
from external stimulus.  Further, in autonomous networks, if a
minimal attractor consists of more
than one
cycle, then there are some fluidity of shifting from one pattern
to another caused by noise, even
with a
change in firing rate in some cases.  This problem may be solved
only in a non-autonomous model
with
input from outside the network.

     Therefore, we want here to construct a PDNN in which $x(t+1)
\in  X = \{ -1,1\} ^{\mathbf{N}}$
depends on
$x(t)$ and the external input $r(t)$.  Let us assume that $r(t)
\in  Y = \{-1,1\} ^{\mathbf{N}\prime }$,
where
$\mathbf{N}\prime  = \{ n+1,n+2,...,2n\}$ is the set of external
input
elements, and that $r(t)$ is a periodic
sequence
defined as $r(t) = V_{t\%k}$ by some $k$-sequence $V =
(V_{0},...,V_{k-1})$ in $Y$. 

    The elemens of $\mathbf{N}\prime $ represent receptor neurons
or
interneurons.  Let the synaptic
connections
between $\mathbf{N}$ and $\mathbf{N}\prime $ be represented by an
efficacy matrix
$C$.  Then we define 
$$
        F(x,y) = \mathrm{Sgn}(Ex+Cy),
\eqno    (9.1.1) 
$$
$$
        x(t+1) = F(x(t),r(t)).
\eqno    (9.1.2) 
$$
In this case, for each given input $V$, a dynamical system
$\varphi _{V}$ on $X=\{ -1,1\} ^{n}$
generated by $F$ and $V$ is defined as $\varphi _{V}: X\times
\mathbf{Z}_{+}\rightarrow  X, \varphi
_{V}(x, t) = x(t)$, and $x(0) = x$.  As in Chapter 6, if $x(t) =
-x(t-1)$
for every $t$, then the orbit $x(0),x(1),...$ is called
$neutral$. 

    In the following, both $\{ -1,1\} ^{\mathbf{N}}$ and $\{
-1,1\}
^{\mathbf{N}\prime }$ are identified with $\{
-1,1\}
^{n}$. The input $k$-sequence $V$ may be  expressed as $V = 
(i_{1} w^{1}, i_{2} w^{2},  .  . , i_{j} w^{j})$ such that
$i_{1}+...+i_{j} = k$ and
$w^{h} \in  \{ -1,1\} ^{n}$.  For example, for $n =
2$, $V = ((1,1), (1,1), (-1,1), (-1, -1), (-1,-1)) = (2 (1,1), 1
(-1,1), 2
(-1,-1) )$.

    We are concerned with properties that are invariant under
shifts in arrival of the input $k$-sequence. 
Therefore, we apply the definition of attractors given in the
next section, which should also be
referred to for the following discussion.  

       There are two extreme cases in the dynamical system
$\varphi _{V}$.  The first case is that
$\mathrm{min}_{x\in \{-1,1\}^{\mathbf{N}}}|E_{i}x|$ is
sufficiently greater than $\mathrm{max}_{j}|C_{ij}|$ for every
$i$ so that $x(t+1) =
\mathrm{Sgn}(Ex(t))$.  In this case we have an
autonomous PDNN described in Chapter 6.  The second case is that
$\mathrm{min}_j|C_{i}V_{j}|$ is sufficiently
greater than $\mathrm{max}_{j}|E_{ij}|$ for every $i$ so
that $x(t+1) = \mathrm{Sgn}(Cr(t))$. 
In this case, the sequence
$(\mathrm{Sgn}(CV_{0}),...,\mathrm{Sgn}(CV_{k-1}))$ is clearly
an attractor.  For example, if $V$ is a $k$-sequence of period
$p$, and if $C = cI$, where
$I$ is the identity matrix, such that $c$ is sufficiently greater
than $|E_{i}|$ for every $i$, then $x(t+1) = r(t)$, so that $V$
is a unique attractor. 
However, we are not interested in such extreme cases and
concerned with only \emph{bi-dependent} attractors.

    Section 9.2 formally defines various concepts concerning
autonomous and non-autonomous PDNNs and
particularly attractors.  Section 9.3 describes Boolean
representations of non-autonomous PDNNs. 
The existence of bi-dependent attractive loops is shown in
Section 9.4.  The existence of bi-dependent, attractive, cyclic
orbits in
skew-circular PDNNs is shown in Section 9.6.   In the final
Section 9.7,
we show how a non-unique attractor in a cicular autonomous PDNN
becomes unique but bi-dependent in a corresponding non-autonomous
PDNN.

\section*{9.2  Definition of attractors}
 A sequence $V$ of $X$ is called \emph{periodic} if there exists
some
 $k$ such that $V_{i} = V_{i\%k}$ for every $i = 0,1,...$.
The minimal such $k$ is called the \emph{period}, and $V$ is
denoted
by the $k$-sequence $(V_{0},V_{1},..,V_{k-1})$; in particular, if
$V_{i} \ne  V_{j}$ for every $i \ne  j$, $0 \leq i,j \leq  k-1$
then $V$ is called \emph{cyclic} (See Chapter 6.1).

    Let $X$ and $Y$ be finite metric spaces.  Let $F$ be a
mapping: $X\times Y \rightarrow 
X$ and $V = (V_{0},V_{1},..,V_{k-1})$ be a $k$-sequence of $Y$. 
Then $F$ defines a mapping $\varphi _{V}: X\times
\mathbf{Z}_{+}\rightarrow  X$ by
\begin{eqnarray*}
        \varphi _{V}(x,0) &=& x,\\
        \varphi _{V}(x,t) &=& F(\varphi _{V}(x,t-1),V_{(t-1)\%k})
\; \mbox{for}\;  t \geq  1,
\end{eqnarray*}
Then $\varphi _{V}$ is called a \emph{finite-state dynamical
system} (FSDS) on $X$
\emph{generated by $F$ and input $V$}.  If $x$ is a point of $X$
and $W = \sigma
^{j}V$ for some $j$, then the sequence
$$
    \mathrm{Orb}_{F,V}x = (\varphi _{V}(x,0),\varphi
_{V}(x,1),\varphi _{V}(x,2),...)
$$
is called an orbit starting at the point $x$.  If $S$ is a subset
of $X$, then $\mathrm{Orb}_{F,V}S$
denotes the set of all orbits starting at some $x \in  S$. 
Further, let $\mathrm{PO}(F,V)$ denote
the set of all periodic orbits in the FSDS $\varphi _{V}$.  A
periodic orbit
$W =(W_{0},W_{1},..,W_{m-1}) \in 
\mathrm{PO}(F,V)$ is called a \emph{limit orbit} of $x$ and
denoted by $\Omega _{F,V}x$, if there
exists some $j \in  \mathbf{Z}_{+}$ such that $j\equiv 0$ mod $k$
and $\varphi _{V}(x,t) = W_{(t- j)\%m}$ for every $t \geq  j$. 
For a subset $S$ of $X$,
$\Omega _{F,V}S$ is the union $\bigcup_{x\in
S}\Omega _{F,V}x$.  Let $\sigma $ denote the round shift defined
by $\sigma W = (W_{1},...,W_{m-1},W_{0})$ for any
\emph{m}-sequence $W=(W_{0},W_{1},..,W_{m-1})$
of any length $m$.
Further, if
$\Phi  = \{ U,..,W\}$, then let $\sigma \Phi  = \{ \sigma
U,...,\sigma W\}$.\\ 

\textbf{Definition 9.2.1}  A subset $\Phi $ of $\mathrm{PO}(F,V)$
is called \emph{attractive} or
an
\emph{attractor} in the FSDS generated by $F$ and $V$, if there
exists some neighborhood
$U_{\epsilon
}(\mathrm{Im}\Phi )$ such that
\begin{enumerate}
   \item $F(U_{\epsilon }(\mathrm{Im}\Phi )\times \mathbf{V})
\subseteq  U_{\epsilon
}\mathrm{Im}\Phi $.
   \item $\Omega _{F,\sigma ^{j}V}(U_{\epsilon }(\mathrm{Im}\Phi
)) = \sigma ^{j}\Phi $ for
any $j$.
\end{enumerate}


In particular, if $\Phi $ consists of one periodic orbit, the
periodic orbit is called an attractor.  The
$basin$
for an attractor $\Phi $ is the set of all points $x$ such that
$\Omega _{F,\sigma ^{j}V}x \in
\sigma
^{j}\Phi $ for any $j$.  If the basin for an attractor is not the
entire space $X$, then the attractor is called
\emph{dependent on initial states}.  If an attractor in the FSDS
generated by $F$ and $V$ is not an attractor in the FSDS
generated by $F$ and some input $V\prime $,
then the attractor is called $dependent on input$.  If an
attractor is dependent on both initial states
and input, then the attractor is called $bi-dependent$.\\

\section*{9.3  Boolean and extended representations}
Let $l$ and $o$ be the elements of $\mathbf{Q}^{n}$ such that
$l_{i}= 1$ and $o_{i} = 0$ or
every $i$. 
The mapping $F: \{ -1,1\} ^{n}\times \{-1,1\} ^{n} \rightarrow 
\{-1,1\} ^{n}$ defined by (9.1.1)
can be
represented by an equivalent mapping $G: \mathbf{Q}^{n}\times
\mathbf{Q}^{n} \rightarrow 
\mathbf{Q}^{n}$ defined by
$$
        G =\mathrm{Bool}\circ F\circ \mathrm{Sgn}.
$$
Specifically,
$$
        G(x,y) = \mathrm{Bool}(2(Ex+Cy)-(E+C)l),
\eqno  (9.3.1) 
$$
since $\mathrm{Sgn}x = 2x-l$ if $x \in  \mathbf{Q}^{n}$. 
Further, (9.1.2)
is replaced by 
$$
        x(t+1) = G(x(t)),s(t)),
\eqno   (9.3.2) 
$$
where $s = \mathrm{Bool}r$.

    Let $p_{i}$ and $q_{i}$ be respectively the projection
functions: $\mathbf{Q}^{n}\times
\mathbf{Q}^{n} \rightarrow  \mathbf{Q}$ defined by
$$
         p_{i}(x,y) = x_{i}, \quad  q_{i}(x,y) = y_{i}.
$$
Let $G: \mathbf{Q}^{n}\times \mathbf{Q}^{n} \rightarrow 
\mathbf{Q}^{n}$ and $G =
(G_{1},...,G_{n})$, where $G_{i} = p_{i}\circ G$. $G$ can be
further represented by
$$
        G = [g_{1},...,g_{n}], \; \mbox{where}\;  g_{i} =
p_{i}\cdot \neg G_{i}.
\eqno    (9.3.3) 
$$

The extended representation $G^{\#}$ of G is defined as in the
previous chapters by
$$
(G^{\# }x)_{i} =  \left\{
\begin{array}{ll}
d_{SH}(P_{\mathbf{N}\backslash \{ i\} }x,g_{i}|1)  & \mbox{if
$x_{i} =
1$,} \\
d_{SH}(P_{\mathbf{N}\backslash \{ i\} }x,(\bar{\neg}g_{i})|0)
&
\mbox{if $x_{i} = 0$}
\end{array} 
  \right.                   
$$
for $i = 1, . . , n$.  It is clear that
$$
          x_{i} \ne  (Gx)_{i} \; \mbox{if and only if
$(G^{\#}x)_{i} \leq  0$.}
$$
If $T$ is an isometry of $\mathbf{Q}^{n}$, then $T$ also
defines an isometry of $\mathbf{Q}^{n}\times \mathbf{Q}^{n}$ by
$T(x,y) = (Tx,Ty)$. 
With this definition, let $Tg$ be defined for an isometry of
$\mathbf{Q}^{n}$ and a function
$g: \mathbf{Q}^{n}\times \mathbf{Q}^{n} \rightarrow  \mathbf{Q}$
by the P\'{o}lya action
as $Tg = g\circ T^{-1}$.  Let $\rho $ be the circular
permutation $(1,2,..,n)$ of $\mathbf{N}$.  A mapping $G:
\mathbf{Q}^{n}\times \mathbf{Q}^{n}\rightarrow 
\mathbf{Q}^{n}$ is called \emph{circular} if $G\circ \rho  = \rho
\circ G$. $G$ is called \emph{skew-circular} if $G\circ \rho
n^{-} = \rho n^{-}\circ G$.  Then $G$ is circular iff $g_{i} =
\rho ^{I-1}g_{1}$ for every $i$ in (9.3.3); $G$ is denoted by
$\langle g_{1}\rangle$ in this case. $G$ is skew-circular if and
only if $g_{i} = (\rho n^{-})^{i-1}g_{1}$ for every $i$; G is 
denoted by $\langle\langle g_{1}\rangle\rangle $ in this case.

     As in previous chapters, if a group $\mathbf{G}$ acts on a
set $X$, then an equivalence
relation $\sim_{\mathbf{G}}$ on $X$ can be defined by $x
\sim_{\mathbf{G}} y$ if there is
an element $\tau \in \mathbf{G}$ such that $\tau x = y$.  Each
equivalence class with
respect to the equivalence relation $\sim_{\mathbf{G}}$, that is,
$\mathrm{Orb}_{\mathbf{G}}x$ is denoted by $[x]$.  Also,
$\mathrm{Orb}_{\mathbf{G}}x$ for a subset $S \subseteq X$ is
denoted by $[S]$.  Further,
the equivalence relation $\sim_{\mathbf{G}}$ is extended to the
set of all non-empty subsets
of $X$, that is, $A \sim_{\mathbf{G}} B$ if
$\mathrm{Orb}_{\mathbf{G}}A =
\mathrm{Orb}_{\mathbf{G}}B$.

    As in Chapter 6, it is often more convenient to consider $-F$
in place of $F$ in (9.1.1), or equivalently,
$\bar{\neg}G$ in place of $G$ in (9.3.1).  Note that $F(x,y) =
-F(-x,-y)$, that is, $G(x,y) = \bar{\neg}G(\bar{\neg}x,
\bar{\neg}y)$.  Therefore, as shown in the following diagram, an
orbit $x$ of $\varphi _{V}$ generated by $G$ and $V$ can be
obtained
from a corresponding orbit $x\prime $ of $\psi _{V\prime }$
generated by $\bar{\neg}G$ and $V\prime $, where $V\prime _{i} =
\bar{\neg}V_{i}$ if $i$  is even, $V\prime _{i} =V_{I}$ 
if $i$ is odd, and $x\prime (0) = \bar{\neg}x(0)$.  Specifically,
$x(t) = \bar{\neg}x\prime (t)$ if $t$ is
even, and $x(t) = x\prime (t)$ if $t$ is odd.
$$
\begin{array}{lll}
             \varphi _{V}   &\qquad & \psi _{V\prime } \\ 
     x(0),    &\qquad &  x\prime (0) = \bar{\neg}x(0), \\     
     V_{0}.  &\qquad &  V\prime _{0} =  \bar{\neg}V_{0}.\\
    x(1) = G(x(0)),v_{0}), &\qquad &x\prime (1) =
\bar{\neg}G(x\prime (0),v\prime _{0})\\
                &\qquad &      = \bar{\neg}G(\bar{\neg}x(0),
\bar{\neg}V_{0})\\
                &\qquad &     = x(1),\\
    V_{1}.         &\qquad &  V\prime _{1} = V_{1}. \\
    x(2) = G(x(1),v_{1}), &\qquad &  x\prime (2) =
\bar{\neg}G(x\prime (1),V\prime _{1})\\
          &\qquad &      = \bar{\neg}G(x(1),v_{1})\\
          &\qquad &    = \bar{\neg}x(2), \\
    V_{2}.  &\qquad &   V\prime _{2} =
\bar{\neg}V_{2}.\\
    x(3) =  G(x(2),v_{2}), &\qquad & x\prime (3) =
\bar{\neg}G(x\prime (2),v\prime _{2})\\
                     &\qquad &    = \bar{\neg}G(\bar{\neg}x(2),
\bar{\neg}V_{2})\\
                    &\qquad &      = x(3),\\
    V_{3}.             &\qquad &        V\prime _{3} = V_{3}.\\  
    .......        &\qquad &     ......... 
\end{array}
$$

\section*{9.4. Attractive loops}

Let $d(x)$ be the density of $x \in  \mathbf{Q}^{m}$, i.e. $d(x)
= |\{ i \mid  x_{i} = 1 \}|$.\\

\textbf{Example 9.4.1}  Let $E_{ij} = \epsilon  > 0$ for every $i
\ne  j$ and $C = \epsilon L$ in (9.1.1),
where $L_{ij} = 1$ for every $i,j$.  \\

\textbf{Lemma 9.4.2} In the PDNN of Example 9.4.1, the
transformation
$G$ defined by (9.3.1) is 
$$
G = \left\{
\begin{array}{ll}
    <p_{1}>, & \mbox{if}\;  0 < \epsilon  < 1/(2n-1);\\
    <p_{1}\cdot S_{1}(\neg p_{2},..,\neg p_{n},\neg
q_{1},..,\neg q_{n})>, &
       \mbox { if } 1/(2n-1) < \epsilon  <
1/(2n-3);\\
    <p_{1}\cdot S_{2}(\neg p_{2},..,\neg p_{n},\neg
q_{1},..,\neg q_{n})>, & \mbox{if}\;  1/(2n-3) < \epsilon  <
1/(2n-5);\\
     \ldots & \ldots
\end{array}
\right.
$$

\begin{proof} $G_{i}(x,y) = \mathrm{Bool}((2(d(x\cdot
y))-2n-1)\epsilon -1)$, if $x_{i} = 1$. Therefore,
$(x,y) \in  g_{i}$ if and only if $x_{i} = 1$ and $(2d(x\cdot
y)-2n-1)\epsilon -1 < 0$.  Then, $(x,y) \in 
g_{i}$ iff $(x_{i} = 1, d(x\cdot y) \leq  2n, \; \mbox{and}\; 
\epsilon  < 1/(2n-1))$, or $(x_{i} = 1, d(x\cdot
y) \leq  2n-1, \; \mbox{and}\;  \epsilon  < 1/(2n-3))$ or $(x_{i}
= 1, d(x\cdot y) \leq  2n-2, \; \mbox{and}\; 
\epsilon  < 1/(2n-5))$, or so forth.  Therefore, the results of
this lemma is obtained.
\end{proof}

Let $0 < 1/(2n-3) < \epsilon  < 1/(2n-5)$ in Example 9.4.1.  Then
by Lemma 9.4.2, $<p_{1}\cdot S_{2}(\neg p_{2},..,\neg p_{n},\neg
q_{1},..,\neg q_{n})>$.  Therefore, $(l)$ is a unique
attractive loop for the input 1-sequence $V = (l)$, since $G(x,l)
= x$ if and only if $x =
l$, and $G(U_{1}(l),l) = l$.  Similarly, $(o)$ is a unique
attractive
loop for the input 1-sequence $V = (o)$.  On the other hand,
$(l)$ is a loop for
$V = (i^{-}l)$, but if $x \in  U_{1}(l)$, then $G(x,i^{-}l) =
\bar{\neg}x$ and $G(\bar{\neg}x,i^{-}l) = x$. 
Therefore, $(l)$ is not an attractor for $V =(i^{-}l)$.  However,
the asymptotic properties of this PDNN are
not completely determined by its input.  For example, if $x \in 
U_{2}(l)$, then $G(x,l) = \bar{\neg}x$ and
$G(\bar{\neg}x,l) = x$, that is, $x$ is on a neutral orbit and
does not converge to the attractor $(l)$. 
Therefore, this attractor is bi-dependent.

    Next, let $0 < 1/(2n-5) < \epsilon  < 1/(2n-7)$.  If $V =
(l)$, then $l$ is a unique attractive loop
for $V$,
since $G(U_{2}l, l) = l$.  Similarly, $(o)$ is a unique
attractive loop for $V = (o)$, and $l$ is an attractor for $V =
(i^{-}l)$.
Further, $l$ is also an attractor for $V = (i^{-}l)$,
since $G(U_{1}l,i^{-}l) = l$. But if $x \in U_{1}l$, then
$G(x,\{i,j\}^{-}l) = -x$ and $G(-x,\{i,j\}^{-}l) =
x$ for $i \ne  j$.  Therefore, $l$ is not an
attractor for $V =(\{i,j\}^{-}l)$ such that $i \ne  j$.  In
summary, we have obtained:\\

\textbf{Theorem 9.4.3}  There exists a PDNN that has a
bi-dependent
attractive loop. \\

\section*{9.5. An attractiveness condition}  
Let $H: \mathbf{Q}^{n}\times \mathbf{Q}^{n}
\rightarrow \mathbf{Q}^{n}$ and $V = (V_{0},V_{1},..,V_{k-1})$ be
a cyclic sequence in $\mathbf{Q}^{n}$.
Then $H$ can define a transformation $H_{V}$ of
$\mathbf{Q}^{n}\times
\mathbf{V}$ by $H_{V}(x,V_{i}) = (H(x,V_{i}),V_{(i+1)\%k})$.  Let
$T$ be an isometry of $\mathbf{Q}^{n}$ such that $TV_{i} =
V_{(i+1)\%k}$ for every $i$.  Then $H_{V}(x,V_{i})
= (H(x,V_{i}),TV_{i})$.

Further, let $H$ be commutative with $T$, i.e. $H\circ T =
T\circ H$.  Then $H_{V}$ is commutative with $T$.
Let $[S]$ for a subset $S$ of $\mathbf{Q}^{n}\times
\mathbf{Q}^{n}$ denote
$\mathrm{Orb}_{\langle T\rangle}S$.  Then, as described in
Chapter 2.6, $H_{V}$ defines a transformation
$H_{V}^{\sim }$ of $[\mathbf{Q}^{n}\times \mathbf{V}]$ by
$H_{V}^{\sim }[x,y] = [H_{V}(x,y)]$. Under these conditions, the
following proposition holds.\\

\textbf{Proposition 9.5.1}  Let $W = (W_{0},W_{1},..,W_{k-1})$
be a cyclic sequence in $\mathbf{Q}^{n}$ such that $TW_{i} =
W_{(i+1)\%k}$
with the length $k$ being the same as that of $V$.  Let 
$H_{V}^{\sim }[U_{\epsilon }\mathbf{W}\times \mathbf{V}]
\subseteq 
[U_{\epsilon }\mathbf{W}\times \mathbf{V}]$ for some $\epsilon 
>0$,
and assume that $\mathrm{CY}(H_{V}^{\sim }|[U_{\epsilon
}\mathbf{W}\times
\mathbf{V}])$ consists of some loops $[Y_1,V_0]\partial ,..,
[Y_m,V_0]\partial $ for
$Y_i \in \mathbf{W}$.  Then $\Phi = \mathrm{Orb}_T\{
Y_1,..,Y_m\}$ is an attractor of
$H$.
\begin{proof} (i) For the proof of $H(U_{\epsilon
}\mathrm{Im}\Phi \times \mathbf{V})
\subseteq U_{\epsilon }\mathrm{Im}\Phi$, let $(x,V_i) \in
U_{\epsilon }\mathrm{Im}\Phi
\times \mathbf{V} = U_{\epsilon }\mathbf{W}\times \mathbf{V}$. 
Then
$[x,V_{i}] \in  [U_{\epsilon }\mathbf{W}\times \mathbf{V}]$,
so that  $H_{V}^{\sim }[x,V_{i}] \in  [U_{\epsilon
}\mathbf{W}\times
\mathbf{V}]$.  Therefore, $H(x,V_{i}) \in [U_{\epsilon
}\mathbf{W}]$.  Since $[U_{\epsilon }S] = U_{\epsilon }[S]$ for
any $S$, and
$\mathbf{W}$ is $\langle T\rangle $-invariant,we have $H(x,V_{i})
\in  U_{\epsilon
}\mathbf{W} = U_{\epsilon }\mathrm{Im}\Phi $.

(ii) For the proof of $\Omega _{H,\sigma^{j}V}
(U_{\epsilon }\mathrm{Im}\Phi ) =  \sigma ^{j}\Phi $ for any $j$,
let $x \in  U_{\epsilon }\mathrm{Im}\Phi =  U_{\epsilon
}\mathbf{W}$.  Then, in
$\mathrm{GRAPH}(H_{V}^{\sim }|[U_{\epsilon }\mathbf{W}\times
\mathbf{V}])$,
 $[x,V_{j}] \rightarrow [(H(x,V_{j}),V_{(j+1)\%k})] \rightarrow 
... \rightarrow 
[W_{u},V_{0}]\partial $ for some $u$ such that $W_u = Y_i$ for
some $i$.  Therefore,
$(x,V_{j}) \rightarrow (H(x,V_{j}),V_{(j+1)\%k}) \rightarrow  ...
\rightarrow  (W_{(u+s)\%k},V_{s} \rightarrow 
(W_{(j+u)\%k},V_{j}) \rightarrow  ...$  for some $s$. 
Therefore, $\Omega _{H,\sigma ^{j}V}x = \sigma
^{j}\mathrm{Orb}_{T}Y_{i}$. 
Therefore, $\Omega _{H,\sigma ^{j}V}(U_{\epsilon }\mathrm{Im}\Phi
) =  \sigma ^{j}\Phi $. 
\end{proof}


\section*{9.6. Attractors in skew-circular PDNNs}  


\textbf{Example 9.6.1}
$$
E =
\begin{bmatrix}
-1 & \epsilon & \cdot & \cdot & \cdot & \cdot & \epsilon \\
-\epsilon & -1 & \epsilon \cdot & \cdot & \cdot & \epsilon \\ 
-\epsilon & -\epsilon & -1&\epsilon&\cdot & \cdot &\epsilon\\ 
\cdot & \cdot & \cdot & \cdot & \cdot & \cdot & \cdot \\
\cdot & \cdot & \cdot & \cdot & \cdot & \cdot & \cdot \\
-\epsilon & \cdot & \cdot & \cdot & \cdot & -\epsilon & -1
\end{bmatrix},
\eqno (9.6.1) 
$$
where $n \geq  5$ and $1/(n-1) < \epsilon  < 1/(n-3)$, and $C =
\delta I$.\\

In Example 9.6.1, if $\delta  = 0$, then the transformation $G$
defined by (9.3.1) is 
$$
        G = \langle\langle p_{1}\cdot S_{1}\{\neg p_{2},..,\neg
p_{n}\}\rangle\rangle ,
$$
which is independent of any input $V$ and generates an autonomous
PDNN.  More specifically in this case, if $G\prime $ is the
transformation of $\mathbf{Q}^{n}$ defined by $G\prime x =
G(x,a)$ for any $a$, then $\bar{\neg}G\prime  = \langle\langle
p_{1}\cdot p_{2}\cdot \cdot \cdot p_{n})\rangle\rangle $,
so that $\mathrm{GRAPH}(\bar{\neg}G\prime )$ consists of loops
and the
unique $2n$-cycle:
$$
        W = (l,...,(\rho n^{-})^{i-1}l,...,(\rho n^{-})^{2n-1}l).
$$
Therefore, this cycle is not attractive in the autonomous FSDS
generated by $\bar{\neg}G\prime $.

    Now, we assume $(n-1)\epsilon -1 < \delta $ and
$1-(n-3)\epsilon  < \delta  < 
1-(n-5)\epsilon $.  Then 
$$
         G = \langle\langle p_{1}\cdot (S_{2}\{\neg p_{2},..,\neg
p_{n}\}\vee \neg q_{1})\rangle\rangle .
$$
Let $H = \bar{\neg}G$. Then
$$
        H = \langle\langle h_{1}\rangle\rangle , h = p_{1}\cdot
q_{1}\cdot S_{n-2}\{p_{2},..,p_{n}\}.
$$
Therefore, $x_{i} = 1$ and $H_{i}(x,y) = 0$, iff $y_{i} = 1$ and
($x = \{ 1,..,i-1\}^{-}l$ or $x = j^{-}\{1,..,i-1\}^{-}l$),
where $j \ne  i$.

    Let $a_{i} = i \% 2$ for every $i$.  Then 
$$
        S_{n-2}\{p_{2},..,p_{n}\}((\rho n^{-})^{-(i-1)}a) = 0,
$$
so that $(\rho n^{-})^{-(i-1)}h(a,y) = 0$ for every $i$ for any
$y$.  Therefore, $H(a,y) = a$ for any $y$; similarly
$H(\bar{\neg}a,y)
= \bar{\neg}a$ for any $y$, that is, ($a$) and ($\bar{\neg}a$)
are loops for any input sequence.

     Now consider the points $l$ of $W$ and let $l^{i}$ denote
${1,..,i}^{-}l$ and
$o^{i}$ denote ${1,..,i}^{-}o$.  We have $l_{1} = (o^{1})_{1} =
1$, $S_{n-2}\{
p_{2},..,p_{n}\} l =1$, and $(o^{1})_{i} = 0$ for any
$i \ne  1$.  Therefore, $H(l,o^{1}) = l^{1}$, so that
$$
     H((\rho n^{-})^{i-1}l,(\rho n^{-})^{i-1}o^{1}) = (\rho
n^{-})^{i-1}l^{1} = (\rho
n^{-})^{i}l.
$$
Therefore, W is a periodic orbit in the FSDS generated by H and
the input 
$$
          V = (o^{1},\rho n^{-}o^{1},...,(\rho
n^{-})^{2n-1}o^{1}).
$$

     H is skew-circular, that is, commutative with $T = \rho
n^{-}$.  The values of the
extended representation $H^{\#}(l,y)$ for $y \in  \mathbf{V}$ are
\begin{eqnarray*}
          H^{\#}(l,o^{1}) &=& (0,1,2,...,n-1),\\
          H^{\#}(l,o^{2}) &=& (0,0,2,3,...,n-1),\\
          H^{\#}(l,o^{3}) &=& (0,0,1,3,4,...,n-1),\\
          H^{\#}(l,o^{4}) &=& (0,0,1,2,4,5,...,n-1),\\
          ..&&...\\
          H^{\#}(l,o^{n-1}) &=& (0,0,1,2,...,n-3,n-1),\\
          H^{\#}(l,l)  &=& (0,0,1,2,...,n-3,n-2),\\
          H^{\#}(l,l^{1}) &=& (1,0,1,2,...,n-2),\\
          H^{\#}(l,l^{2}) &=& (1,1,1,2,3,...,n-2),\\
          H^{\#}(l,l^{3}) &=& (1,1,2,2,3,...,n-2),\\
          H^{\#}(l,l^{4}) &=& (1,1,2,3,3,4,...,n-2),\\
          ..&&...\\
          H^{\#}(l,l^{n-1}) &=& (1,1,2,...,n-2,n-2),\\
          H^{\#}(l,l^{n})   &=& (1,1,2,,...,n-2,n-1),
\end{eqnarray*}

Therefore, in $\mathrm{GRAPH}(H_{V}^{\sim })$,
$$
\begin{array}{ccccccccc}
\,   [l,l] &\rightarrow & [l,o^{n-1}]& \rightarrow & [l,o^{n-2}]&
\rightarrow  &...&
\rightarrow & [l,o^{2}]\\
       & & & & & & & &                   \downarrow \\
\,   [2^{-}l,l^{2}]& \leftarrow & [l,l^{1}]& & & & & &            
   [l,o^{1}]\partial \\
       & & & & & & & &                  \uparrow \\
\,   [l,l^{2}]& \rightarrow & [l,l^{3}] &\rightarrow &  ...& 
\rightarrow  &[l,l^{n-1}]&
\rightarrow  & [l,l^{n}] 
\end{array}
\eqno  (9.6.2)
$$

Now, consider $H(j^{-}l,y)$ for $3 \leq  j \leq  n-1$ and $y \in 
\mathbf{V}$.  First for $y
= l,o^{n-1},..,o^{1},$
\begin{eqnarray*}
          (H^{\#}(j^{-}l,y))_{1} &=& 0,\\  
          (H^{\#}(j^{-}l,y))_{2} &\geq & 1,\\
          (H^{\#}(j^{-}l,y))_{i} &\geq & 2 \; \mbox{if $3 \leq  i
\ne  j$.}
\end{eqnarray*}

Therefore, $[j^{-}l,y] \rightarrow  [1^{-}l,\rho n^{-}y] = [l,y]$
or
$$
\,        [j^{-}l,y] \rightarrow  [{1,j}^{-}l,\rho n^{-}y] =
[(j-1)^{-}l,y],
$$
so that
$$
\begin{array}{llllll}
\,         [j^{-}l,y] &\rightarrow & ...& \rightarrow &
[2^{-}l,y]& \mbox{or}\\
\,         [j^{-}l,y] &\rightarrow & ...& \rightarrow &
[k^{-}l,y]& \mbox{for some
$k$}.
\end{array}
\eqno (9.6.3)
$$
On the other hand, for $y = l^{1},l^{2},..,l^{n}$,
\begin{eqnarray*}
          (H^{\#}(j^{-}l,y))_{1} &\geq & 1,\\
          (H^{\#}(j^{-}l,y))_{2} &\geq & 1,\\
          (H^{\#}(j^{-}l,y))_{i} &\geq & 2 \; \mbox{if $3 \leq  i
\ne  j$.}
\end{eqnarray*}
Therefore,
$$
\,        [j^{-}l,y] \rightarrow  [l,\rho n^{-}y] \; \mbox{or} \;
[j^{-}l,y] \rightarrow 
[j^{-}l,\rho n^{-}y],
$$
so that 
$$
\begin{array}{llllll}
\,        [j^{-}l,y] &\rightarrow & ...&\rightarrow &
[j^{-}l,o^{1}]& \mbox{or}\\
\,        [j^{-}l,y] &\rightarrow  &...&\rightarrow & [l,l^{k}]&
\mbox{for some $k
\geq  2$.} 
\end{array}
\eqno (9.6.4)
$$

     Now, consider $H(2^{-}l,y)$ for $y \in  \mathbf{V}$.  First
for $y = l,o^{n-
1},..,o^{3}$,
\begin{eqnarray*}
          (H^{\#}(2^{-}l,y))_{1} &=& 0, \\ 
          (H^{\#}(2^{-}l,y))_{2} &=& n-2,\\
          (H^{\#}(2^{-}l,y))_{3} &=& 0,\\ 
          (H^{\#}(2^{-}l,y))_{i} &\geq & 1 \; \mbox{if $i \geq 
4$.}
\end{eqnarray*}
Therefore,
$$
\,         [2^{-}l,y] \rightarrow  [l^{3},\rho n^{-}y] = [l,(\rho
n^{-})^{-2}y].
\eqno (9.6.5)
$$
Next, $H^{\#}(2^{-}l,o^{2}) = (0,n-2,1,2,..,n-2)$, so that
$$
\,        [2^{-}l,o^{2}] \rightarrow  [l^{2},o^{3}] = [l,o^{1}].
\eqno (9.6.6)
$$
Also, $H^{\#}(2^{-}l,o^{1}) = (0,n-3,1,2,..,n-2)$, so that
$$
\,        [2^{-}l,o^{1}] \rightarrow  [l^{2},o^{2}] = [l,o].
\eqno (9.6.7)
$$
On the other hand, for $y = l^{3},..,l^{6}$,
\begin{eqnarray*}
          (H^{\#}(2^{-}l,y))_{1} &\geq & 1,\\
          (H^{\#}(2^{-}l,y))_{2} &=& n-3,\\
          (H^{\#}(2^{-}l,y))_{i} &\geq & 1 \; \mbox{if $i \geq 
3$.}
\end{eqnarray*}
Therefore,
$$
\, [j^{-}l,y] \rightarrow  [2^{-}l,\rho n^{-}y] \rightarrow .. 
\rightarrow  [2^{-},o^{1}].
\eqno (9.6,7)
$$
Further, $H^{\#}(2^{-}l,l^{2}) = (1,n-3,0,1,..,n-3)$, so that
$$
\,   [(n-2)^{-}l,l] = [{2,3}^{-}l,l^{3}]\leftarrow 
[2^{-}l,l^{2}].
\eqno (9.6.8)
$$
Also, $H^{\#}(2^{-}l,l^{1}) = (1,n-2,0,1,2,..,n-3)$, so that
$$
\,   [(n-2)^{-}l,o^{n-1}] = [{2,3}^{-}l,l^{2}]\leftarrow 
[2^{-}l,l^{1}].
\eqno (9.6.9)
$$
Therefore, $H_{V}^{\sim }[U_{1}\mathbf{W}\times \mathbf{V}]
\subseteq [U_{1}\mathbf{W}\times \mathbf{V}]$, and
$\mathrm{CY}(H_{V}^{\sim}|[U_{1}\mathbf{W}\times \mathbf{V}])$
consists of a unique loop $[l,o^{1}]\partial $.  To
confirm this, it suffices to check the limit cycles of all
vertices incident to the left arrows
$\leftarrow $ in (9.6.2), (9.6.8), and (9.6.9).  Therefore, by
Proposition 9.5.1, $W$ is an
attractor.  By translating from $H$ into $G = \bar{\neg}H$, we
have obtained:\\

\textbf{Theorem 9.6.2}  In the PDNN generated by $E$ of (9.6.1)
and the input cyclic sequence 
$$
    V = (l^{1},(\rho n^{-})^{(n+1)}l^{1},..,(\rho
n^{-})^{(n+1)(i-1)}l^{1},...),
$$
the cyclic orbit
$$
    W = (o,(\rho n^{-})^{(n+1)}o,..,(\rho
n^{-})^{(n+1)(i-1)}o,...) 
$$
is a bi-dependent attractor.  If the initial state is either $a$
or $\bar{\neg}a$, where $a_{i} = i\%2$,
then every
neuron performs spontaneous firing for any input.\\

In the above theorem, if $n$ is even, then both $V$ and $W$ are
cyclic $2n$-sequences, since $n+1$ and $2n$ are relatively prime.
If $n$ is odd, then both $V$ and $W$ are cyclic $n$-sequences,
since $\mathrm{gcd}(n+1,2n) = 2$.  In this case,
$$
W\prime  = (o^{1},(\rho n^{-})^{(n+1)}o^{1},..,(\rho n^{-
})^{(n+1)(n-1)}o^{1})
$$
 is an attractor in the PDNN generated by $E$ and input
$$
V\prime  = (l^{2},(\rho n^{-})^{(n+1)}l^{2},..,(\rho
n^{-})^{(n+1)(n-1)}l^{2}).
$$
In both cases, note that each attractor
consists of one cyclic orbit.  Therefore, there is no possibility
of shifting from one cyclic orbit to another
within an attractor, caused by some noise, unlike in some
autonomous neural networks.  Computational results expect
that $U_{1}(\mathbf{W})$ is the basin for the attractor and that
the limit orbit of any point outside the basin is neutral. 

\section*{9.7 Attractors in circular PDNNs}  
As an example of autonomous circular transformations, we showed
the existence
of a non-unique attractive $2m$-cycle in Chapter 7 (Example
7.1.2).
We also showed the existence of a non-unique attractor consisting
of
 two $(2m-1)$-cycles (Example 7.1.4).
In this section we show how an autonomous network is modified by
input in a corresponding non-autonomous network, so that such a
non-unique attractor becomes a unique but
bi-dependent attractor.  The bi-dependence means that asymptotic
properties for this attractor are not
completely dictated by the input and are also dependent on
initial states.  Corresponding to Example 7.1.2 of
Chapter 7, the following example shows the existence of a
bi-dependent non-autonomous circular threshold
transformation having a unique attractor.\\

\textbf{Example 9.7.1}  Let $n = 2m$, and let 
$H = <h>: \mathbf{Q}^{2m}\times \mathbf{Q}^{2m}
\rightarrow  \mathbf{Q}^{2m}$ be defined by
$$
        h = p_{1}\cdot p_{m}\cdot \neg p_{2m}\cdot \neg
q_{1}\cdot q_{m}\cdot \neg q_{2m}.
$$

In Example 9.7.1, $x_{i} = 1$ and $H_{i}(x,y) = 0$, iff
$x_{i+m-1}
= 1$, $x_{i+2m-1} = 0$,
$y_{i} = 0$,
$y_{i+m-1} = 1$, and $x_{i+2m-1} = 0$.

    Let $\rho $ be the cyclic permutation $(1,2,..,2m)$.  Let $o$
be the zero vector of
$\mathbf{Q}^{2m}$
and $o^{i}$ be defined as in the previous sections.  Let $W$ be
the $2m$-sequence
$$
        W = (o^{m},\rho o^{m},...,\rho ^{2m-1}o^{m}).
$$
Let $v = \rho ^{-1}l^{m-1}$ and let the input $2m$-sequence $V$
be 
$$
        V = (v,\rho v,...,\rho ^{2m-1}v).
$$
We have
$$
     H(\rho ^{i}o^{m},v) = \left\{
\begin{array}{ll}
     \rho \rho ^{i}o^{m} & \mbox{if}\;  0 \leq  i \leq  m-3,\\
     \rho ^{i}o^{m+1} & \mbox{if}\;  i = m-2,\\
     \rho ^{i}o^{m} & \mbox{if}\;  m-1 \leq  i \leq  2m-1.
\end{array}
\right.
\eqno (9.7.1) 
$$
Therefore, for any $i$ such that $0 \leq  i \leq  m-3$, $H(\rho
^{j}\rho ^{i}o^{m}, \rho ^{j}v) =
\rho
^{j+1}\rho ^{i}o^{m}$ for any $j$.
Therefore,
$$
      \mathrm{Orb}_{H,V}(\rho ^{i}o^{m})
      = ( \rho^{i}o^{m},\rho^{i+1}o^{m},...,\rho^{i-1}o^{m})
      = \sigma ^{i}W \in PO(H,V)
$$
for $0 \leq  i \leq  m-3$.  Let
$$
          \Phi  = \{ \sigma ^{i}W|0 \leq  i \leq  m-3\}.
$$
Our objective is to prove $\Phi$ is an attractor in the FSDS
generated by $H$ and $V$.

Let
$$
        C_{i} = \{ x \mid x \in \mathbf{Q}^{2m}, d_{H}(\rho
^{m}x,x) = 2i\}.
$$
Then clearly $H(x,y) = x$ for any $x \in  C_{0}$ for any $y \in 
\mathbf{Q}^{2m}$; in
particular, $(x)$
is a loop for the input $\sigma ^{i}V$ for any $i$.\\

\textbf{Lemma 9.7.2} $C_{0}^{c} = U_{m-1}\mathbf{W}$, where the
prefix $^c$ denotes the
complement of a subset.

\begin{proof}  For $x \in  \mathbf{Q}^{2m}$, let $j$ be such that
$n_{j} = |\{ i \mid  i \leq m, (\rho ^{j}x)_{i} = 1\}|$
is the maximum.  Let $y = \rho ^{j}x$.  Let
\begin{eqnarray*}
  \alpha &=& |\{ i \mid i \leq m, y_{i} = 1, y_{m+i} = 0\}|,\\
   \beta &=& |\{ i \mid i \leq m, y_{i} = 0, y_{m+i} = 1\} |,\\
  \gamma &=& |\{ i \mid i \leq m, y_{i} = 1, y_{m+i} = 1\}|, \\
   \delta &=& |\{ i \mid i \leq m, y_{i} = 0, y_{m+i} = 0\}|. 
\end{eqnarray*}
Let $x \in  C_{0}^{c}$.  Then $x \in  C_{k}$ for some $k \geq 
1$, so that $\beta < \alpha$.  Therefore, $d_{H}(y,o^{m}) =
2\beta +\gamma +\delta < \alpha+\beta +\gamma +\delta  = m$.
Therefore, $y \in U_{m-1}o^{m}$, so that $x \in
U_{m-1}\mathbf{W}$. 
Conversely, let $x \in  U_{m-1}\mathbf{W}$. 
Then $2\beta +\gamma +\delta  \leq m-1$, so that $\beta <
\alpha$. 
Therefore, $\alpha \geq  1$, so that $x \in  C_{0}^{c}$.
\end{proof}

The following Lemmas are clear.\\                                

\textbf{Lemma 9.7.3}   (i) If $x_{i-1} = x_{m+i-1}$, then
$$
(H_{i}(x,v),H_{m+i}(x,v)) = (x_{i},x_{m+i})
$$
for every $i$.  (ii) If $(H_{i}(x,v),H_{m+i}(x,v)) = (0,1)$ then
$$
(x_{i-1},x_{m+i-1}) = (0,1)\; \mbox{or}\; (x_{i-1},x_{m+i-1}) =
(0,1)
$$
for every $i$.\\

\textbf{Lemma 9.7.4} (i)  If $(x_{i},x_{m+i}) = (0,1)$ for some
$1
\leq  i \leq  m$, then
$$
(H_{i}(x,v),H_{m+i}(x,v)) = (0,1).
$$
(ii)  If $(x_{1},x_{m+1}) = (0,1)$, then
$$
(H_{2}(x,v),H_{m+2}(x,v)) = (0,1).
$$\\

\textbf{Lemma 9.7.5}  (i) If $(x_{i},x_{m+i}) = (1,0)$ for some
$1
\leq  i \leq  m$, and $(x_{i-1},x_{m+i-1}) \ne (0,1)$, then
$$
(H_{i}(x,v), H_{m+i}(x,v)) = (1,0).
$$
(ii) If $(x_{i},x_{m+i}) = (1,0)$ for some $1
\leq  i \leq  m-2$, and $(x_{i-1},x_{m+i-1}) = (0,1)$, then
$$
(H_{i}(x,v), H_{m+i}(x,v)) = (0,1).
$$\\

\textbf{Lemma 9.7.6}  (i) If  $(x_{m},x_{2m}) = (1,0)$, then
$$
(H_{m}(x,v), H_{2m}(x,v)) = (1,0) \;\mbox{and}\;  (H_{1}(x,v),
H_{m+1}(x,v)) = (0,1). 
$$
(ii) If $(x_{m-1},x_{2m-1}) = (1,0)$ and $(x_{m-2},x_{2m-2}) =
(0,1)$, then
$$
(H_{m-1}(x,v),H_{2m-1}(x,v)) = (1,1).
$$
(iii)  If $(x_{m-2},x_{2m-2}) = (1,1)$ and $(x_{m-3},x_{2m-3}) =
(0,1)$, then
$$
(H_{m-2}(x,v),H_{2m-2}(x,v)) = (0,1).
$$ \\

In Example, 9.7.1, $H$ is circular, so that $H$ is commutative
with $T = \rho$.  Further,
$TV_{i} = V_{(i+1)\%k}$ for every $i$.  Therefore, the
transformation $H_{V}$ of
$\mathbf{Q}^{n}\times
\mathbf{V}$  defined by $H_{V}(x,V_{i}) =
(H(x,V_{i}),V_{(i+1)\%k})$  is commutative
with $\rho$.  Let $[S]$ for a subset $S$ of $\mathbf{Q}^{n}\times
\mathbf{Q}^{n}$ denote
$\mathrm{Orb}_{\langle \rho \rangle}S$.  Then, we have
$[U_{m-1}\mathbf{W}\times
\mathbf{V}] = [U_{m-1}\mathbf{W}\times v]$.  Therefore, 
$[U_{m-1}\mathbf{W}\times \mathbf{V}] = [C_{0}^{c}\times
v]$ by Lemma 9.7.2.  We write $x \rightarrow  y$ if $y = \rho
^{-1}H(x,v)$ i.e. $H_{V}(x,v) = \rho (y,v)$ in the following. 
Now let $x \in 
C_{0}^{c}$.  

Case 1:  Assume there is no $i$ such that $1 \leq i \leq m$ and
$(x_{i},x_{m+i}) = (0,1)$. 
Let $1 \leq  k \leq  m$ be the minimum $k$ such
that $(x_{k},x_{m+k}) = (1,0)$.  Then, by Lemma 9.7.3 (i),
$x \rightarrow ... \rightarrow  x\prime $, where
$(x\prime _{1},x\prime _{m+1}) = (1,0)$. 
If there is some $j$ such that $2 \leq  j \leq  m$ and $(x\prime
_{j},x\prime _{m+j}) = (0,1)$, then go to Case 2.  Suppose there
is no $j$ such that $2
\leq  j \leq  m$ and $(x\prime _{j},x\prime _{m+j}) = (0,1)$. 
Let $x\prime  \rightarrow 
x\prime\prime $.  First, let $(x\prime_{m},x\prime_{2m}) =
(1,0)$.  Then, by Lemma
9.7.6 (i), $(x\prime\prime_{m},x\prime\prime_{2m}) =
(x\prime\prime_{m-1},x\prime\prime_{2m-1})
= (1,0)$.  Further, there is no $j$ such that $1 \leq j  \leq
m-2$ and $(x\prime\prime
_{j},x\prime\prime _{m+j}) = (0,1)$ by Lemma 9.7.3 (ii). 
Therefore, $x\prime\prime 
\rightarrow  ... \rightarrow  o^{m}$.
If $(x\prime _{m},x\prime _{2m}) \ne (1,0)$, so that
$x\prime _{m} = x\prime _{2m}$, then
$(x\prime\prime_{m},x\prime\prime{2m}) = (0,1)$
by Lemma 9.7.3 (i), and we go to Case 2.
 
Case 2: Assume $(x_{k},x_{m+k}) = (0,1)$ for some $1 \leq k \leq
m$.  Then  $x
\rightarrow  ... \rightarrow  x\prime $, where
$(x\prime _{1},x\prime _{m+1}) = (0,1)$ by Lemma 9.7.4 (i). 
Let $x\prime  \rightarrow x\prime\prime$.   Then $(x\prime\prime
_{1},x\prime\prime
_{m+1}) = (0,1)$ by Lemma 9.7.4 (ii), and
$(x\prime\prime_{m},x\prime\prime
_{2m}) = (1,0)$ by Lemma 9.7.4 (i).
Let $x\prime\prime  \rightarrow  y$.  Then, $(y_{1},y_{m+1}) =
(0,1)$ by Lemma 9.7.4
(ii),  $(y_{m-1},y_{2m-1}) = (1,0)$ by Lemma 9.7.6 (i), and
$(y_{m},y_{2m}) = (1,0)$
by Lemma 9.7.4 (i).  Let $y \rightarrow y\prime $.

Suppose $(y_{m-2},y_{2m-2}) = (0,1)$.  Then, 
$(y\prime _{m-3},y\prime_{2m-3}) = (0,1)$ by Lemma 9.7.4 (i),
$(y\prime _{m-2},y\prime _{2m-2}) = (1,1)$ by Lemma 9.7.6 (ii),
$(y\prime_{m-1},y\prime _{2m-1})=(1,0)$ by Lemma 9.7.6 (i),
$(y\prime _{m},y\prime _{2m}) = (1,0)$ by Lemma 9.7.4 (i), and
$(y\prime _{1},y\prime _{m+1}) = (0,1)$ by Lemma 9.7.4 (ii).
Let $y\prime  \rightarrow y\prime\prime $.  Then,
$(y\prime\prime _{m-4},y\prime\prime _{2m-4}) = (0,1)$ by Lemma
9.7.4 (i),
$(y\prime\prime_{m-3},y\prime\prime _{2m-3}) = (0,1)$  by Lemma
9.7.6 (iii),
$(y\prime\prime _{m-2},y\prime\prime _{2m-2}) =
 ((y\prime\prime_{m-1}, y\prime\prime_{2m-1}) = (1,0)$ by Lemma
9.7.5 (i), 
$(y\prime\prime _{m},y\prime\prime _{2m}) = (1,0)$ by Lemma 9.7.4
(i), and
$(y\prime\prime_{1},y\prime\prime_{m+1}) = (0,1)$ by Lemma 9.7.4
(ii).  Therefore,
$y\prime\prime  ... \rightarrow  \rho^{m-3}o^{m}$.

Suppose $(y_{m-2},y_{2m-2}) \ne  (0,1)$.  Then
 $(y\prime_{m-2},y\prime _{2m-2}) =
(y\prime _{m-1},y\prime _{2m-1}) = (1,0)$ by Lemma 9.7.5 (i),
$(y\prime _{m},y\prime _{2m}) = (1,0)$ by Lemma 9.7.4 (i), and 
$(y\prime _{1},y\prime _{m+1}) = (0,1)$ by Lemma 9.7.4 (ii).
Therefore, $y\prime \rightarrow y\prime\prime $, where
$(y\prime\prime _{m-2},y\prime\prime_{2m-2})
 =  (y\prime\prime _{m-1},y\prime\prime _{2m-1})
 = (y\prime\prime _{m},y\prime\prime_{2m}) = (1,0)$, and
$(y\prime\prime_{1},y\prime\prime_{m+1}) = (0,1)$.
Let $y\prime\prime \rightarrow  z$.  Then
$(z_{m-3},z_{2m-3}) = (1,0) \; \mbox{or}\; (0,1)$ by Lemma 9.7.5
(i), (ii),
$(z_{m-2},z_{2m-2}) = (z_{m-1},z_{2m-1}) = (z_{m},z_{2m}) =
(1,0)$, and
$(z_{1},z_{m+1}) = (0,1)$.  Therefore,
$z \rightarrow  ... \rightarrow \rho ^{i}o^{m}$ for some $1 \leq 
i \leq  m-3$.

Therefore, $\Phi $ is an attractor in
the FSDS generated by $H$ of Example 9.7.1 and the input $V$.  By
translating from $H$ into
$\bar{\neg}H$, we have obtained:\\

\textbf{Theorem 9.7.7}  In the PDNN generated by $\bar{\neg}H$
for Example 9.7.1 and the input cyclic sequence
$\mathrm{Orb}_{\rho \bar{\neg}}\rho ^{-1}o^{m-1}$,
$\mathrm{Orb}_{\rho^{m+1}}\{\rho ^{i(m+1)}l^{m}|0 \leq i
\leq  m-3\}$
is a bi-dependent attractor. Further, any point of $C_{0}$ is on
a neutral orbit, and $C_{0}^{c}$ is the basin for the attractor.

\end{document}