\documentclass{amsart}
\usepackage{amssymb,amsfonts,latexsym,amsmath}

\begin{document}
                         
\title{Chapter 8 Attractors in second-order neural networks}
\begin{abstract}  Autonomous primitive dynamical neural networks
 (PDNNs) that incorporate temporal summation and a spontaneous
firing
rate of $1/3$ per unit time are constructed. The PDNNs are
second-order neural networks defined as $x(t+1) =
\mathrm{Sgn}(Ax(t)+Bx(t-1)-l)$ such that all the diagonal
elements of the efficacy matrices $A$ and $B$ are negative,
but with additional imaginary delay neurons, they are made to
be a class of McCulloch and Pitts networks $x(t) =
\mathrm{Sgn}(Ex(t-1)-h)$ on $\{ -1,1\} ^{2n}$. PDNN-
definable threshold transformations are characterized in terms
of Boolean functions.  The existence of minimal attractors in
circular PDNNs are proved for the general dimension $n$ by
construction using [\;]-representation of Boolean
transformations.  These attractors include an attractive loop,
an attractive 4-cycle, $3n$-cycle, two $(3/2)n$-cycles, two
$2n$-cycles, and six $(2/3)n$-cycles and provide a variety of
stable periodic firing patterns.
\end{abstract}
\maketitle
\section*{8.1  PDNNs of spontaneous firing rate 1/3}
In Chapter 6, we described a primitive dynamic neural network
(PDNN)
model of spontaneous firing rate $1/2$.  Let $\mathbf{N}$ be the
residue 
class ring $\{ 1,2,..,n\}$.  The state space $\{ -1,1\}
^{\mathbf{N}}$
denoted by $\{ -1,1\} ^{n}$ of this PDNN is a finite metric space
with the
integer-valued Hamming distance $d_{H}$ defined by
 $d_{H}(x,y) = |\{ i \mid  x_{i} \ne  y_{i}\} |$, where $|S|$
denotes the number
of elements of the set $S$.  The PDNN is a finite-state dynamical
system (FSDS) on the state space  $\{ -1,1\}^{n}$ generated by
the threshold transformation $F$ of $\{ -1,1\} ^{n}$.
\begin{eqnarray*}
        Fx &=& \mathrm{Sgn}(Ex),\\
        x(t+1) &=& F(x(t)),    
\end{eqnarray*}
where $E$ is an $n\times n$ real matrix such that $E_{ii} = -1$
for every $i$.  In this model, every neuron oscillates at a
constant spontaneous firing rate $1/2$ per unit time in its
prototype, in which all non-diagonal elements of $E$ are
zero.  In other words, when there is no synaptic input from other
neurons, each neuron performs a neutral activity represented by a
cycle of period 2.  This definition of neutral activity enabled
us to distinguish significant neural activity from insignificant
activity and sort out a great number of loops or 2-cycles often
appearing in the McCulloch and Pitts network (6.2.1).\\

    However, the firing rate of any neuron can not exceed two
times the spontaneous firing rate in this model, and this problem
is due to a greater problem that any state $x(t+1)$ depends only
on $x(t)$ for the given efficacy matrix $E$ and time $t$. 
Therefore the postsynaptic potential $(Ex(t))_{i}$ is spatial
summation.  As suggested by Sherrington's classical experiment on
reflex responses that a small amount of rapidly repeated taps on
the knee tendon produces tendon reflex, the postsynaptic
potential should also include temporal summation.  Further, if
the instantaneous firing rate per unit time of a neuron is
recognized by a neural network, a simple and plausible mechanism
is through temporal summation.\\

    According to McCulloch and Pitts (1943, p. 22), "Temporal
summation may be replaced by spatial summation.  This is obvious:
one need merely introduce a suitable sequence of delaying
chains."  Therefore, temporal summation can be expressed in the
McCulloch-Pitts network by creating auxiliary imaginary delay
neurons.  Let us assume that a single neuron disconnected from
any other neuron performs a spontaneous periodic firing of the
action potentials with period 3, that is, $... \rightarrow  1
\rightarrow  -1 \rightarrow  -1 \rightarrow  1 \rightarrow  -1
\rightarrow  -1 \rightarrow  1 \rightarrow  ...$  In this
sequence, the state of this neuron at time $t+1$ clearly depends
on the states of this neuron at $t$ and $t-1$.

     If we want a PDNN in which $x(t+1) \in  \{ -1,1\} ^{n}$
depends on $t$ and $t-1$, and if we assume that the periodic
firing of the action potentials with period 3 of any neuron, that
is, $... \rightarrow  1 \rightarrow  -1 \rightarrow  -1
\rightarrow  1 \rightarrow  -1 \rightarrow  -1 \rightarrow  1
\rightarrow  ...$ is neutral, then we should create the set of
imaginary delay neurons $\mathbf{N}\prime  = \{ n+1,
n+2,...,2n\}$.  Then
the prototype DNN that is to be modified by further synaptic
connections
can be generated by a threshold transformation $H$ of $\{ -1,1\}
^{2n}$ defined by
$$
        Hx = \mathrm{Sgn}(Dx - e),\quad
    D =
\begin{bmatrix}
   -I & -I \\
    I & O
\end{bmatrix},
\quad e =
\begin{bmatrix}
   l \\
   o 
\end{bmatrix}
\eqno (8.1.1) 
$$
where $I$ is the $n\times n$ identity matrix, $O$ is the $n\times
n$ zero matrix, $l$ is the column $n$-vector whose every
component is 1, and $o$ is the column $n$-vector whose every
component is 0.  Fig. 1 illustrates the state transition of
neurons in the prototype DNN for $n = 4$ in the case where their
initial transition is $1100 \rightarrow  1010$, that is,
$10101100$ is the initial state for $H$.\\


$$
\begin{array}{ccccccccccccccccccccc}
&\mid& \mid& \cdot& \cdot& \mid& \cdot& \cdot& \mid& \cdot&
\cdot& 
 \mid&  \cdot& \cdot& \mid& \cdot& \cdot& \mid& \cdot& \cdot &\\
&\mid& \cdot& \cdot& \mid& \cdot& \cdot& \mid& \cdot& \cdot&
\mid& 
 \cdot&  \cdot& \mid& \cdot& \cdot& \mid& \cdot& \cdot& \mid &\\
&\cdot& \mid& \cdot& \cdot& \mid& \cdot& \cdot& \mid& \cdot&
\cdot& \mid&  \cdot& \cdot& \mid& \cdot& \cdot& \mid& \cdot&
\cdot &\\
&\cdot& \cdot& \mid& \cdot& \cdot& \mid& \cdot& \cdot& \mid&
\cdot& \cdot& \mid& \cdot& \cdot& \mid& \cdot& \cdot& \mid& \cdot
& 
\end{array}
$$
\textbf{Fig. 1}  Prototype DNN. $\mid$ denotes the action
potential and $\cdot$ denotes the
resting potential.  The first 2 columns represent the initial
states at $t= 0$ and $t = 1$; The
second column onward displays spontaneous firing.\\

Then any modified threshold transformation $H$ of $\{ -1,1\}
^{2n}$ obtained by modifying the prototype DNN (8.1.1) is
expressed by 
$$
    Hx = \mathrm{Sgn}(Ex-e),\quad E = 
\begin{bmatrix}
   A & B\\
   I & O
\end{bmatrix},
\eqno (8.1.2) 
$$
where $A$ and $B$ are $n\times n$ real matrix such that $A_{ii} =
B_{ii} = -1$ for every $i$.

    Further, in order to eliminate the threshold vector $e$, we
can add an extra imaginary neuron $2n+1$ and define a self-dual
threshold transformation $F$ of $\{ -1,1\} ^{2n+1}$ as follows:
$$
    Fx = \mathrm{Sgn}(E^{+}x), \quad E^+ =
\begin{bmatrix}
A & B & -l\\
I & O & o\\
o^\mathrm{T} & o^\mathrm{T} & 1
\end{bmatrix},
\eqno (8.1.3) 
$$
where $^{T}$ denotes the transpose.  The relation between $H$ and
$F$ is 
$$
\begin{bmatrix}
 Hx\\
  1
\end{bmatrix}
= F
\begin{bmatrix}
 x\\
 1
\end{bmatrix}.
$$

A topology of the state space $\{ -1,1\} ^{2n}$ can be defined
from the Hamming distance $d_{H}$ on $\{ -1,1\} ^{n}$.  Thus we
can define PDNNs of spontaneous firing rate $1/3$ as follows.\\

\textbf{Definition 8.1.1} A second-order primitive dynamical
neural network
(PDNN) of spontaneous firing rate $1/3$ is a parametrized FSDS on
$\{ -1,1\}^{\mathbf{N}_{2n}}$ with parameters
$$
(\alpha_{12},..,\alpha _{1n},\alpha _{21},\alpha _{23},..,\alpha
_{2n},..,\alpha _{n1},..,\alpha _{nn-1},
 \beta _{12},..,\beta _{1n},\beta _{21},\beta _{23},..,\beta
_{2n},..,\beta
_{n1},..,\beta _{nn-1})
$$
 $\in  \mathbf{R}^{2n(n-1)}$ and generated
by the threshold transformation $H$ defined by (8.1.2) or $F$
defined by (8.1.3), where $A$ and $B$ are $n\times n$ real
matrices such that $A_{ii} = B_{ii} = -1$ for every $i$ and
$A_{ij} = \alpha _{ij}$, $B_{ij} = \beta _{ij}$ for every other
$i$ and $j$.  The $n$ is called the \emph{dimension} of the
PDNN.\\

In this definition, any point $uv \in  \{ -1,1\} ^{2n}$
represents a state transition such that $x(t) = u$ and $x(t-1) =
v$ of real neurons.  The direct recursive equation for $x$ is
obtained from (8.1.1) as 
$$
        x(t+1) = \mathrm{Sgn}(Ax(t)+Bx(t-1)-l).
\eqno (8.1.4) 
$$

Note that if $A_{ij}\cdot B_{ij} \geq  0$ for every $i \ne  j$,
then (8.1.4) is a special deterministic case of the discrete
model constructed by Bressloff and Taylor (1991, (3.6), p. 793),
since the weights $\omega _{ii}(t)$ in that were held fixed at a
negative value for each $i$, and since an equivalent
transformation can be defined by the same efficacy matrix $E$
with the change of the state space from $\{ -1,1\} ^{2n}$ into
$\{ 0,1\} ^{2n}$.  However, $\omega _{ii}(t)$ were made negative
not because of the spontaneous release of chemical transmitters
but because of relative refractory periods.  Some of the examples
given in later sections are in the case where $A_{ij}\cdot B_{ij}
< 0$ for some $i \ne  j$, which may require a different
interpretation.\\

\section*{8.2  PDNN-definable transformations}
The result of this section, which is Proposition 8.2.5, is not
directly used in later sections, but the readers should
familiarize themselves with the concepts, expressions, and
operations defined in this section.  As in the last chapter, we
first give the following definition about PDNN-definable
transformations.\\
      
\textbf{Definition 8.2.1}  If a transformation $F$ of $\{ -1,1\}
^{2n+1}$ can be defined by (8.1.3) for some $n\times n$ real
matrices $A$ and $B$ such that $A_{ii} = B_{ii} = -1$ for every
$i$, then $F$ is called \emph{PDNN-definable}.\\

Let $F_{i}$ be the component function of $F$ defined by $F_i =
p_{i}F$.  The following proposition follows from Theorem
4.2 of Chapter 6 and (8.1.3).\\

\textbf{Proposition 8.2.2} A self-dual threshold transformation
$F$
of $\{ -1,1\} ^{2n+1}$ is PDNN-definable, if and only if 
\begin{enumerate}
 \item $\mathrm{Var}(i^{-}F) \leq  \mathrm{Var}(F) \; \mbox{for
every}\;  i \in  \mathbf{N}$,
 \item $(i,i+n)F_{i} = F_{i} \; \mbox{and}\; (i,2n+1)F_{i} =
F_{i} \; \mbox{for every}\;  i \in  \mathbf{N}$,
 \item $F_{i} = p_{i-n} \; \mbox{for every}\;  i \in 
\mathbf{N}\prime ,
\; \mbox{and}$
 \item $F_{2n+1} = p_{2n+1}$. 
\end{enumerate}

We now describe a PDNN-definable transfpormation of
$\mathbf{Q}^{2n}$  or of
$\mathbf{Q}^{2n+1}$, which is respectively corresponding to $H$
and $F$ defined by (8.1.2) and (8.1.3). $F$ is self dual, so that
$F$ can be
represented by
$$
          f_i = p_i\cdot \neg F_i
\eqno (8.2.1) 
$$
for $i \in \mathbf{N}_{2n+1}$.  Conversely,
$$
        F_{i} = p_{i}\cdot \neg f_{i}\vee \bar{\neg}f_{i}.
\eqno (8.2.2)
$$ 

\textbf{Lemma 8.2.3}  Let $F = [f_{1},...,f_{2n+1}]$ and let
$f_{i}$ for some $i$ be expressed as
$$
        f_{i} = p_{i}\cdot (p_{j}\cdot u_{i}\vee v_{i}),
$$
where $j \ne  i$, $u_{i},v_{i}: 
\mathbf{Q}^{\mathbf{N}_{2n+1}\backslash
\{ i,j\} } \rightarrow  \mathbf{Q}$, and $v_{i} \subseteq 
u_{i}$. 
Then $(i,j)F_{i} = F_{i}$ if and only if $\bar{\neg}v_{i} = \neg
v_{i}$.
\begin{proof}  Let $g_{i} = p_{j}\cdot u_{i}\vee v_{i}$.  Then
\begin{eqnarray*}
    \neg g_{i} &=& (\neg p_{j}\vee \neg u_{i})\cdot \neg v_{i} =
\neg p_{j}\cdot \neg v_{i}\vee \neg u_{i}\cdot \neg v_{i} =
p_{j}\cdot \neg v_{i}\vee \neg u_{i},\\
     \bar{\neg}g_{i} &=& \bar{\neg}p_{j}\cdot \bar{\neg}u_{i}\vee
\bar{\neg}v_{i} =
\neg p_{j}\cdot \bar{\neg}u_{i}\vee \bar{\neg}v_{i}. 
\end{eqnarray*}
Then, by (8.2.2)
\begin{eqnarray*}
    F_{i} &=& p_{i}\cdot \neg f_{i}\vee \bar{\neg}f_{i} =
p_{i}\cdot
\neg (p_{i}\cdot g_{i})\vee  \bar{\neg}(p_{i}\cdot g_{i})\\
       &=& p_{i}\cdot (\neg p_{i}\vee \neg g_{i})\vee
\bar{\neg}p_{i}\cdot
\bar{\neg}g_{i}\\
       &=& p_{i}\cdot p_{j}\cdot \neg u_{i}\vee p_{i}\cdot \neg
p_{j}\cdot \neg v_{i}\vee \neg p_{i}\cdot p_{j}\cdot
\bar{\neg}v_{i}\vee \neg p_{i}\cdot
\neg p_{j}\cdot \bar{\neg}u_{i}.
\end{eqnarray*}
Therefore, since $(i,j)u_{i} = u_{i}$ and $(i,j)v_{i} = v_{i}$, 
$$
    (i,j)F_{i} = p_{i}\cdot p_{j}\cdot \neg u_{i}\vee p_{i}\cdot
\neg p_{j}\cdot \bar{\neg}v_{i}\vee \neg p_{i}\cdot p_{j}\cdot
\neg
v_{i}\vee \neg p_{i}\cdot \neg p_{j}\cdot \bar{\neg}u_{i}.
$$
Therefore, $(i,j)F_{i} = F_{i}$ if and only if $\bar{\neg}v_{i} =
\neg
v_{i}$.
\end{proof}

\textbf{Lemma 8.2.4} 
 Let $v,w: \mathbf{Q}^{\mathbf{N}_{2n+1}\backslash \{ k\}}
\rightarrow  \mathbf{Q}$,
 and $w \subseteq  v$.  Then
 $\bar{\neg}(p_{k}\cdot v\vee w) = \neg (p_{k}\cdot v\vee w)$
 if and only if $w = \neg (\bar{\neg}v)$.
\begin{proof} 
\begin{eqnarray*}
    \bar{\neg}(p_{k}\cdot v\vee w) &=& \neg p_{k}\cdot
\bar{\neg}v\vee \bar{\neg}w =
\neg p_{k}\cdot \bar{\neg}v\vee  p_{k}\cdot \bar{\neg}w.\\
    \neg (p_{k}\cdot v\vee w) &=& (\neg p_{k}\vee \neg v)\cdot
\neg w = \neg p_{k}\cdot \neg w\vee \neg v\\
             &=& \neg p_{k}\cdot \neg w\vee  p_{k}\cdot \neg v.
\end{eqnarray*}
Therefore, $\bar{\neg}(p_{k}\cdot v\vee w) = \neg (p_{k}\cdot
v\vee w)$ if and only if
$\bar{\neg}v = \neg w$ and $\bar{\neg}w = \neg v$, that
is, $w =\neg (\bar{\neg}v)$.
\end{proof}

\textbf{Theorem 8.2.5}  A self-dual threshold transformation $F =
[f_{1},..,f_{2n+1}]$ of $\mathbf{Q}^{2n+1}$ is PDNN-definable, if
and only if 
\begin{enumerate}
 \item $f_{i} = p_{i}\cdot ( p_{i+n}\cdot p_{2n+1}\cdot u_{i}\vee
(p_{i+n}\vee p_{2n+1})\cdot v_{i}\vee \neg (\bar{\neg}v_{i}))$
for $i \in  \mathbf{N}$, such that $\neg (\bar{\neg}v_{i})
\subseteq v_{i} \subseteq  u_{i}$,
where
$u_{i}, v_{i}: \mathbf{Q}^{\mathbf{N}_{2n+1}\backslash \{
i,i+n,2n+1\}} \rightarrow 
\mathbf{Q}$.
\item $f_{i} = p_{i}\cdot \neg p_{i-n}$ for $i \in 
\mathbf{N}\prime $, and
 \item $f_{2n+1} = \emptyset $. 
\end{enumerate}
\begin{proof}  Assume condition 1 of the present theorem.  Then
clearly, $(i,i+n)F_{i} = F_{i}$ for every $i \in  \mathbf{N}$.
$(i,2n+1)F_{i} = F_{i}$ for every $i \in  \mathbf{N}$ by Lemmas
8.2.3 and
8.2.4.  Therefore, condition 2 of Proposition 8.2.2 is satisfied. 
We
have $|\neg \bar{\neg}v_{i}| = 2^{2n+1}-|v_{i}|$ and
$|v_{i}| \geq |\neg (\bar{\neg}v_{i})|$. 
Therefore,
$|v_{i}| \geq  �\times 2^{2n+1}$, so that $|f_{i}| \geq 
1/2\times
3/4\times |v_{i}| + 1/2\times 1/4\times
(2^{2n+1}-|v_{i}|) = 2^{2n-2} + 1/4\times |v_{i}|
\geq  2^{2n-2} + 1/4\times 1/2 \times 2^{2n+1} = 2^{2n-1}$. 
Therefore, $\mathrm{Var}(i^{-}F )\leq  \mathrm{Var}(F)$, so that
condition 1 of Proposition 8.2.2 is also satisfied.
 
    Assume F is PDNN-definable.  Any threshold function is unate
(see Theorem 4.2.5 of Chapter 4).  Also, $(i, i+n)F_{i} = F_{i}$
by condition 2 of Proposition 8.2.2.  Therefore,
$$
     f_{i} = p_{i}\cdot (p_{i+n}\cdot p_{2n+1}\cdot u_{i}\vee
(p_{i+n}\vee p_{2n+1})\cdot v_{i}\vee w_{i}),
$$
for every $i \in  \mathbf{N}$, or
$$
     f_{i} = p_{i}\cdot (\neg p_{i+n}\cdot \neg p_{2n+1}\cdot
u_{i}\vee (\neg p_{i+n}\vee \neg p_{2n+1})\cdot v_{i}\vee 
w_{i}),
$$
for every $i \in  \mathbf{N}$, where 
$u_{i},v_{i},w_{i}: \mathbf{Q}^{\mathbf{N}_{2n+1}\backslash
\{ i, i+n,2n+1\}} \rightarrow  \mathbf{Q}$,
 and $w_{i} \subseteq v_{i} \subseteq  u_{i}$.  Suppose $f_{i}$
takes the second form. 
Since $F$ is defined by (8.1.3) with the diagonal elements of $A$
and $B$ being $-1$,  $f_{i} = p_{i}\cdot (u_{i}\vee v_{i}\vee
w_{i})$,
 so that $f_{i}$ takes the first form.  Then the condition $(i,
i+n)F_{i} = F_{i}$ in
condition 2 of Proposition 8.2.2 implies $w_{i} = \neg
(\bar{\neg}v_{i})$ by Lemmas 8.2.3
and 8.2.4. 
Therefore, condition 2 of the present proposition is satisfied. 
Conditions 2 and 3 of the present theorem are respectively
equivalent to conditions 3 and 4 of Proposition 8.2.2.
\end{proof}

Let $\tau $ denote the permutation
$(1,2,..,n)(n+1,..,2n)$.  A
second-order PDNN of spontaneous firing rate $1/3$ generated by a
PDNN-definable $H$ of $\mathbf{Q}^{2n}$ or $F$ of
$\mathbf{Q}^{2n+1}$ respectively defined by (8.1.2) and (8.1.3)
is called \emph{circular}, if $F\tau  = \tau F$ or $H\tau  = \tau
H$.  The PDNN is circular if and only if $F_{i} = \tau ^{i-
1}F_{1}$, or $f_{i} = \tau ^{i-1}f_{1}$ for $i = 1,..,n$, where
$f_{i}$ is defined by (8.2.1).  Therefore, a circular PDNN
generated by $F$ is denoted by $F = \langle f_{1}\rangle$. 
Further, a circular PDNN generated by $H$ is denoted by $H =
\langle h^{1}, h^{0}\rangle$, where
$$
\begin{array}{lll}
  h^{1}(x_{1},..,x_{2n}) &=&  f_{1}(x_{1},..,x_{2n},1),\\
  h^{0}(x_{1},..,x_{2n}) &=&  f_{1}(\neg x_{1},..,\neg
x_{2n},1).
\end{array}
\eqno (8.2.3)
$$

If $i \leq  n$ and $x_{i} = 1$, then $(Hx)_{i} = 0$ if and only
if $x \in \tau ^{i-1}h^{1}$.  If $i \leq  n$ and $x_{i} = 0$,
then $(Hx)_{i} = 1$ if and only if $x \in \tau ^{i-1}h^{0}$.

A point $x$ of $\mathbf{Q}^{2n}$ is expressed by $uv$, where $u$
and $v$ are points on $\mathbf{Q}^{n}$.  Let $[uv]$ denote
$\mathrm{Orb}_{\langle \tau \rangle }uv$. 
If $D$ is a subset of $\mathbf{Q}^{2n}$, then let $[D]$ denote
$\mathrm{Orb}_{\langle \tau \rangle }D$.\\

As in Chapter 7, we define the \emph{extended representation} of
a Boolean function as
follows.  For $H = \langle h^{1}, h^{0}\rangle$. let $h^1|1$
and $h^0|0$ be the Boolean functions defined on
$\mathbf{Q}^{\mathbf{N}\cup \mathbf{N}\prime \backslash \{ i\}}$
such that   
\begin{eqnarray*}
      (h^1|1)(x_2,..,x_{2n}) &=& h^1(1,x_2,..,x_{2n}),\\
      (h^0|0)(x_2,..,x_{2n}) &=& h^0(0,x_2,..,x_{2n}).
\end{eqnarray*}

\textbf{Definition 8.2.6 } Let $x$ be an element of
$\mathbf{Q}^{2n}$ and $H = \langle
h^{1}, h^{0}\rangle$.  Then the extended representation $H^{\# }$
of $H$ is the function
from $\mathbf{Q}^{n}$ to $\mathbf{Z}^{n}$ defined by
$$
  (H^{\# }x)_{i} = \left\{
  \begin{array}{ll} 
     d_{SH}(P_{\mathbf{N}\cup \mathbf{N}\prime \backslash \{ i\}}
\tau^{-(i-1)}x,h^{1}|1) & \mbox{ if $x_i = 1$}\\
     d_{SH}(P_{\mathbf{N}\cup \mathbf{N}\prime \backslash \{ i\}}
\tau^{-(i-1)}x,h^{0}|0) & \mbox{ if $x_i = 0$}
  \end{array} 
           \right.
$$
for $i = 1, . . , n$.  It is clear 
$$
          x_{i} \ne (Hx)_{i} \; \mbox{ if and only if $(H^{\#
}x)_{i} \leq 0$}.
\eqno (8.2.4)
$$

\section*{8.3  Attractive loops}
  In this section it is assumed that $\mathbf{Q}^{2n}$ is a
metric space with the distance $d_{H}$ on $\mathbf{Q}^{2n}$.
In this and later sections we prove the existence of a variety of
attractors in circular
second-order PDNNs.  The situation is more complex than in our
first-order PDNNs, since a
kind of symmetry due to the self-duality of generating
transformations no more exists.  In fact, we have to deal with
the non-symmetric pair of Boolean functions $h^{1}$ and $h^{0}$
defined by (8.2.3).\\

\textbf{Example 8.3.1}  Consider the prototype DNN, where $A_{ij}
=
B_{ij} = 0$ for every $i \ne  j$ in (8.1.2).  Let the
corresponding transformation of $\mathbf{Q}^{2n+1}$ be also
denoted by $F$.  Then $F_{i} = S_{2}\{ \neg p_{i},\neg
p_{i+n},\neg p_{2n+1}\}$, that is, $F = [f_{1},..,f_{2n+1}]$,
\begin{eqnarray*}
    f_{i} &=& p_{i}\cdot (p_{i+n}\vee p_{2n+1}) \mbox { for } i
\in  \mathbf{N}.\\
    H &=& <h^{1},h^{0}>,\quad h^{1} = p_{1},\quad h^{0} = \neg
p_{1}\cdot \neg p_{i+n}.    
\end{eqnarray*}

\textbf{Example 8.3.2}  Let $n \geq  3$, $A_{ij} = \epsilon $,
$B_{ij} = \epsilon $ for every $i \ne  j$ in (8.1.2), where
$\epsilon  > 3/2$.  Let the corresponding transformation of
$\mathbf{Q}^{2n+1}$ be $F$.  Then
\begin{eqnarray*}
    F_{1} &=& S_{n}\{
p_{2},p_{3},..,p_{n},p_{n+2},p_{n+3},..,p_{2n}\}\\
       & & \vee  S_{2}\{ \neg p_{1},\neg p_{n+1},\neg
p_{2n+1}\}\cdot S_{n-1}\{
p_{2},p_{3},..,p_{n},p_{n+2},p_{n+3},..,p_{2n}\}.
\end{eqnarray*}
Therefore, $F = \langle f \rangle $, 
\begin{eqnarray*}
    f &=& p_{1}\cdot (p_{n+1}\vee p_{2n+1})\cdot 
            S_{n-1}\{ \neg p_{2},\neg p_{3},..,\neg p_{n},\neg
p_{n+2},\neg p_{n+3},..,\neg p_{2n}\}\\
      & & \vee p_{1}\cdot S_{n}\{ \neg p_{2},\neg p_{3},..,\neg
p_{n},\neg p_{n+2},\neg p_{n+3},..,\neg p_{2n}\}.
\end{eqnarray*}
It is confirmed that $f$ satisfies 1 of Theorem 8.2.5.  Further,
$H
= <h^{1},h^{0}>$,
\begin{eqnarray*}
    h^{1} &=& p_{1}\cdot S_{n-1}\{ \neg p_{2},\neg p_{3},..,\neg
p_{n},\neg p_{n+2},\neg p_{n+3},..,\neg p_{2n}\},\\
    h^{0} &=& \neg p_{1}\cdot S_{n}\{
p_{2},p_{3},..,p_{n},p_{n+1},p_{n+2},p_{n+3},..,p_{2n}\}.
\end{eqnarray*}       

\textbf{Theorem 8.3.3}  In Example 8.3.2, $(ll)$ and $(oo)$ are
the
only loops of $H$.  Further, they are attractors in the PDNN
generated by $H$.
\begin{proof}  Let $x$ be a fixed point of $H$,
and let $x_1 = 1$. Then
$$
    x \notin  p_{1}\cdot S_{n-1}\{ \neg p_{2},\neg
p_{3},..,\neg p_{n},\neg p_{n+2},\neg p_{n+3},..,\neg p_{2n}\};
$$
otherwise $(Hx)_{1} = 0 \ne  x_{1}$. On the other hand,
$(Hx)_{i+n} = x_{i}$ and $(Hx)_{i+n} = x_{i+n}$.  Therefore,
$x_{i} = x_{i+n}$ for every $i \in \mathbf{N}$. 
Therefore, at least $n+2$ of $x_{1},... ,x_{2n}$ are 1.  Suppose
$x_{j} = x_{j+n} = 0$ for some $j$.  Then $x \in 
\bar{\neg}f_{j}$, contrary to the assumption that $x$ is a fixed
point of $H$. Therefore $x_{i} = 1$ for every $i \in
\mathbf{N}_{2n}$. Similarly if $x$ is a fixed point of $H$ and
$x_{1} = 0$, then $x_{i} = 0$ for every $i \in \mathbf{N}_{2n}$. 
Therefore, if $x$ is a fixed point of $H$, then $x$ is $ll$ or
$oo$.  In fact, each of them is a fixed point.


Let $x \in  U_{n-2}(ll)$.  Then if $x_{i} = 0$ for $i \leq n$,
then $(Hx)_{i} = 1$.  If $x_{i} = 1$ for $i \leq n$, then
$(Hx)_{i} = 1$.   Therefore, $Hx \in U_{n-2}(ll)$ and $H^{2}x =
ll$.  Let $U_{n-1}(oo)$.  Then if $x_{i} = 0$ for $i \leq n$,
then $(Hx)_{i} = 0$.  If $x_{i} = 1$ for $i \leq n$, then
$(Hx)_{i} = 0$.  Therefore,  $Hx \in  U_{n-1}(oo)$ and
$H^{2}x = oo$.  Therefore, $(ll)$ and $(oo)$ are attractors of
$H$.
\end{proof}

\textbf{Example 8.3.4}  Let $A_{ij} = 1$, $B_{ij} = 1$ for every
$i
\ne  j$ in (8.1.2).  This PDNN is structurally stable, since
$2n+1$ is odd.  Let the corresponding transformation of
$\mathbf{Q}^{2n+1}$ be $F$.  Then
$$
    F_{1} = S_{n+1}\{ \neg p_{1},p_{2},..,p_{n},\neg
p_{n+1},p_{n+2},..,p_{2n},\neg p_{2n+1}\}.
$$
Therefore, $F = \langle f \rangle $,
\begin{eqnarray*}
    f &=& p_{1}\cdot S_{n+1}\{ p_{1},\neg p_{2},..,\neg
p_{n},p_{n+1},\neg p_{n+2},..,\neg  p_{2n},p_{2n+1}\}\\
      &=& p_{1}\cdot \cdot S_{n}\{ \neg p_{2},..,\neg
p_{n},p_{n+1},\neg p_{n+2},..,\neg  p_{2n},p_{2n+1}\}.
\end{eqnarray*}
Therefore, $H = <h^{1}, h^{0}>$,
\begin{eqnarray*}
    h^{1} &=& p_{1}\cdot S_{n-1}\{ \neg p_{2},\neg p_{3},..,\neg
p_{n},p_{n+1},\neg p_{n+2},..,\neg p_{2n}\},\\
    h^{0} &=& \neg p_{1}\cdot S_{n}\{ p_{2},p_{3},..,p_{n},\neg
p_{n+1},p_{n+2},..,p_{2n}\}.
\end{eqnarray*}
This example has the same attractors as Example 8.3.2.\\


\section*{8.4  Attractive 4-cycles} 
In this section it is assumed that $\mathbf{Q}^{2n}$ is a metric
space with the distance
$$
        d(uv,yz) = \mathrm{max}(d_{H}(u,y), d_{H}(v,z)),
$$
where $u, v, y, z \in  \mathbf{Q}^{n}$.  Further, if $a \in 
\mathbf{Q}^{n}$, then let
$$
    U_{ij}(ab)
     = \{ uv \mid  u,v \in  \mathbf{Q}^{n}, d_{H}(u,a) = i,
d_{H}(v,b) = j\}.
$$
Clearly, $U_{1}(ab)$ is a mutually disjoint union of
$U_{10}(ab)$, $U_{01}(ab)$, $U_{11}(ab)$, and $\{ ab\}$.\\


\textbf{Example 8.4.1}  Let $n$ be even and $n \geq  4$.  By
modifying Example 8.3.2, let $H = \langle h^{1}, h^{0} \rangle$,
\begin{eqnarray*}
  h^{1} &=& p_{1}\cdot S_{n-1}\{ p_{2},\neg p_{3},p_{4},..,\neg
p_{n-1},p_{n},\neg p_{n+2},p_{n+3},\neg p_{n+4},..,p_{2n-1},\neg
p_{2n}\},\\
  h^{0} &=& \neg p_{1}\cdot S_{n}\{ \neg p_{2},p_{3},\neg
p_{4},..,p_{n-1},\neg p_{n},\neg p_{n+1},p_{n+2},..,\neg
p_{2n-1},p_{2n}\}.
\end{eqnarray*}

\textbf{Proposition 8.4.2}  Let $c = 1010\cdot \cdot \cdot 10 \in 
\mathbf{Q}^{n}$.  Then
$$
        cc \rightarrow  (\bar{\neg}c)c \rightarrow 
(\bar{\neg}c)(\bar{\neg}c)
\rightarrow  c(\bar{\neg}c) \rightarrow  cc. 
$$
is a cycle of $H$ defined in Example 8.4.1.
\begin{proof} We prove the proposition for $n = 4$. In this case,
$c = 1010$, and
\begin{eqnarray*} 
     h^{1} &=& 1\cdot S_{3}\{ 2,\neg 3,4,\neg 6,7,\neg 8\} ,\\
     h^{0} &=& \neg 1\cdot S_{4}\{ -2,3,-4,-5,6,-7,8\} .
\end{eqnarray*}
Then 
$$
     H^{\# }(cc) = (0,0,0,0), H^{\# }((\bar{\neg}c)c) =
(4,3,4,3).
$$
Therefore,
$$
     cc \rightarrow  (\bar{\neg}c)c \rightarrow 
(\bar{\neg}c)(\bar{\neg}c).
$$
Since $\rho c = \bar{\neg}c$,
$$  
     cc \rightarrow  (\bar{\neg}c)c \rightarrow 
(\bar{\neg}c)(\bar{\neg}c) \rightarrow 
c(\bar{\neg}c) \rightarrow  cc.
$$ 
\end{proof}

\textbf{Theorem 8.4.3}  Let $c = 1010\cdot \cdot \cdot 10 \in
\mathbf{Q}^{n}$
 and $n \geq  4$.  Then
$$
        A =
(cc,(\bar{\neg}c)c,(\bar{\neg}c)(\bar{\neg}c),c(\bar{\neg}c),cc)
$$
is an attractive cycle of $H$ defined in Example 8.4.1. 
\begin{proof}  We have $[cc] = \{ cc,
(\bar{\neg}c)(\bar{\neg}c)\}$ and $[(\bar{\neg}c)c]
= \{ (\bar{\neg}c)c, c(\bar{\neg}c)\}$ .  Also, $U_{10}[cc] =
[(1^{-}c)c]\cup [(2^{-}c)c]$
and $U_{01}[cc] = [c(1^{-}c)]\cup [c(2^{-}c)]$.  We prove the
theorem for $n = 4$
without loss of generality. 

     (i) Consider a point $x$ of $U_{01}((\bar{\neg}c)c)$. As
shown in the proof of
Proposition 8.4.2, $(H^{\# }((\bar{\neg}c)c))_{i} \geq  3$  for
every $i$.  Therefore,
$(H^{\# }x)_{i} \geq  2$  for every $i$, so that
$Hx = (\bar{\neg}c)(\bar{\neg}c)$.

     (ii)  Let $x \in  U_{01}(cc)$.  It suffices to consider $x =
c(1^{-}c)$ or $c(2^{-}c)$ . 
First, if $i$ is even then $(H^{\# }x)_{i} = 1$.  Specifically,
$H^{\# }(c(1^{-}c)) =
(0,1,1,1)$ and $H^{\# }(c(2^{-}c)) = (1,1,1,1)$.  Therefore,
$H(c(1^{-}c)) =(1^{-}c)c$ and
$H(c(2^{-}c)) = cc$.

     (iii) Let $x = uv \in  U_{10}((\bar{\neg}c)c)$.  It suffices
to consider $x = (1^{-
}\bar{\neg}c)c$ or 
$(2^{-}\bar{\neg}c)c$.   $H^{\# }(1^{-}\bar{\neg}c)c)= (-3,2,3
2)$ and $H^{\# }(2^{-
}\bar{\neg}c)c) = (3.-3,3,2)$.  Therefore,
$H((1^{-}\bar{\neg}c)c) = (\bar{\neg}c)(1^{-}\bar{\neg}c)$ and
$H((2^{-}\bar{\neg}c)c) =
(\bar{\neg}c)(2^{-}\bar{\neg}c)$.  Therefore,
$Hx \in  U_{01}((\bar{\neg}c)(\bar{\neg}c))$.

     (iv)  Note that $|(H^{\# }x)_{i}| \geq  2$ for every
$i$.  Therefore, if 
$x \in   U_{11}((\bar{\neg}c)c)$, then $Hx \in 
U_{01}((\bar{\neg}c)(\bar{\neg}c))$ also.

     (v)  Let $x \in  U_{10}(cc)$.  Then it suffices to consider
$x = (1^{-}c)c$ or $(2^{-
}c)c$. 
$H^{\# }((1^{-}c)c) = (1,-1,-1,-1)$ and $H^{\# }((2^{-}c)c) =
(-1,0,-1,-1)$.  Therefore,
$H((1^{-}c)c) = (\bar{\neg}c)(1^{-}c) \in  
U_{01}((\bar{\neg}c)c)$ and $H((2^{-}c)c) =
(2^{-}\bar{\neg}c)(2^{-}c) \in   U_{11}((\bar{\neg}c)c)$.

     (vi) Let $x \in  U_{11}(cc)$.  It suffices to consider $x =
(1^{-}c)(j^{-}c)$ or $(2^{-
}c)(j^{-}c)$ for some $j$.  $H^{\# }((1^{-}c)c) = (1,-1,-1,-1)$
by (iv), so that $(H^{\#
}((1^{-}c)(j^{-}c)))_{i} \leq  0$ for 
$i = 2, 3, 4$,  so that $(H((1^{-}c)(j^{-}c))_{i}  =
(H((1^{-}c)c))_{i}$ for $i = 2,3,4$.  
$H^{\# }((2^{-}c)c) = (-1,0,-1,-1)$ by (iv), so that $(H^{\#
}((2^{-}c)(j^{-}c)))_{i} \leq  0$
for $i = 1, 3, 4$,  so that $(H((2^{-}c)(j^{-}c))_{i}  =
(H((2^{-}c)c))_{i}$ for $i = 1,3,4$. 
Therefore  $Hx \in  U_{11}((\bar{\neg}c)c)$.

     Therefore, the subsets of $U_{1}\mathbf{A}$ are mapped as
shown in the following
flow subgraph.\\

$$
\begin{array}{ccccc}
    \qquad U_{11}[cc] \cup  [(2^-c)c]  & & & &\\
     \qquad   \downarrow & & & &\\
U_{10}[(\bar{\neg}c)c] \cup U_{11}[(\bar{\neg}c)c]& \rightarrow & 
[c(1^-c)] \cup [c(2^-c)] &   & [(\bar{\neg}c)c)]\\
  & \swarrow &  & \searrow &  \updownarrow \\                     
\quad  [(1^-c)c]& \rightarrow &   U_{01}[(\bar{\neg}c)c] &
\rightarrow &   [cc].
\end{array}
$$

Therefore,  $H(U_{1}\mathbf{A}) \subseteq  \mathbf{A}$ and
$\mathbf{\omega
}_{H}(U_{1}\mathbf{A}) = \mathbf{A}$.  
\end{proof}
  
   Fig. 2 illustrates the state transition of neurons in Example
   8.4.1 for $n = 4$ in the case where their initial transition
is
   $0010 \rightarrow  1110$, that is, $11100010 \in 
   U_{1}\mathbf{A}$ is the initial state for $H$.\\

$$
 \begin{array}{ccccccccccccccccccccccccccc}
&&\cdot& \mid& \cdot& \cdot& \cdot& \mid& \mid& \cdot& \cdot&
  \mid& \mid& \cdot& \cdot& \mid& \mid& \cdot& \cdot& \mid& \mid& 
 \cdot& \cdot& \mid& \mid &&\\
&&\cdot& \mid& \cdot& \mid& \cdot& \cdot& \cdot& \mid& \mid&   
 \cdot& \cdot& \mid& \mid& \cdot& \cdot& \mid& \mid& \cdot&  
 \cdot& \mid& \mid& \cdot& \cdot&&\\ 
&&\mid& \mid& \cdot& \cdot& \cdot& \mid& \mid& \cdot& \cdot&
 \mid& \mid& \cdot& \cdot& \mid& \mid& \cdot& \cdot& \mid& \mid&
\cdot& \cdot& \mid& \mid&& \\
&&\cdot& \cdot& \mid& \mid& \mid& \cdot& \cdot& \mid& \mid&
 \cdot& \cdot& \mid& \mid& \cdot& \cdot& \mid& \mid& \cdot&
\cdot& \mid& \mid& \cdot& \cdot&& 
   \end{array}
   $$
   \textbf{Fig. 2} Attractor of Example 4.4.1. Any consecutive 4
columns from the sixth
column display the attractor. \\

  \section*{8.5  Attractive cycles for $n \equiv  2$ mod 6}
   It is assumed hereafter that $\mathbf{Q}^{2n}$ is a metric
   space with the distance
           $d(uv,yz) = 2d_{H}(u,y)+d_{H}(v, z))$,
   where $u, v \in  \mathbf{Q}^{n}$.
  Also $H^{\# }$ is defined as the same way as the last section. 
  Further, in this and the next section, $n$ is even, 
$l = 1^{n/2}$, and $o = 0^{n/2}$.  The cyclic permutation
$(1,2,..,n)$ is
   denoted by $\rho $.\\
 
   \textbf{Example 8.5.1}  By modifying Example 8.3.4, let $H =
<h^{1}, h^{0}>$,
   $$
   \begin{array}{lll}
     h^{1} &=& p_{1}\cdot S_{n-1}\{ p_{2},..,p_{n/2},\neg
   p_{n/2+1},..,\neg p_{n},p_{n+1},..,p_{n+n/2}, \neg
   p_{n+n/2+1},..,\neg p_{2n}\},\\
       h^{0} &=& \neg p_{1}\cdot S_{n}\{ \neg p_{2},..,\neg
   p_{n/2},p_{n/2+1},..,p_{n},\neg p_{n+1},..,\neg p_{n+n/2},    
      p_{n+n/2+1},..,p_{2n}\} .
   \end{array}
   \eqno  (8.5.1)   
   $$ 
   

Clearly,
$$
          (H^{\# }(\bar{\neg}x))_{i} = \left\{
          \begin{array}{ll}
         (H^{\# }(x))_{i} + 1 & \mbox{if $x_{i} = 1$},\\
     (H^{\# }(x))_{i} - 1   & \mbox{if $x_{i} = 0$}.
      \end{array} 
              \right.,\\
$$
and
$$
          (H^{\# }(i^{-}x))_{i} = -(H^{\# }(x))_{i}. 
$$        
Let $x \in  \mathbf{Q}^{2n}$ and $i\leq  n$.  Then, $x_{i}\cdot
(Hx)_{i} = 0$ if and only
if $(H^{\# }x)_{i} \leq  0$.\\

Let $a = lo\rho ^{-r}(lo)$.  Then $(H^{\# }a)_{i} =
4(i-1)-(n-2r)$ for $1 \leq  i \leq  \in $. 
Therefore, if $(H^{\# }a)_{r} = 0$, then $r = (n+4)/6$, so that
$n \equiv  2$ mod 6.\\
      
\textbf{Definition 8.5.2}  Let $n \equiv  2$ mod 6 and $r =
(n+4)/6$.  Let $a = lo\rho
^{-r}(lo)$ and
$$
          S = [{a,(n/2)^{-}a,(2n-r+1)^{-}a}].
$$

 \textbf{Proposition 8.5.3}  If $n \equiv  2$ mod 6 and $r =
(n+4)/6$, then $S$ is an
invariant set of $H$.
\begin{proof}  We prove for $r = 2$ and $n = 8$.  $H^{\# }a =
(-4,0,3,4,-3,1,4,5)$, so
that $Ha = \tau ^{2}(4^{-}a)$.  $H^{\# }(4^{-}a) =
(-3,1,5,-4,-2,2,6,6)$, so that $H(4^{-
}a) = \tau (15^{-}a)$.  $H^{\# }(15^{-}a) =
(-5,-1,2,5,-4,0,3,6)$, so that $H(15^{-}a) =
\tau ^{2}a$.  Therefore, $HS = S$.
\end{proof}   
 
 \textbf{Theorem 8.5.4}  If $n \equiv  2$ mod 6 and $n \geq  8$,
then $H(U_{1}S)
\subseteq  U_{1}S$ and $\mathbf{\omega }_{H}(U_{1}S) = S$. 
Therefore,
$\mathrm{Orb}_{H}S$ is an attractor.
\begin{proof}  Let $j \in  \mathbf{N}\prime$.  Since $(H^{\#
}(a))_{2} = 0$,
 $(H^{\# }(j^{-}a))_{2} = 1$ or $(H^{\# }(j^{-}a))_{2} = -1$.
 Let $(H^{\# }(j^{-}a))_{2} = 1$.  Then, $(H^{\# }(j^{-}a))_{6} =
2$ and $(H^{\# }(j^{-
}a))_{i} = (H^{\# }a)_{i} \pm 1$ for every other $i$.  Therefore,
$H(j^{-}a) = 2^{-}(Ha)
= \tau (\{ 11,15\} -a) \in  \tau U_{1}(15^{-}a)$.  Let $(H^{\#
}(j^{-}a))_{2} = -1$.  Then,
$(H^{\# }(j^{-}a))_{6} = 0$ and $(H^{\# }(j^{-}a))_{i} = (H^{\#
}a)_{i} \pm 1$ for every
other $i$.  Therefore, $H(j^{-}a) = 6^{-}(Ha) = \tau ^{2}a \in 
S$.

     $(H^{\# }(j^{-}(4^{-}a))_{i} = (H^{\# }(4^{-}a))_{i} \pm 1$
for every $i$.  Let
$(H^{\# }(j^{-}(4^{-}a))_{2} = 0$.  Then
$$
 H(j^{-}(4^{-}a)) = 2^{-}H(4^{-}a) = 2^{-}\tau (15^{-}a) = \tau
^{2}({4,10}^{-}a \in 
\tau ^{2}U_{1}(4^{-}a),
$$
and $(H^{\# }(10^{-}(4^{-}a)))_{2} \ne  0$.  If $(H^{\#
}(j^{-}(4^{-}a)))_{2} \ne  0$, then
$H(j^{-}(4^{-}a)) = H(4^{-}a) \in  S$.

     Let $(H^{\# }(j^{-}(15^{-}a)))_{2} = 0$.  Then $(H^{\#
}(j^{-}(15^{-}a)))_{6} =
1$  and $(H^{\# }(j^{-}(15^{-}a))_{i} = (H^{\# }(15^{-}a))_{i}
\pm 1$ for every other $i$. 
Therefore, $H(j^{-}(15^{-}a)) = 6^{-}H(15^{-}a) = \tau
^{2}(4^{-}a) \in  S$.  If
$(H^{\# }(j^{-}(15^{-}a)))_{2} \ne  0$, then $H(j^{-}(15^{-}a)) =
H(15^{-}a) \in  S$.
     Therefore, subsets of $U_{1}S$ are mapped as shown in the
following flow subgraph. 

 $$
\begin{array}{ccccc}  
           \; [j^{-}a] &  \rightarrow  &  [k^{-}(15^{-}a)]& & \\  

                (H^{\# }(j^{-}a))_{2} = 1 & & &          
\searrow  & \\
     & & & & \\
       \;  [j^{-}(4^{-}a)]&    \rightarrow  &    [k^{-}(4^{-}a)]
&         \rightarrow  &  
     S \\
      (H^{\# }(j^{-}(4^{-}a)))_{2} = 0&   & (H^{\#
}(k^{-}(4^{-}a)))_{2} \ne  0 & & \\
     & & &                               \nearrow  & \\     
       &    &      [j^{-}a] & &  \\
           &    &           (H^{\# }(j^{-}a))_{2} = -1& &  
\end{array}            
$$

Therefore, $H(U_{1}S) \subseteq  S$ and $\mathbf{\omega
}_{H}(U_{1}S) = S$.
\end{proof} 

 \textbf{Proposition 8.5.5}  The attractor $\mathrm{Orb}_{H}S$
consists of one $3n$-cycle
if $r$ is even, and two $(3/2)n$-cycles if $r$ is odd.
\begin{proof}  $H^{3}a = \tau ^{3r-1}a$ follows from the proof of
Theorem 8.5.4. 
Further,
 $\mathrm{gcd}(n,3r-1) = \mathrm{gcd}(2(3r-2),3r-1)$.  Therefore,
$\mathrm{gcd}(n,3r-1)
= 1$ if $r$ is even, 2 if $r$ is
odd.  
\end{proof}
   
   Computational results for $n = 8$ expect that Theorem 8.5.4
   generally holds for $U_{i}(S)$ for some $i \geq  2$,
   e.g., $i = 2, 3$ for $n= 8$.  we give here the proof for $i =
  2$.   Details that are the same as in the proof of Proposition
8.5.3
  or  Theorem 8.5.4 are skipped.  Note that the state space
   $\mathbf{Q}^{2n}$ consists of approximately 268 million
 28-bit
   words for $n = 14$, but $\mathrm{Orb}_{H}S$ that is expected
to be the only
   attractor consists of only $42$ words (oo is a fixed point
 but  not an attractor).   The following proposition shows that
the
   basin for the attractor $\mathrm{Orb}_{H}S$ contains $U_{2}S$. 
 The  proof is tedious, so that the reader may skip it.\\

  \textbf{Proposition 8.5.6}  If $n \equiv  2$ mod 6, then
  $H(U_{2}S) \subseteq  U_{2}S$ and $\omega  _{H}(U_{2}S) = S$.
   \begin{proof} We prove for $n = 8$ and $r = 2$.

     (i) Let $x = \{ j,k\} ^{-}a$ for $j, k \in 
\mathbf{N}\prime$.  If $(H^{\# }(j^{-
}a))_{2} = 1$
 and $(H^{\# }(k^{-}a))_{2} = 1$, then $Hx = r^{-}(Ha) \in 
U_{1}S$.  If $(H^{\# }(j^{-
}a))_{2} = -1$
 and $(H^{\# }(k^{-}a))_{2} = -1$, then $Hx = 6^{-}(Ha) \in  S$.
  If $(H^{\# }(j^{-}a))_{2} = 1$ and $(H^{\# }(k^{-}a))_{2} =
-1$, then $Hx = Ha$.

     Let  $x = \{ j,k\} ^{-}(4^{-}a)$ for $j, k \in 
\mathbf{N}\prime$.
  If $(H^{\# }(j^{-}(4^{-}a)))_{2} \ne  0$ and
$(H^{\# }(k^{-}(4^{-}a)))_{2} \ne  0$, then $Hx = H(4^{-}a)$. 
 If $(H^{\# }(j^{-}(4^{-}a)))_{2} = 0$ and $(H^{\#
}(k^{-}(4^{-}a)))_{2} \ne  0$,
 then $Hx = 2^{-}H(4^{-}a) \in  U_{1}S$.  If $(H^{\#
}(j^{-}(4^{-}a)))_{2} = 0$ and
$(H^{\# }(k^{-}(4^{-}a)))_{2} = 0$, then
$Hx = \{ 2,6\} ^{-}H(4^{-}a) = 10^{-}a \in  U_{1}S$.

     Let $x = \{ j,k \} ^{-}(15^{-}a)$ for $j,k \in 
\mathbf{N}\prime$.  If $(H^{\#
}(j^{-}(15^{-}a)))_{2} \ne  0$ and
 $(H^{\# }(k^{-}(15^{-}a)))_{2} \ne  0$, then $Hx = Ha$.  If
$(H^{\# }(j^{-}(15^{-
}a)))_{2} = 0$ and $(H^{\# }(k^{-}(15^{-}a)))_{2} \ne  0$, then
$Hx = 6^{-}H(15^{-}a)
\in  U_{1}S$.  If $(H^{\# }(j^{-}(15^{-}a)))_{2} = 0$ and $(H^{\#
}(k^{-}(15^{-}a)))_{2}
= 0$,  then $Hx = \{ 2,6\} ^{-}H(15^{-}a) = \tau
(11^{-}(15^{-}a)) \in U_{1}S$.

     Consequently, if $x = \{ j,k\} ^{-}q$, for some $j,k \in 
\mathbf{N}\prime$ and
$q \in  S$, then $Hx \in  U_{2}S$ and $\mathbf{\omega }_{H}x
\subseteq  S$ by Theorem
8.5.4.

     (ii) Let $x = j^{-}a$ for $j \in  \mathbf{N}$.  Let $(H^{\#
}(j^{-}a))_{2} = -1$. 
Then
 \begin{eqnarray*}
     Hx &=& \{ 6,8+j\} ^{-}(Ha) = \{ 6,8+j\} ^{-}\tau
^{2}(4^{-}a)\\ 
          &=& \tau ^{2}(8+(j-2)\% 8)a \in  U_{1}S.
\end{eqnarray*}
  Let $(H^{\# }(j^{-}a))_{2} = 1$.  Then
\begin{eqnarray*}
     Hx &=& \{ 2,8+j\} ^{-}(Ha) = \{ 2,8+j\} ^{-}\tau
^{2}(4^{-}a)\\
     &=& \tau ^{2}\{ 4,8,8+(j-2)\% 8\} ^{-}a 
= \tau ^{2}\tau ^{-1}\{ 11,8+(j-1)\% 8\} ^{-}(15^{-}a) \in 
U_{2}S,
\end{eqnarray*} 
and $\{ 11,8+(j-1)\% 8\} ^{-}(15^{-}a)$ is in the case (i).  Let
$j = 2$.  Then
 $(H^{\# }(j^{-}x))_{2} = -(H^{\# }(x))_{2} = 0$ and $(H^{\#
}(j^{-}x))_{6} = 2$, so that
 $Hx = \{ 2,10\} ^{-}(Ha) = \{ 2,8+j\} ^{-}(Ha)$ as above.

     Let $x = j^{-}(4^{-}a)$ for $j \in  \mathbf{N}$.  Let
$(H^{\# }(j^{-}(4^{-}a)))_{2}
= 0$.  Then 
$Hx = \{ 2,8+j\} ^{-}H(4^{-}a) = \{ 2,8+j\} ^{-}\tau (15^{-}a)
= \tau \{ 1,8+(j-1)\% 8,15\} ^{-}a = \tau ^{2}\{ 10,8+(j-2)\% 8\}
^{-}(4^{-}a) \in 
U_{2}S$,
 and $\{ 10,8+(j-2)\% 8\} ^{-}(4^{-}a)$ is in the case (i).  If
$j \ne  2$ and  $(H^{\# }(j^{-
}(4^{-}a)))_{2} \ne  0$, then 
$Hx = (8+j)^{-}H(4^{-}a) \in  U_{1}S$.  Let $j = 2$.  Then,
$(H^{\# }(j^{-}(4^{-
}a)))_{2} = -(H^{\# }(4^{-}a))_{2} = -1$,
 so that $Hx = (8+j)^{-}H(4^{-}a) \in  U_{1}S$.

     Let $x = j^{-}(15^{-}a)$ for $j \in  \mathbf{N}$.  Let
$(H^{\# }(15^{-}a))_{2} =
0$.  Then
 $Hx = \{ 6,8+j\} ^{-}H(15^{-}a) = \{ 6,8+j\} ^{-}\tau ^{2}a
= \tau ^{2}\{ 4,8+(j-2)\% 8\} ^{-}a = \tau ^{2}(8+(j-2)\%
8)^{-}(4^{-}a) \in  U_{1}S$. 
 If $j \ne  2$ and $(H^{\# }(15^{-}a))_{2} \ne  0$, then $Hx =
(8+j)^{-}H(15^{-}a) \in 
U_{1}S$.  Let $j = 2$.  Then, $(H^{\# }(j^{-}(15^{-}a)))_{2} =
-(H^{\# }(15^{-}a))_{2}
= 1$ and $(H^{\# }(j^{-}(15^{-}a)))_{6} = 1$,
 so that $H(j^{-}(15^{-}a)) = 6^{-}H(15^{-}a) = 6^{-}\tau ^{2}a =
\tau ^{2}(4^{-}a) = S$. 
 
     Consequently, $if x = j^{-}q$ for some $j \in  \mathbf{N}$
and $q \in  S$, then
$Hx \in  U_{2}S$ and $\mathbf{\omega }_{H}x \subseteq  S$ by
Theorem 8.5.4. 
 \end{proof}
       
   \section*{8.6  Attractive cycles for $n \equiv  0$ or 4 mod 6} 
   
 \textbf{Example 8.6.1}  By modifying Example 8.5.1, let $H =
   <h^{1}, h^{0}>$,
   \begin{eqnarray*}
       h^{1} &=& p_{1}\cdot S_{n-1}\{ p_{2},..,p_{n/2},\neg
   p_{n/2+1},..,\neg  p_{n},p_{n+1},..,p_{n+n/2-1},
           \neg p_{n+n/2},..,\neg p_{2n-1},p_{2n}\},\\
       h^{0} &=& \neg p_{1}\cdot S_{n}\{ p_{2},..,p_{n/2},\neg
   p_{n/2+1},..,\neg  p_{n},p_{n+1},..,p_{n+n/2-1},
            \neg p_{n+n/2},..,\neg p_{2n-1},p_{2n}\}.
   \end{eqnarray*}

If $a = lo\rho ^{-r}(lo)$, then $(H^{\# }a)_{i} = 4(i-1) -
(n-2(r-1))$ for $1 \leq  i \leq  n$. 
Therefore, if $(H^{\# }a)_{r} = 0$, then $r = (n+6)/6$, so that
$n \equiv  0$ mod 6.  The
following theorem is similarly obtained as Theorem 8.5.4 and
Proposition 8.5.5.  Fig.3
illustrates the periodic firing pattern expressed by the
attractor.\\     

 \textbf{Theorem 8.6.2}  If $n \equiv  0$ mod 6 and $r =
  (n+6)/6$,
   then $\mathrm{Orb}_{H}S$ defined by Definition 8.5.2 is an
attractor of $H$ in    Example 8.6.1, and $\mathrm{Orb}_{H}S$
consists of one
$3n$-cycle if $r$ is  even and two $(3/2)n$-cycles if $r$ is
odd.\\

 $$
 \begin{array}{cccccccccccccccccccccccccccccccccc}
&&\cdot& \cdot& \mid& \mid& \cdot& \cdot& \mid& \cdot& \cdot&
\mid&\mid& \cdot& \cdot& \mid& \mid& \cdot& \cdot& \mid& \cdot&
\cdot&
\mid& \mid& \cdot& \cdot& \mid& \cdot& \cdot& \mid& \mid&
\cdot&&\\
&&\mid& \cdot& \cdot& \mid& \cdot& \cdot& \mid& \mid& \cdot&
\cdot&\mid& \mid& \cdot& \cdot& \mid& \cdot& \cdot& \mid& \mid&
\cdot&
\cdot& \mid& \cdot& \cdot& \mid& \mid& \cdot& \cdot& \mid&
\mid&&\\
&&\mid& \cdot& \cdot& \mid& \mid& \cdot& \cdot& \mid& \mid&
\cdot&
\cdot& \mid& \cdot& \cdot& \mid& \mid& \cdot& \cdot& \mid& \cdot&
\cdot& \mid& \mid& \cdot& \cdot& \mid& \mid& \cdot& \cdot&
\mid&&\\
&&\mid& \mid& \cdot& \cdot& \mid& \mid& \cdot& \cdot& \mid&
\cdot& 
\cdot& \mid& \mid& \cdot& \cdot& \mid& \cdot& \cdot& \mid& \mid&
\cdot& \cdot& \mid& \mid& \cdot& \cdot& \mid& \cdot& \cdot&
\mid&&\\
&&\cdot& \mid& \mid& \cdot& \cdot& \mid& \cdot& \cdot& \mid&
\mid&
\cdot& \cdot& \mid& \cdot& \cdot& \mid& \mid& \cdot& \cdot& \mid&
\mid& \cdot& \cdot& \mid& \cdot& \cdot& \mid& \mid& \cdot&
\cdot&&\\
&&\cdot& \cdot& \mid& \cdot& \cdot& \mid& \mid& \cdot& \cdot&
\mid&\cdot& \cdot& \mid& \mid& \cdot& \cdot& \mid& \mid& \cdot&
\cdot&
\mid& \cdot& \cdot& \mid& \mid& \cdot& \cdot& \mid& \cdot&
\cdot&&
\end{array}
$$
\textbf{Fig. 3}  Attractor of Example 8.6.1 for  $n = 6$. Any
consecutive 18 columns display
the attractor.\\
   
   \textbf{Example 8.6.3}  By modifying Example 8.5.1, let $H =
   \langle h^{1}, h^{0} \rangle $,
   \begin{eqnarray*}
       h^{1} &=& p_{1}\cdot S_{n-1}\{p_{2},..,p_{n/2},\neg
   p_{n/2+1},..,\neg p_{n},p_{n+1},..,\\
          & &     p_{n+n/2-2},\neg p_{n+n/2-1},..,\neg p_{2n-
   2},p_{2n-1},p_{2n}\},\\
       h^{0} &=& \neg p_{1}\cdot S_{n}\{p_{2},..,p_{n/2},\neg
   p_{n/2+1},..,\neg p_{n},p_{n+1},..,\\
          & &     p_{n+n/2-2},\neg p_{n+n/2-1},..,\neg p_{2n-
   2},p_{2n-1},p_{2n}\}.
   \end{eqnarray*}
 
If $a = lo\rho ^{-r}(lo)$, then $(H^{\# }a)_{i} = 4(i-1) -
(n-2(r-2))$ for $1 \leq  i \leq 
n/2$.  Therefore, if $(H^{\# }a)_{r} = 0$. then $r = (n+8)/6$, so
that $n \equiv  4$ mod
6.  The following theorem is similarly obtained as Theorem 8.5.4
and Proposition 8.5.5.\\

\textbf{Theorem 8.6.4}  If $n \equiv  4$ mod 6 and $r = 
(n+8)/6$,  then
$\mathrm{Orb}_{H}S$ defined by Definition 8.5.2 is an
attractor of $H$ in Example 8.6.3, and $\mathrm{Orb}_{H}S$
consists of one
$3n$-cycle if $r$ is  even
and two $(3/2)n$-cycles if $r$ is odd.\\
   
\section*{8.7  Attractive cycles for odd $n$}  In this section,
$n$ is odd, $l = 1^{(n-1)/2}$,
and $o = 0^{(n+1)/2}$.\\

\textbf{Example 8.7.1}  By letting $A_{1j} = B_{1j} = -1$ for
   every $j \leq  (n-1)/2$, $A_{1(n+1)/2} = B_{1(n+1)/2}  = 0$,
   and $A_{1j} = B_{1j} = 1$ for every $j \geq  (n+2)/2$ in 
(8.1.2), and making $F$
circular, we obtain $F = \langle f \rangle $,
   \begin{eqnarray*}
       f &=& p_{1}\cdot p_{n+1}\cdot p_{2n+1}\cdot 
S_{n- 3}\{p_{2},..,p_{(n-1)/2},\neg p_{(n+3)/2},..,\neg p_{n},\\  
                  & & p_{n+2},..,p_{n+(n-3)/2},\neg
   p_{n+(n+1)/2},..,\neg p_{2n-1},p_{2n}\}\\
         & & \vee  p_{1}\cdot (p_{n+1}\vee p_{2n+1})\cdot 
S_{n- 2}\{ p_{2},..,p_{(n-1)/2},\neg p_{(n+3)/2},..,\neg p_{n},\\ 
              & & p_{n+2},..,p_{n+(n-3)/2},\neg
   p_{n+(n+1)/2},..,\neg p_{2n-1},p_{2n}\}\\
         & & \vee  p_{1}\cdot S_{n-1}\{ p_{2},..,p_{(n-1)/2},\neg
   p_{(n+3)/2},..,\neg p_{n},\\
         & &    p_{n+2},..,p_{n+(n-3)/2},\neg
   p_{n+(n+1)/2},..,\neg p_{2n-1},p_{2n}\} .
   \end{eqnarray*}
   Therefore, $H = \langle h^{1}, h^{0} \rangle$,
   $$
   \begin{array}{lll}
       h^{1} &=& p_{1}\cdot S_{n-2}\{p_{2},..,p_{(n-1)/2},\neg
   p_{(n+3)/2},..,\neg p_{n},p_{n+1},\\                     & & 
   p_{n+2},..,p_{n+(n-1)/2},\neg p_{n+(n+3)/2},..,\neg
  p_{2n}\},\\
       h^{0} &=& \neg p_{1}\cdot S_{n-1}\{\neg p_{2},..,\neg
   p_{(n-1)/2},p_{(n+3)/2},..,p_{n},\neg p_{n+1},\\
          & & \neg p_{n+2},..,\neg
p_{n+(n-1)/2},p_{n+(n+3)/2},..,p_{2n}\}.
   \end{array}
   \eqno  (8.7.1) 
   $$
     
If $a = lo\rho ^{-r}(lo)$, then $(H^{\# }a)_{i} = 4(i-1) -
(n-2r-1)$ for $1 \leq  i \leq  (n-
1)/2$.  Therefore, if $(H^{\# }a)_{r} = 0$, then $r = (n-1)/6$,
so that $n \equiv  1$ mod
6.  The following theorem is similarly obtained as Theorem 8.5.4
and Proposition 8.5.5.\\
  
   \textbf{Definition 8.7.2}  Let $a = lo\rho ^{-r}(lo)$ and
   $$
       S = [\{a,\{n+2r,2n-r\}^{-}a,(2n-r)^{-}a,((n+1)/2)^{-}a\}].
   $$
   
   \textbf{Theorem 8.7.3}  If $n \equiv  1$ mod 6 and $r =
  (n-1)/6$,    then $\mathrm{Orb}_{H}S$ defined by Definition
8.7.2 is an
attractor of $H$ in    Example 8.7.1, and $\mathrm{Orb}_{H}S$
consists of two
$2n$-cycles.   \\

   \textbf{Example 8.7.4} By modifying Example 8.7.1, let $H =
  \langle h^{1}, h^{0} \rangle$ . 
   \begin{eqnarray*}
      h^{1} &=& p_{1}\cdot S_{n-2}\{p_{2},..,p_{(n-1)/2},\neg
   p_{(n+3)/2},..,\neg p_{n},p_{n+1}, \\
          & & \quad p_{n+2},..,p_{n+(n-3)/2},\neg
   p_{n+(n+1)/2},..,\neg p_{2n-1},p_{2n}\},\\
       h^{0} &=& \neg p_{1}\cdot S_{n-1}\{\neg p_{2},..,\neg
   p_{(n-1)/2},p_{(n+3)/2},..,p_{n},\neg p_{n+1},\\              
     & & \quad \neg p_{n+2},..,\neg p_{n+(n-
   3)/2},p_{n+(n+1)/2},..,p_{2n-1},\neg p_{2n}\}.
   \end{eqnarray*}
     
   If $a = lo\rho ^{-r}(lo)$, then $(H^{\# }a)_{i} = 4(i-1) -
(n-2r+1)$ for $1 \leq  i \leq 
(n-1)/2$.  Therefore, if $(H^{\# }a)_{r} = 0$, then $r =
(n+1)/6$, so that $n \equiv  5$
mod 6.  The following theorem is similarly obtained as Theorem
8.5.4 and 8.5.5. \\   
      
   \textbf{Theorem 8.7.5}  If $n \equiv  5$ mod 6 and $r =
  (n+1)/6$, then $\mathrm{Orb}_{H}S$ defined by Definition 8.7.2
is an
attractor of $H$ in Example 8.7.4, and $\mathrm{Orb}_{H}S$
consists of two
$2n$-cycles.\\
     
   \textbf{Example 8.7.6}  By modifying Example 8.7.1, let $H =
   \langle h^{1}, h^{0}\rangle$,
   \begin{eqnarray*}
       h^{1} &=& p_{1}\cdot S_{n-2}\{ p_{2},..,p_{(n-1)/2},\neg
   p_{(n+3)/2},..,\neg p_{n},p_{n+1},\\
          & & \quad   p_{n+2},..,p_{n+(n-3)/2},\neg
   p_{n+(n+1)/2},..,\neg p_{2n-1},p_{2n}\} ,\\
       h^{0} &=& \neg p_{1}\cdot S_{n-1}\{ \neg p_{2},..,\neg
   p_{(n-1)/2},p_{(n+3)/2},..,p_{n},\neg p_{n+1},\\              
       & & \quad \neg p_{n+2},..,\neg p_{n+(n-
   3)/2},p_{n+(n+1)/2},..,p_{2n-1},\neg p_{2n}\} .
   \end{eqnarray*}
 
If $a = lo\rho ^{-r}(lo)$, then $(H^{\# }a)_{i} = 4(i-1) -
(n-(r-1)-(r-2))$ for $1 \leq  i \leq 
(n-1)/2$
and $r \geq  2$.  Therefore, if $(H^{\# }a)_{r} = n-2$, then $r =
(n+3)/6$, so that $n
\equiv  3$ mod 6.  The following theorem is similarly obtained as
Theorem 8.5.4 and
Proposition 8.5.5.\\
   
\textbf{Theorem 8.7.7}  If $n \equiv 3$ mod 6 and $r = (n+3)/6$,
then
$\mathrm{Orb}_{H}S$ defined by Definition 8.7.2 is an attractor
of
$H$ in Example 8.7.6, and $\mathrm{Orb}_{H}S$ consists of six
$(2/3)n$-cycles if   $2r+1$ is divisible by 3 and two $2n$-cycles
otherwise.\\
   
\section*{8.8  A temporary review}
 The present second-order PDNNs that incorporate a spontaneous
 firing rate of 1/3 per unit time at least partially  
incorporate
  temporal summation in postsynaptic potentials, and the firing
   rate of a neuron can reach three times the spontaneous firing
   rate.  Yet the PDNNs are still represented in the classical
   McCulloch and Pitts network.  Attractors in the PDNNs provide
   stable periodic firing patterns that are expressed in terms 
of
   pulses of action potentials and have more variety than those
in
   the PDNNs of spontaneous firing rate 1/2.  Simplifications, 
such
   as putting the firing mechanism in a black box, discrete-time
   modeling, representation of each neuron's activity by binary
   values, and synchronization of firing for all neurons, are
more
   or less inevitable for global analysis, in particular, by
combinatorial methods, 
which reveal the richness and difficulties of the finite-state
dynamical system
 due to non-linearity of threshold transformations.  The present
chapter
   eased the difficulties by further limiting the global
   description of exemplary PDNNs to circular ones, but could not
   get rid of tedious details.  The fundamental limitation of
 the present PDNNs is that they are autonomous, that is, the
stable periodic firing patterns that are represented by
attractors are completely determined by
the efficacy matrices of synaptic connections and the initial
states of the neurons at
time $t = 0$ and $t = 1$.  In autonomous models, if a minimal
attractor consists of more
than one cycle, then there are some fluidity of shifting from one
pattern to another caused by
noise, even with a change in firing rate in some cases.  This
problem may be solved only in
a non-autonomous model with input from   outside the network.
   
\end{document}