Professional Documents
Culture Documents
Among the second group of modelers (by far the diodes. An earlier version was described (HARMON,
most populous) the idea is to explore the large-scale 1959) in which the functions were less accurate and
behavior of m a n y quasi-neuron elements arranged in flexible than they are presently. A detailed circuit
randomly connected nets. Inasmuch as the properties analysis of t h a t version appears in HARMON & WOLFE
of these elements are greatly simplified abstractions (1959). The present version, which retains the basic
of known neural functions, and the network connec- structure of the earlier one, is shown in Fig. 1.
tivity bears no necessary resemblance to biological Five excitatory inputs are integrated by their as-
structure, it seems unlikely t h a t results bearing real sociated 150 K~9 resistors and the 0.02/~F capacitor
physiological significance will emerge. on the base of the first transistor T 1. Depending on
There is reason to confine our present studies to how the input (dendritic) leads are connected to
peripheral mechanisms. Little enough is known of various (axon or receptor) sources, the integrating
function at the transducer levels and the early relay time constant can be varied from 0.46 to 1.2 mscc.
stages; firm knowledge of cortical events is even less Threshold is determined by the relationship between
detailed and secure. The peripheral structures are the potential on the base of T~ (corresponding to the
simpler, are better known anatomically and physio- time course of input signals) and the potential on the
logically, and are more accessible to experimentation emitter of T 1 (depending upon the recent firing history
than are the central structures. Furthermore, it would of the unit). T 1 and T~ together form a monostable
appear useful to understand the mechanisms of the multivibrator whose function is to produce a single
retina and the optic nerve signal encodings, for ex- pulse if threshold is exceeded. T~ and T 4 form an
ample, before attempting to comprehend more central amplifier and low impedance output source such t h a t
events. Certain recent studies (KUFFL]~R, 1953; HUBEL as m a n y as 100 other units m a y be driven directly
& WIESEL, 1959; LETTWN, MATURA~A, McCuLLOCH & if desired. The function of T5 is to invert an other-
1)ITTS, 1959) illustrate the utility of this view. wise excitatory signal. The inverted signal is summed
with the five excitatory inputs to produce inhibition.
The Model
The refractory function is obtained from the time
Most of the models proposed to date have incor-
constant associated with T 1 and T2, whose pulse-
porated the classical axonal properties of temporal
emitting activity m a y be inhibited by a graded change
summation, variable threshold, inhibition, refractory in their virtual threshold. The circuit provides an
recovery and all-or-none output. I t has become appar- absolute refractory period which is approximately
ent, however, t h a t the properties of the entire neuron 1.0 msec long (the output pulse duration) and which
must be simulated. This means t h a t adequate pro- is immediately followed by a decaying exponential
vision must also be made for the graded dendritic recovery to resting threshold having a time constant
functions having longer time constants and decre- of 2.7 mscc. As will be shown, this and the input
mental conduction. time constant m a y be readily modified over a wide
Although a considerable amount of important range to simulate a variety of neural mechanisms.
activity is assignable to these continuous-variable in- The neuromime has a transmission delay t h a t
put functions (as opposed to axonal discrete-variable varies from 0.1 to 1.0 msec depending on input con-
behavior), there is no compelling reason to believe nections and on firing frequency. Because of the
t h a t axon action potentials do not represent the prin- nature of the active electronic components, the unit
cipal language of the peripheral nervous system. We permits unidirectional propagation only, i.e., no anti-
recognize BULLOCK'S (1959) observation that axon dromic stimulation is possible.
spikes are not seen in some neural structures and his Inhibition is a continuous (graded) and essentially
conclusion t h a t they m a y be incidental and super- linear function of the potential applied to the inhibi-
fluous to function. Nevertheless, if we take as a func- tion input. Two quite different kinds of inhibition
tional neural unit the system from dendritic ramifica- have been described for biological preparations. In
tions to axonal termini, it is clear t h a t however diffuse, one, an inhibitory impulse increases resting membrane
graded, and smeared in time the input signals m a y potential, thereby increasing the threshold for ex-
be, the functional output is unique, standardized (to eitatory impulses (B~ocK, CooM~s, & ECCL~S 1952).
a first-order approximation) and is crisply defined. I n the other, inhibitory impulses diminish any local
All-or-none activity assuredly accounts for more than polarization or depolarization which m a y be present,
long-distance transmission; for example, the concomi- tending to restore resting potential (FATT & KATZ,
t a n t refractory phenomena clearly play an important 1953). Thus in the first case inhibition m a y be viewed
role in modifying signals. as an active potentiating process while in the second
Thus, it is useful to consider t h a t there are two it is effectively a passive attenuation. The former
kinds of information processing present. I n one, the process is subtractive, the latter multiplicative. In
intracellular continuous-variable processes are signifi- the present design of the neuromime the first process
cant. I n the other, discrete but variable signals be- is modeled.
tween cells operate to transmit and to modify in- Any number of excitatory or inhibitory inputs are
formation. Both are essential to a complete description possible, of course, since multiple inputs can be con-
of neural events. verged into any combination of the five excitatory
The model 1 we use is electronic, being a circuit of and one inhibitory leads. Each input can have addi-
five transistors and associated resistors, capacitors and tional resistive networks appended to obtain decre-
In a proposal for a systematic nomenclature of devices mental conduction, and auxiliary capacitors m a y be
which simulate biological functions, VAN BERGEIJK (1960) added to obtain long time constant, graded dendritic
suggested the term neuromime for neuron models. We adopt
this designation in subsequent references to the artificial behavior. This leads to latencies which act in addition
neuron. to the simple internal delay mentioned above. Diode
Band 1, :Heft3
Dezember 1961
LEON D. HARMOn: Studies with Artificial Neurons. I 91
isolation between units can be arranged, if required, of the pulse generator, excitation does not grow after
to minimize interference or loading between multiple removal of a stimulus. Therefore, the model is, with
driving units. t h a t exception, passive. A third omission is the pos-
There appears to be some evidence t h a t the trans- sibility of making a given input lead excitatory at
mission properties of synapses in the CNS v a r y with one time and inhibitory at another; thus, there is
use (EcoLES & MCI~TYRE, 1953). Although we have no equivalent here to the presumed functional change
developed models for obtaining variable synaptic im- of presynaptic fibers dependent on post-synaptic
pedance, they are not being employed in present membrane potential (FATT, 1954; GRUNDFEST, 1959).
experiments and will not be described here. We do We shall show, however, t h a t despite these deficiencies,
not introduce them for two reasons: first, the evidence the model is sufficiently complete to simulate a number
for variable synaptic behavior is incomplete and of biological phenomena.
controversial; second, peripheral (e.g. retinal) structure The neuromime has three primitive input-output
m a y be assumed to be relatively invariant, and there properties. They are one-to-one, where a single spike
seems little necessity at present to invoke synaptic elicits a single spike; one-to-many, where a step func-
changes as functions of experience at this level. tion or steady potential results in a train of output
The primitive functions of the neuromime are sum- pulses (as in peripheral transducers) ; and many-to-one,
marized in Table 1.
Excitatory /.,V~-]=
Table 1
The magnitudes of the voltages employed in the in which spatial or temporal summation occurs. (We
model do not correspond to biological levels. This is ignore for the present a fourth possibility, namely
not important, however, because the voltage ratios those outputs which result from no input, as in spon-
are similar to the natural ratios. For example, the taneous firing.) These properties which include such
6.7 : 1 ratio of spike amplitude to rheobase in the model time-varying characteristics as excitation, threshold
approximates the 7:1 safety-factor described for bio- and inhibition are sufficient to produce a vast number
logical preparations (I-IoDoKI~ 1937, 1948). The tem- of logical functions. These include "and", "or", " n o t " ,
poral relationships closely approximate those in natural "if-and-only-if", addition, subtraction and division, as
neurons. While it m a y be argued t h a t a rectangular well as more complex operations such as filtering,
output pulse of 1 msec duration having no subnormal differentiation, and counting.
or supernormal phases is' too gross a representation
of the spike potential, our feeling is t h a t the discre- Some properties o[ the model
pancies introduce negligible differences in over-all Some of the typical characteristics of the neuro-
functional (logical) behavior. Should the discrepan- mime are illustrated in this section. These are the
cies turn out to be significant, it would be relatively primitive properties of the circuit shown in Fig. 1 and
easy, as will be shown, to modify the model to provide do not include functions such as accommodation and
a differentiated output or post-output polarizations. adaptation. I t will be shown later how these more
Several natural neural properties are not incor- complex operations can be built up from the primitive
porated in the model. For one, conduction velocity ones.
along input or output leads can be made neither slow Due to component tolerances, there is some spread
nor variable. This means t h a t there is no " e v e n t " (on the order of 10%) in performance between units.
delay other than two simple types of internal latencies The following results show the measured responses
which will be described. However, the neuromime of an average unit.
can also be considered as a section of myelinated Time-intensity trade. For a given pulse input to
axon rather t h a n as a complete cell. If a number a biological neuron, there is a minimum time required
of units are connected serially, then conduction is for a given intensity to elicit a response (A~VANITA_KI,
saltatory; each unit represents a node, while the con- 1938). The behavior of the model closely approximates
nections between them represent rapidly conducting A~VAN~TAKI'S strength-duration results as is shown in
internodes. I n this way, various transmission delays Fig. 2. We see, for example, t h a t a 2.7 V stimulus
m a y be achieved. A second omission is that, with the must be at least 1.0 msec long to cause firing, while
exception of the very rapid dynamic flip-flop action one having an amplitude of 1.68 V must last for a
7*
92 L~oN D. H~MON: Studies with Artificial Neurons. I Kybernetik
minimum of 3.0 msec. The horizontal asymptote of Latency. Latency in the model is conventionally
this curve, roughly 1.5 V, represents the rheobase. defined as the delay of impulse initiation in response
The shape of the curve is a direct consequence of to a stimulus of long duration. Since there is no
the exponential input characteristic. The instanta- internal regenerative process contributing significantly
neous potential E (t) available at the summing input is to firing delay after the removal of a stimulus, latency
E (t) = Era= (1-- ~-tl~) (]) is principally due to the gradual build-up of potential
in the input integrator. Firing occurs as soon as
~5 threshold is exceeded. Hence, the strength-duration
curve of Fig. 2 is also a strength-latency curve. These
results are similar to those described for motoneurons
(F~ANK & FUOI~TES, 1956). The curve is asymptotic
to about 0.5 msec which represents the minimum
firing delay in the pulse generating circuit.
Temporal summation. The time-intensity relation-
ships required for the sum of two subthreshold pulses
to elicit firing are given in Fig. 3. Each point along
the abscissa indicates the elapsed time between the
end of the first pulse and the onset of the second.
Points along the ordinate indicate the minimum pulse
amplitude (equal for both pulses) required to just
elicit a response. This behavior too, is the straight-
forward result of the input integration characteristic,
/'~0 / 2 3 g 5 and it is similar to the synaptic summation described
Time in milliseconds by ECCLES (1946).
Fig. 2. Tinle-intens~ty trade for threshold firing. Since firing occurs just I t is seen that time and intensity in the model
at the termination of the stimulus, the curve also represents the
strengthlatency relationship can be traded over a range of about one volt, which
amounts to approximately 10% of the amplitude of
Time between fwo fu/se~ in msec
a typical output pulse. Summation can be achieved
-2.0 0.~ 0.8 lg 1.6 2.0 gq
over a time ranging from 0 (temporal coincidence)
(o -22
to about 3 msec, for which time the required amplitude
of the first pulse is sufficient to exceed resting thresh-
old. Longer time constants of facilitation may be
readily obtained by introducing additional integrating
elements (R and C) at the input.
Re/raetory recovery. Typical recovery after firing
is shown in curve A of Fig. 4. This is similar to
-3.0 ~ ~-~ recovery of biological neurons after inactivation (Era
L A ~ G : E ~ & G A S S E R , 1937; H O D G K I N & H U X L E Y , 1952a).
-3.2 The neuromime is absolutely refractory for approxi-
Threshold for s in~zle-
O.eTasec pulse mately 1 msec, the duration of the outpu~ pulse; the
Fig. 3. Temporal summation for two 0.8 msee pulses of equal amplitude. relative refractory state, beginning at t = 0 in the
Indicated times are from the termination of the first Imlse to the onset illustration, starts at the cessation of the pulse
of the second. Amplitudes are those required to just elicit firing
(ADI~IAN, 1921). This exponential recovery toward
resting threshold has a time constant of 2.7 msec.
Recovery is effectively complete in about 20 msec
since a rheobasic stimulus applied at this time will
once again elicit firing.
This curve results from the use of one input lead.
In B, three leads, used in parallel, increase the effect
of a stimulus. Thus, the refractory level will appear
reduced while its time constant remains unchanged.
Output pulse. The all-or-none rectangular output
-'~0 / 2 3 z/ 5 6'
pulse has relatively constant amplitude; its width,
T/me in milliseconds however, changes somewhat as a function of firing
Fig. 4. Exponential refractory recovery of artificial neuron. This relatively frequency. This is shown in Fig. 5. This behavior
refractory phase is preceded by the absolutely refractory time (output is a consequence of several interacting circuit para-
pulse duration) which terminated at t =0. Curve A results from the use
of one input lead; the recovery in B is due to more effective excitation meters and was not deliberately designed. I t has a
derived from three input leads used in parallel slight effect on temporal integration (reducing its
efficiency with increasing frequency up to 500 pps I)
where Emax is the amplitude of the pulse stimulus, and but probably negligibly alters the neuromime's be-
k is the time constant of the input integrator (1.2 msec). havior. Diminution of spike amplitude with increasing
Each point on the curve represents the condition frequency has been described by RENSHAW (1942),
E (t) ----Erheobase= const. Thus as the intensity (Emax) KUFFL]~R (1953) and others. If the subsequent effec-
is increased, t must decrease. This is the equivalent tiveness of a spike is determined by its energy (time-
of the familiar relationship amplitude integral), then reduction of pulse-width with
Irheobase~-~/stimulus (1 -- 8-tlk ). 1 pps = pulses per second.
Band 1, Heft
Dezember 19613 LEON ~D.HARMON: Studies with Artificial Neurons. I 93
increasing frequency can have the same end result as Diode isolation 1 is required to prevent C from dis-
amplitude change. Such duration-amplitude inter- charging into the output line. The effect of this
changeability is known to hold, for example, in the circuit is simply to prolong the relative refractory time.
well documented time-intensity trade relationships. For R = 0.5 MY2 and C = 1.O/~F, stable repetitive
D e l a y . This term refers to the time between the firing continues smoothly down to 5 pps. This is
beginning of a driving pulse and the onset of an output shown as dotted extensions of the curves of Fig. 7.
pulse during continuous pulse stimulation. I t is not ]./
latency in the usual sense. Latency, generally measur-
ed for single pulse or for step-function stimulation,
does not involve post-firing refractory phenomena. zo
Delay in the model is a function both of excitatory
processes (input integration) and of refractory reco-
very. Typical behavior is shown in Fig. 6. "~ 03
Curve A depicts the delay as firing frequency is
varied and applied to a single input lead (k ~- 1.2 msec). "~ a8
As a driving frequency of about 250 pps is reached,
the driven unit "drops out", and its firing frequency
falls discontinuously to a lower value. This pheno-
menon is due to some interesting relationships between I00 200 300 ~00 500 600
F/ping frequency in ~.p.s.
the input and refractory time constants which will be Fig. 5. Variation of output pulse width (duration) with firing frequency.
discussed later. Pulse a m p l i t u d e remains constant
Curve B illustrates the behavior for all five inputs I.C
strapped together making k-----0.46 mscc. Here the
driven unit "follows" the input pulses up to its
m a x i m u m rate. In addition, the amount and varia-
bility of the firing delay is reduced. In both cases A
and B delay increases with frequency because pro-
o.8
.~ 0.6
I A
mically related to the stimulus intensity on receptors system, for i n s t a n ~ minute variations in threshold
(HARTLINE,1938 ; GRANIT,1947 ; KATZ,1950) ; in other or excitation.
studies firing frequency has been found to be linearly I n the case of repetitive firing of the model under
dependent on generator potential up to 60 pps d.c. stimulus, the lowest firing frequency is about
(FuORTES, 1959) and up to 300 pps (KATz, 1950). The 50 pps. J u s t below this minimal stimulus, firing ceases
frequency-excitation curves of the neuromime (Fig. 7) abruptly. The effective recovery time thus is about
20 msec. Since the refractory time constant T is
Excile-,,,x~ 2.7 msec, recovery is complete in about 7.5 time con-
In Out
stants. W h a t does this imply about noise in the
system ? W h y is the firing frequency not arbitrarily
low ?
The threshold recovery function, defined as R (t), is
R (t) = R (1 --s--t~'), (2)
where R is the threshold increase at the start of the
relative refractory period. We can calculate R(t) for
different values of t in order to find the extent of
Fig. S. Circuit arrangement to extend low-frequency limit of stable firing. refractory recovery at various times. For example,
Self-inhibition prolongs the refractory period
at 3 v,
R (t) = R (1 __~-3) : .950 R ;
are quasi-logarithmic. The low-frequency sections are
recovery is complete to within 5 % of resting threshold.
essentially linear; characteristic A lies within 3 % of
I f the refractory recovery spans, say 10.5 V (see A,
linearity up to 300 pps, while B lies within this limit
Fig. 4), then the distance to resting threshold at 3 T
up to about 250 pps. When driven b y nonlinear trans-
is 5% X 10.5 V = 525 mV. At this point a signal, an
ducer elements such as photo-resistors, the over-all
uncertainty, or a noise having a magnitude of 525 m V
response of the neuromime is very close to log-linear
(and the proper polarity) superimposed on the recover-
over more than four decades of light intensity change.
ing threshold will bring it to the asymptotic (resting)
/0 t level. A plot of such points, based on a threshold of
I
Volts
RffilO.5
1.5 V and a refractory time constant of 2.7 msec is
/0 7 _~m.+ shown in Fig. 9.
Rather than have an abscissa calibrated in time,
~ me /. it is more revealing to indicate frequency. Thus the
1
33 point discussed above appears at-~--~-E~ m s e c ~
E 105
.~ / 123 pps.
The low-frequency cutoff of the neuromlme
( ~ 5 0 pps) corresponds to a recovery within 6 mV
(0.4%) of resting threshold. Thus, any fluctuation
I0 I Equiv.alen/
noi3e in the " s t e a d y " stimulus, or noise in the circuit will
cause firing at this time if it exceeds 6 inV. If reliable
repetitive firing down to 25 pps is required, then no
more than 4/,V noise m a y be present, a most un-
iO t likely situation.
I Low
.~-freauencj On the average, the measured peak noise signals
i0 o ]cutoff at the input in the absence of a stimulus are typically
/0-~ ~ iOp.p.s 1.5 mV with occasional peaks of 2 or 3 times this
amplitude. The stimulus lead was found to have
0 gO 80 120 160 200 2~0 2000 320 360
Firing frequency in "p.p.s, about i mV peak noise on its "steady" signal. This
Fig. 9. The lower limit of firing frequency is determined by noise in the
establishes the theoretical low-frequency cutoff at
system. During exponential refractory recovery, the difference between between 45 and 50 pps, in good agreement with ob-
instantaneous threshold and resting threshold at any instant can be stated served behavior.
as a voltage. Firing will occur whenever a superimposed signal {referred
to as equivalent noise) exceeds this difference. If time is expressed as This approach m a y have some utility in the in-
frequency, then for any level of equivalent noise there is a corresponding vestigation of internal fluctuations in biological neu-
repetitive firing frequency. See text for detailed explanation
rons. Some experimental and theoretical work has
already been done on threshold variations and internal
Repetitive firing as a function of exponential re- noise (P~cH~R, 1939; FATT & KATZ, 1952; FRISrr-
fractory recovery exemplifies an interesting charac- KOPF & ROSEZ~BLITI~, 1958); the technique described
teristic of dynamic systems which is often neglected, above might be used to amplify such results.
namely, the role played b y noise. Consider, for ex-
ample, a neuron which has an exponential refractory
period having a time constant of 2.7 msec. Theoreti- Complex properties o/the model
cally, resting threshold is approached only asymptoti- I n order to exhibit some of the more complicated
cally and is reached in an infinite time. Practically, functions of neurons, external circuits must be ap-
of course, refractory recovery is measured as complete pended to the neuromime. Such phenomena as ac-
in only a few time constants, generally less than commodation, adaptation, self-sustained discharge and
20 msec. This is true only because of noise in the post-spike polarization m a y be achieved.
B a n d 1, H e f t 8
D c z e m b e r 1961
LEON D. HARMON: Studies with Artificial Neurons. I 95
m e c h a n i s m s h a v e a p p r o p r i a t e t i m e constants, t h e n t h e
t w o p o i n t s of view are identical. - (2)
These two e q u i v a l e n t concepts are s c h e m a t i z e d in V
+
Fig. 10. The t h r e e sets of curves on t h e left i l l u s t r a t e
t h e Hill m o d e l of a c c o m m o d a t i o n where V is t h e e b'
v a r i a b l e e x c i t a t i o n a n d U is t h e r e s u l t a n t threshold.
F i r i n g occurs o n l y if V>U, n o t therefore p r e s e n t for
c o n d i t i o n (1), j u s t b a r e l y occurring in (2), a n d lasting
IL
for t h e i n t e r v a l s h a d e d in (3). (3)
I n t h e curves on t h e right, t h r e s h o l d T is fixed.
F o r firing to occur, e x c i t a t i o n E less i n h i b i t i o n I m u s t
result in a n e t e x c i t a t i o n N which exceeds threshold. Fig. 1O. E q u i v a l e n t concepts of f i r i n g criteria. I n ttIZI,'s m o d e l (left).
excitation V m u s t exceed a c c o m m o d a t e d threshold U for f i r i n g to occur,
A g a i n t h e r e is no response in (1), m o m e n t a r y firing A n a l t e r n a t i v e v i e w (right) a s s u m e s a c o n s t a n t threshold. The criterion
in (2) where N ~ T , a n d s u s t a i n e d firing in (3) as for f i r i n g is t h a t n e t excitation N exceeds threshold T. N e t excitation
is the difference b e t w e e n a n e x c i t a t o r y stimulus E a n d a concurrent
long as N > T. inhibition I
B o t h e x c i t a t i o n a n d i n h i b i t i o n are o b t a i n e d from
a s t i m u l u s b y t h e circuit of Fig. 11. T h e i n h i b i t i o n to cause firing. One i n p u t l e a d was used (k = 1.2 msec),
t i m e c o n s t a n t , d e t e r m i n e d b y t h e choice of R a n d C, a n d t h e t i m e c o n s t a n t of i n h i b i t i o n (analogous to
is m a d e large c o m p a r e d to t h e e x c i t a t i o n t i m e c o n s t a n t . HILL'S 2) was 3.7 msec 1.
A n u m b e r of well-known n e u r a l p h e n o m e n a are RvsHrO~'S curves were a s y m p t o t i c to resting
r e p r o d u c e d b y using this circuit. T h e y are described t h r e s h o l d ; no residual a c c o m m o d a t i o n a p p e a r e d . This
in t h e following sections.
Transient change o/excitability. The " a c c o m m o d a -
t i o n " process d e s c r i b e d in t h e foregoing section can
o Exc/r ,_
be used to s i m u l a t e m a n y of t h e w e l l - d o c u m e n t e d
t r a n s i e n t e x c i t a b i l i t y p h e n o m e n a . This follows d i r e c t l y
from t h e f a c t t h a t a s u b t h r e s h o l d s t i m u l u s will p r o d u c e
an e a r l y a n d fast r e d u c t i o n in t h r e s h o l d as t h e ex-
c i t a t o r y i n p u t i n t e g r a t o r b u i l d s u p p o t e n t i a l . Con-
s e q u e n t l y , t h e a d d i t i o n a l stimulus r e q u i r e d d u r i n g this
p e r i o d to exceed t h r e s h o l d a n d cause firing is reduced.
2
T h r e s h o l d s u b s e q u e n t l y rises (more slowly) to its rest- Fig. 11. The circuit configuration used to obtain s t i m u l u s - d e r i v e d inhibi-
ing v a l u e or b e y o n d as i n h i b i t i o n (accommodation) tion. P a r t of a n y stimulus (excitation) voltage is i n t e g r a t e d a n d applied
to the inhibition i n p u t
supervenes.
One m e t h o d of e x p l o r i n g t r a n s i e n t e x c i t a b i l i t y uses
a s u b t h r e s h o l d c o n s t a n t stimulus. RUSttTON'S (1932) is due to t h e fact t h a t t h e c o n d i t i o n i n g s t i m u l u s was
results are t y p i c a l . H e showed t h a t d u r i n g t h e applica- r e m o v e d j u s t a f t e r t h e t e s t s t i m u l u s was applied. T h e
t i o n of a s u b t h r e s h o l d c o n s t a n t current, t h e a m p l i t u d e curves of Fig. 12 a p p r o a c h a c o n s t a n t i n h i b i t i o n level
of a brief t e s t pulse r e q u i r e d to elicit firing decreases. r e p r e s e n t i n g a h i g h e r - t h a n - r e s t i n g t h r e s h o l d ; t h e con-
F o r t h e p a r t i c u l a r n e r v e tested, this decrease h a d a 1 We adopt the symbol ~t for the inhibition time constant.
s h a r p onset, r e a c h e d its m a x i m u m v a l u e a t a b o u t To calculate ;t, the 53 Ks total resistance of the inhibition
1.0 msec, a n d t h e n slowly subsided t o i n i t i a l conditions input circuit (comprising the 24 Kg2 resistor and the base
in 5 - - 1 0 msec. resistance of Ts) must be included. For C=0.1/~F and
R = 120 KD,
1 Threshold is constant in this view only for prefiring 53- 10-s x 120.10 -~
(resting) processes. I t is considered to be a variable, however, ;t •
173 • 10 -3
with respect to the refractory period. 3.7 msec.
96 L~oN D. HARMON':Studies with Artificial Neurons. I Kybernetik
ditioning stimulus was left on indefinitely in this case. follows, having a peak (threshold maximum) at about
Any asymptote can be chosen simply by modification 0.5 msec for the class of fiber tested, which gradually
of the inhibition network. Similarly the time at which returns to resting threshold within 3 or 4 msec. Similar
peak excitability occurs also is variable, gheobase behavior in the neuromime follows from the mech-
anisms operating for a constant conditioning stimulus
as described in the preceding section. These changes
in threshold have been converted into proportional
excitability and plotted in the curves of Fig. 13. Each
3 kga? point on a curve represents the percent ratio of resting
threshold to instantaneous threshold.
A subthreshold conditioning pulse of 0.5 msec dura-
-~.~ tion was terminated at t = 0 , and a variable amplitude
test pulse 0.5 msee long was used to determine thresh-
old at a number of time intervals. Curve A was derived
from a very weak conditioning stimulus; curves B
and C were taken for increasing amplitudes of con-
ditioning pulse. I n D, the conditioning stimulus ex-
\ ~ ~eobase 2.5V ceeded threshold and the unit fired. This curve then
represents refractory recovery. These results closely
parallel those of ERLANGER & GASSER. Recovery in
o I 2 3 ~ 5 each case is to resting threshold (100% excitability)
"]'/'me in m i l l i s e c o n d s
Fig. 12. T r a n s i e n t change of excitability u p o n the application of a constant
since there is no long-lasting stimulus to maintain
subthreshold stimulus. Ordinate values represent the a m p l i t u d e s required accommodation.
of a 0.5 msec test pulse a t v a r i o u s t i m e s to cause firing. M a x i m u m facili- P u l s e trains. For a stimulus of indefinite duration
tation occurs a t ~ 1.3 msec. E a c h c u r v e represents different levels of
conditioning s t i m u l u s whose level is above threshold but below some critical
value, a burst of output pulses will occur for a time t,
becomes 2.5 V for the configuration shown, due to the during which accommodation has not yet overtaken
concurrent inhibition. excitation (HILL, 1936; KATZ, 1936; HODOKIN, 1948).
Another technique for exploring transient excita- KATZ demonstrated a linear relation between the
bility uses a brief pulse rather than a constant current logarithm of relative excitation (in I/Io) and burst
duration, obtaining a family of such lines for various
20O accommodation time constants 2.
This behavior in the neuromime also follows di-
rectly from the stimulus-derived inhibition described
above. The results shown in Fig. 14 closely approxi-
160 mate those described by KATZ.
The measured average excitation time constant k
was 1.0 msec for the units tested; this value was main-
t
tained for all results described below. The inhibition
"~/20
time constant 2 was adjusted by varying C (Fig. l l)
from 0.5ttF ( 2 = 2 1 m s e c ) to 5 . 0 # F ( 2 = 2 1 0 m s e c ) ,
holding R constant at 200 KD. I t is seen that the burst
J duration is a linear function of In E / E o where E is
f
the stimulus level used to elicit the burst and E 0 is
rheobase. As KATZ pointed out, I-~ILL'Sresults imply
J that the relationship of burst duration to stimulating
r
f- current, threshold, and excitation and accommodation
time constants is
~tla _ ] _I/Is~/~ (3)
/
0 l 2 3 # 5
D'me /n milliseconds and if k/~ ~ 0, as is found in nerves showing repetitive
Fig. 13. T r a n s i e n t change of excitability following a 0.5 msec conditioning response, then
stimulus. Ordinate vahles indicate excitability relative to resting threshold t ~ 2 In I / I o. (4)
(100%). E x c i t a b i l i t y is established b y m e a s u r i n g the a m p l i t u d e of a
second 0.5 msec test pulse r e q u i r e d to elicit firing. Threshold is lowered This is the result obtained from the model and plotted
i m m e d i a t e l y following the conditioning stimulus, passes t h r o u g h rheobase
in less t h a n 0.4 msec, a n d reaches a m a x i m u m a t ~ 0.8 msee, g r a d u a l l y
in Fig. 14.' We expect to find that when I = 610, t = 2 .
r e t u r n i n g to its resting value. Curve A results f r o m a low a m p l i t u d e The points shown in the figure are located at these
stimulus; B a n d C represent increasing levels. I n D the s t i m u l u s w a s expected values. The measurements were made for
suprathreshold, t h u s the t i m e course of excitability follows the r e f r a c t o r y
r e c o v e r y characteristic voltages E rather than for currents I.
Burst duration lengthens as excitation is increased
as a subthreshold conditioning stimulus. This is fol- to a critical level, beyond which sustained firing occurs.
lowed by a test pulse to determine threshold. Typi- This occurs because the inhibition circuit saturates
cally it has been found (E~LANOER & GASSIER, 1937) at a somewhat lower level than does the excitation
that the facilitating effect of the conditioning pulse circuit. Consequently at very high stimulus levels,
is proportional to its strength, diminishing rapidly steady-state net excitation is above threshold, and
after the stimulus is removed. A subnormal phase the burst becomes indefinitely long.
B a n d 1, H e f t 3
D e z e m b e r 1961
LEoN D. HARMON: Studies with Artificial Neurons 9 I 97
Increased duration of burst implies greater num- steady-state the net excitation lies just below thresh-
bers of pulses per burst, since frequency of firing also old. Both excitation and inhibition are exponential
goes up with increased excitation. The discrete steps functions; thus, net excitation
observed in this process are shown in Fig. 15A. E(t) ~ Emax (1--e -tl~) -- Emax (1--e -t/~) (6)
For an inhibition time constant of 5.2 msec there where Emax is the stimulus amplitude, and k and
are only four levels before sustained firing occurs9 As )~ respectively are the excitation and inhibition time
is increased so that accommodation lags more and
constants 9 For the production of bursts, accommoda-
more, greater numbers of levels occur9
'
In Fig. 15B it is seen that for 2~-210 msec, bursts
of more than 1 l0 pulses occur9 Individual steps have 8
not been plotted for these curves beyond the first
few levels. Sample points were taken at the higher 7
levels and smooth curves drawn for simplicity.
For an excitation time constant which is short
relative both to typical interpulse periods and to in- G
hibition time constant, net excitation rises quickly and
2.5 I
~s
)~=~//2 / 8 5 /B5 2/o
2
/.5 // ./ !
0
2 3 q 5 6 7 8310 15 gO
Excitation in Volts
/ Fig. 15A. The nunlber of pulses in a t r a n s i e n t burst increases with ex-
citation a n d with inhibition t i m e constant 2. Above a critical stimulus
level, continuous firing occurs
I
50 I00 150 200 250 300 350 400 #50 120
Bur# dural/on in milliseconds Cont/'nuau~ i ~ - ~
Fig. 14, Relation between multiple-response (burst) d u r a t i o n a n d relative II0
excitation for v a r i o u s t i m e constants of inhibition (accommodation). The i
T is the time at which firing ceases. This occurs I t is implicit in the work of KATZ and in the exponen-
at a definite firing frequency, /c, which is the same tial nature of accommodation, as was shown above.
for all 2 (about 50 pps) as shown in Fig. 16. Hence Confirmation could be useful, since counting the num-
from Eqs. (7) and (8) ber of spikes in a burst may provide an index of
/~ ~ cEmax 8-Tl~. (11) accommodation time constant more easily obtained
sloe; than by the use of conventional experimental techni-
ques. I t should also be noted that for the conditions
discussed, adaptation mirrors accommodation; their
~ Total No. time courses are identical. Hence in this case the
6~
~ 300 of pulses accommodation time constant can be inferred from a
k~ 21 8
~t2 /5 measurement of adaptation.
By varying the inhibition parameters (level and
125 q-G time constant), a wide range of accommodation and
~ 2o0 210 77
adaptation can be exhibited. Without inhibition, no
accommodation and no adaptation occurs; rheobase
stimulation elicits repetitive discharge which is con-
~- zoo t\W\ \ \ stant in frequency. For moderate inhibition, as de-
scribed, rheobase stimulation elicits one or two pulses ;
increased levels of stimulation evoke bursts or repeti-
~--21 q2 85 125 270 tive discharges which exhibit accommodation and
adaptation. This variety of behavior is seen in the
0 /0o 200 300 ~00 500 results described by HODGKIN (1948) for a large class
Time in milliseconds of motor neurons and sensory receptors.
F i g . 16. A d a p t a t i o n d u r i n g a b u r s t for v a r i o u s inhibition t i m e constants.
A step-function stimulus of 7.4 • rheobase causes initial firing r a t e s of Sell-sustained discharge. There are two basic kinds
~ 3 5 0 pps in each case; concurrent inhibition s u p e r v e n i n g a t a rate de- of mechanisms which can give rise to continuous acti-
p e n d e n t on ~ e v e n t u a l l y t e r m i n a t e s each b u r s t a t ~ 5 0 pps, the
lowfrequency cutoff vity (i.e., oscillation) in the absence of external stimuli.
I n one, a closed chain of neurons propagates reverbera-
Substituting (11) into (10), we have tory activity around a loop (LoI~NT~ DE N6, 1949).
N ~ Jt ( c E m a x - - l c ) " The minimum number of units required in the loop
Since c, Emax, and /c are constant, is determined by pulse amplitudes and durations and
N :r 4, approximately. by refractory time constants. Any given unit must re-
8O
cover sufficiently before the next stimulus comes along.
The second class of self-sustained oscillators de-
pends on positive feedback around only one neuron.
This feedback m a y be due to internal mechanisms
-o
GO i ;i or to a feedback loop around the entire unit, as may
occur in recurrent axon collaterals. An output-derived
"~ 50 I potential in the neuromime can easily be integrated
and fed back to form an adequate excitation stimulus
% for self-sustained discharge. In this case, maximum
;/ J firing frequency is limited by the refractory time con-
stant and the available output level.
..o
The two kinds of circuits are shown in Figs. 18
~ 2o f and 19. I n Fig. 18, where curve A is for one excitation
- r Z input lead, a minimum of seven units is required for
oscillation. As more units are added, oscillation con-
tinues, but because transit time through the loop in-
0 50 I00 150 200 250 creases, firing frequency decreases. If all five input
Inh/bl'f/on //me cons/an/ )u
Fig. 17. The total n u m b e r of pulses in a t r a n s i e n t burst is a linear function
leads are strapped together (curve B), the more effec-
of the inhibition t i m e c o n s t a n t for a n y g i v e n stimulus level. A results tive excitation causes firing earlier in the recovery
f r o m a stimulus E where In E/Eo = 2. I n B, In B/E~,= 1.5; in C, In E/Eo = cycle; hence fewer units and higher frequencies are
1.0. T h i s is predictable f r o m the n a t u r e of the t i m e course of the
s t i m u l u s d e r i v e d inhibition. See t e x t for d e t l i l e d analysis possible.
In this type of self-sustained discharge circuit, the
The approximate nature of the relationship arises from upper firing frequency is determined by the transit
the nonlinear firing frequency characteristic, the fact time in the shortest possible loop. There is no limit
that final net excitation is not exactly zero, and the on the lowest firing frequency; this is determined
fact that k is not vanishingly small. For the cases solely by the total number of units in the loop.
considered, the approximation is accurate to within I n the single-unit oscillator of Fig. 191, the firing
5%, hence with this limitation we can write freqqency is independent of C as C is made large;
N~, (12) 1 Note how this circuit differs from that of Fig. 8. In
that is, the total number of pulses in a burst is directly both circuits a potential derived from the output is fed back
proportional to the inhibition time constant. The close- to an input. However, in the case of the oscillator (Fig. 19),
ness of the approximation is evident in the linearity the feedback is positive; an output signal is applied to an
excitatory input. This action is regenerative. In contrast,
of the curves in Fig. 17. the circuit of Fig. 8 utilizes negative feedback since the output,
I t would be interesting to know if this result can applied to the inhibitory input, tends to depress further
be verified experimentally for biological preparations. activity.
B a n d 1, H e f t 3
D e z e m b e r 1961
L~oN D. HARMON: Studies with Artificial Neurons. I 99
a sufficiently large capacitor insures a fully smoothed was originally unity, progresses integrally l, 2, 3, 4 . . . .
o u t p u t waveform. T h a t is, the o u t p u t pulses are This is w h a t one reasonably expects to happen, b u t
integrated to yield a feedback excitation voltage which in fact it does not. W h a t does occur is shown in
is relatively constant, being the peak amplitude of Fig. 20. The ratio of /2//1 has been plotted for con-
the o u t p u t pulse. As C is decreased, the average venience. I t is seen t h a t for decreasing excitation
feedback voltage is diminished, and firing frequency at the second unit, the integral steps expected show
drops off until available excitation does not exceed up, but considerably more complex behavior also
instantaneous threshold. Thus the lowest possible appears. Thus the predicted steps are seen as l : l ,
firing rate for this t y p e of reverberatory circuit is 1 : 2, 1 : 3 . . . 1 : l0 (frequency division has been oh-
fixed, unlike the multiple unit loop circuit. The upper
800
limit of firing frequency, which is determined b y the
peak o u t p u t voltage, is likewise bounded.
Curve A depicts the case for three input leads
strapped together. Oscillation becomes impossible if • 600
more leads are connected in parallel, i.e., if the input
time constant ]c is made smaller t h a n 0.66 msec. There
are two reasons for this. As /c is reduced, latency is
decreased. Hence excitation derived from an o u t p u t #00
pulse occurs earlier, when refractory recovery is less
complete. Secondly, reducing/c means t h a t C is dis-
charged sooner, with the result t h a t the feedback A
~. 200
excitation is drained more rapidly. Curve B results
from using one i n p u t ; here latency is increased, hence
the repetition rate is lower.
A special case o / s u m m a t i o n . During the course of
these experiments an unusual and unsuspected pheno- 0 2 ~ 6 8 /0
between pulses in both N 1 and N 2. This can be seen as indicated by the arrow. The voltage at the base
in Fig. 21. of T 1 in the driven unit falls exponentially to a parti-
The top line shows the output of the driving unit. cular level as the pulse from 371 charges up the input
Seven down-going pulses appear. The bottom row of integrating capacitor. Meanwhile the emitter of T 1
16
(3rd trace) rises from a deep, fully inhibited level,
O~ heading for the resting threshold level at a rate
0.6---- governed by the relative refractory time constant (the
unit is just recovering from having been fired by the
07 .12:3 previous N 1 pulse). At the time that the potential
at the base of T 1 is equal to the potential at the
l~ O5
1:2 I emitter, the firing threshold condition is satisfied, and
an output pulse in N 2 is initiated. This occurs just
before the end of the second N 1 pulse. The base of T 1
reflects this firing by terminating the input pulse
integrating process and by being rapidly pulled in the
positive (upward) direction.
.g At termination of the output pulse, the base of T,
is again free to integrate the next pulse out of N 1.
O.2 I ]3..~e I I I Meanwhile the relative refractory phase, reflected in
the slow rise of potential on the emitter of T1, has
begun again as soon as the N~ output pulse terminates.
Note, however, t h a t as this action repeats, the time
at which hr2 fires becomes progressively later with
respect to the N 1 driving pulse. This follows from the
d~O 1 2 3 it 5 fact t h a t the instantaneous magnitude of the N~
E.Yci/ation in Volis threshold (refractory curve) is not phase-locked to the
Fig. 20. Firing response of one unit being driven by another. "Following"
is precise at high levels of excitation. As the driven unit receives pro-
N 1 driving pulses. Thus a phase slippage can and
gressively smaller stimuli, it begins to drop pulses. The 1:2 and 1:3 steps does occur. Eventually (at the fifth pulse) this slip
result from ordinary temporal summation. Other ratios of firing frequencies accumulates to the point t h a t by the time the driving
stem from another phenomenon, caused by asynchrony between driving
pulses and the refractory period of the driven unit pulse ends (the base of T 1 has reached its m a x i m u m
excitation level), the refractory recovery has not been
pulse respresents the output of the driven unit. Note sufficient to allow firing. Consequently a pulse is
that, including the partial pulse at the right end, only missed.
six appear. One pulse, corresponding to the fifth one Refractory recovery continues during this blank
from N 1 has dropped out. This sequence repeats (not interval, more and more closely approaching resting
threshold. Then the next N 1 pulse comes along. Since
threshold is now quite low, firing occurs relatively
early. Again the base of T 1 is discharged and awaits
the next input signal. Thus the cycle is completed;
this point in time corresponds to the situation at the
second pulse where the tracing of the sequence began.
I t would be interesting to examine biological pre-
parations for the presence of this type of coding.
Auditory volleying, for example, m a y utilize such
mechanisms.