1981]

2.
3.

ON MAXIMIZING FUNCTIONS BY FIBONACCI SEARCH

3hl

N. J. A. Sloane. "The Persistence of a Number." J. Rec. Math. 6 (1973):

97-98.
M. Gardner. "Mathematical Games." Scientific
American 240 (1979):16-24.
*****

ON MAXIMIZING FUNCTIONS BY FIBONACCI SEARCH

Department

of Statistics,

REFAEL HASSIN
Tel Aviv University,
1.

Tel Aviv

69978

INTRODUCTION

The search for a local maximum of a function f(x)

involves a sequence of
function evaluations, i.e.s observations of the value of f(x)
for a fixed value
of x. A sequential search scheme allows us to evaluate the function at different points, one after the other, using information from earlier evaluations to
decide where to locate the next ones. At each stage, the smallest interval in
which a maximum point of the function is known to lie is called the interval
of
uncertainty,
Most of the theoretical search procedures terminate the search when either
the interval of uncertainty is reduced to a specific size or two successive
estimates of the maximum are closer than some predetermined value. However, an
additional termination rule which surprisingly has not received much attention
by theorists exists in most practical search codes, namely the number of function evaluations cannot exceed a predetermined number, which we denote by .N.
A well-known procedure designed for a fixed number of function evaluations
is the so-called Fibonacci search method. This method can be applied whenever
the function is unimodal and the initial interval of undertainty is finite. In
this paper, we propose a two-stage procedure which can be used whenever these
requirements do not hold. In the first stage, the procedure tries to bracket
the maximum point in a finite interval, and in the second it reduces this interval using the Fibonacci search method or a variation of it developed by
Witzgall.
2.

THE BRACKETING ALGORITHM

A function / is unimodal on [a9b] if there exists a <_ x <. b such that f(x)
is strictly increasing for a <_ x < ~x and strictly decreasing for x < x <_ b. It
has been shown (Avriel and Wilde [2], Kiefer [6]) that the Fibonacci search
method guarantees the smallest final interval of uncertainty among all methods
requiring a fixed number of function evaluations. This method and its variations (Avriel and Wilde [3],Beamer and Wilde [4] , Kiefer [6], Oliver and Wilde
[7], Witzgall [10]) use the following idea:
Suppose y and z are two points in [a9b] such that y < z9 and f is unimodal,
then
f(y) < f(z)
implies y ~x_<_b9
f(y) > f(%)
implies a <_ x <_ z9 and
f(y) = / 0 0
implies
y x z .
Thus the property of unimodality makes it possible to obtain, after examining f(y) and f(z)9
a smaller new interval of uncertainty. When it cannot be
said in advance that / is unimodel, a similar idea can be used.
Suppose that f(x1)9
f(x2)9
and f(x3)
are known such that
(1)

xx

< x2

< x3

and

f(x2)

^> max{/(xx), /(x 3 )},

3^8

ON MAXIMIZING FUNCTIONS BY FIBONACCI SEARCH

[Oct.

then a local maximum of / exists somewhere between xx and xs.

Evaluation of
the function at a new point xk in the interval (x19 xs) will reduce the interval of uncertainty and form a new set of three points x{9 x29 xsf satisfying
equation (1):
Suppose x1 < x^ < x2 , then if f(xh)
_> f(x2)
let x{ = x19 x2 = x^9 x3 - x2
and if f(xk)
< f(x2)
let x[ - xh9 x2 = x2, and #3 = x3.
Similarly for x2 < xh
< x39 if f(x^)
> f(x2)
then let x[ = x29 x[ = xk9 x3 = x39 and if f(x^)
< f(x2)
tinen xeu x-\ x-\ 9 x2 == x29 x3 = Xu
When applying quadratic approximation methods, the new point xk is chosen
as the maximum point of a quadratic function which approximates /. The assumption behind this method is that / is nearly quadratic, at least in the neighborhood of its maximum. However, when the number of function evaluations is
fixed in advance, this method may terminate with an interval of uncertainty
which is long relative to the initial one.
The quadratic approximation algorithm of Davies, Swann, and Campey [5] includes a subroutine that finds three equally spaced points satisfying equation
(1). A more general method developed by Rosenbrock [8] can serve as a preparatory step for a quadratic approximation algorithm (Avriel [1]).
We now describe the search for points satisfying equation (1) in a general
form that allows further development of our algorithm. The input data includes
the function /, the number of evaluations N, and a set of positive numbers a^,
i = 3, ..., N.

Bnajck&ting AlgofLvtkm:
Stzp

1.

Evaluate f at two distinct points. Denote these points by x and

Set
x2 so that fix-^) f{x2).
k = 3.

Stup

2.

Evaluate f at xk = xk_1 + ^}i{xli,1 - xk_2) .

If f(xk)
j fixk-i.)i
stop. (A local maximum exists between xk_2
xk.)
If f(xk)
> f{xk_x)9
set k + k + 1.

Stdp

3.

and

If k = N + 1, stop. (The search failed to bracket a local maximum. )

If k <. N9 return to Step 2.

If the algorithm terminates in Step 2, then the function was evaluated

k <_ N times and a local maximum was bracketed between x^_2 and xk.
The interval of uncertainty may now be further reduced by evaluating the function at
N - k new points xk + 19 ..., xN.
Notice that there is already one point, xk_19
in the interval of uncertainty, for which / is known.
3.

REDUCTION OF THE INTERVAL OF UNCERTAINTY

In this section, we propose and analyze alternatives for selecting the increment multipliers ak . Let F0 = F - 1 and Fn = Fn_2 + 4-i> n = 2, 3, ...,
denote the Fibonacci numbers. If either
(2)

X]<_1

= Xk_2

xk_1

= xk

FN _k
+ "
-(Xk
L
N-k+1

Xk_2)

or
^N

(3)

- -

-k

~(xk

"

k-i^

then xk_x is one of the two first evaluations in a Fibonacci search with N - k
+ 1 evaluations, on the interval bounded by xk_2 and xk.,
In this case, xk + l9
..., xN can be chosen as the next points in this Fibonacci search. This choice

1981]

ON-MAXIMIZING FUNCTIONS BY FIBONACCI SEARCH

3^9

guarantees the smallest final interval of uncertainty among all other methods
requiring N - k additional evaluations.
If both (2) and (3) do not holds the next points can be chosen according to
Witzgall's algorithm [10]. This algorithm guarantees the smallest final interval of uncertainty in a fixed number of function evaluations when, for some
reason, the first evaluation took place at some argument other than the two
optimal ones.
We now show how to choose the increment multipliers a^9 i = 3,...,/lf-l,
so that equations (2) or (3), according to our preference, will hold when the
bracketing algorithm terminates after k < N evaluations.
Equation (2) implies that
Fpj _
x

"

k-i

k-z

[ (x k

"p

"

k-i^

(xk-i

"

k-2^

or
F

N-k

+i

k-i

+ 1 = ou + 1.
x

*N-k

k-i

k-2

Denote the value of a^ which satisfies the above equation by ajj,1 , then
(4)

<*<" - % * ^ -

1 = % ^ < 1 .

Equation (2) holds for k < N if and only if ak = a^,1) .

Similarly, equation (3) implies that
F

N-k

FN-k

or

N-k+l

(Xk

^_

(xk-l

k-2^

+ l
l "" xk-2

kx

FN_k

J_
a

k-l

k
(2 )

Denote the value of ak which s a t i s f i e s t h i s equation by a^

, then

a(2) = - ^ - = - ^ - > 1

(5)

Olk

'

N-k-l

Equation (3) holds for k < N if and only if a k = a [ 2 ) .

Let dk = |tffc-tffc-i | * /c = 1, . .., N9 denote the search increments, then
(6)

d2 = \x2 - x |
d

k = akdk-i

= a

and

k ' ak-i

a3|a?2 - a j ,fc= 3, ..., fl.

1)

Denote the search increments by d[

and d(k2) when a^1} and a[2) are chosen,
respectively, for k = 2, ..., N - 1. Then equations (4) and (6) yield
Q )

dl1}

N-k-l
*N-k

N-k

= -=

^N-k

N->+

N-k-l\

* F2 - xi I =
r

*ff-3

+1

\x2 - x I,

N-3

k = 2, ...,/!/- 1.
If the bracketing algorithm terminates after k < N evaluations, then the
maximum is located in an interval of length
I

,(1)

1}
" ^ - 2 I = d\"fc

,M)

+ dkk~^1 i =

N-k-l+FN-k>

TT
F
N-

\xz - a j =

N-k

+ l.

la?2 - a j .

350

ON MAXIMIZING FUNCTIONS BY FIBONACCI SEARCH

[Oct.

This interval is further searched by a Fibonacci search with N - k -f- 1 evaluations (including the one in xk_) which reduces its length by a factor (FN_k + 1 ) ~ 1 .
Consequently, the length of the final interval is

independent of k.

N-3

This length is satisfactorily small in comparison with

X

1*2 F

lI

N-Z

which can be achieved by 717 2 evaluations if / is known to be unimodal with a

maximum between xx and xz
Suppose, however, that the bracketing algorithm terminates after N evaluations without bracketing a local maximum. The total size of the searched interval is
N-i

- x

\x2

fc=2

N-3

In fact, when N is large, this sum approaches (1 + x)\x 2 satisfies x 2 = 1 + x. The cost of obtaining a small final
success is in searching a relatively small interval and
chances that the bracketing algorithm will fail.
This default can be overcome by using ak2>> rather than
F

i(2)
k

N -3

N-k

N-k-l

+l

N-3

N-k

\X2

1 ~

J?

x1| where x = 1.618

interval in case of
thus increasing the
a^

. In this case,

N-3

N - k - l

\X2

k = 2 , -...,'N

- 1.

The sequence dk
increases with k so that a larger interval is scanned,
and it is less likely that the bracketing algorithm will fail. In practice,
some of the last increments dk*
may be replaced by smaller increments, possibly by d(k1) .
4.

SUMMARY

We suggest a two-stage search procedure for maximizing functions by a fixed

number of evaluations. The first stage is a quite standard bracketing subroutine and the second is either the regular Fibonacci search or the modified
method of Witzgall. During the first stage, the kth evaluation is at the point
xk calculated from xk = %k_1 + aj<(xj<_-1 ~ xk-z^'
We suggest three alternatives:
A.

Let ak = a^1? <_ 1. In case of success, proceed by Fibonacci search to

obtain a small final interval.

B.

Let ak = a^ 2) > 1.

In case of success, proceed by Fibonacci search.

The chances for success are better than in case A, but the final interval is longer.
C.

Let ak > 0 be arbitrary and proceed by WitzgallTs method.

We note that different alternatives may be chosen for different values of k.

1981]

2,3 SEQUENCE AS BINARY MIXTURE

351

REFERENCES
1. M. Avriel. Non-Linear
Programming: Analysis
and Methods. Englewood Cliffs,
N.J.: Prentice-Hall, Inc., 1976.
2. M. Avriel & D. J. Wilde. "Optimality Proof for the Symmetric Fibonacci
Search Technique." The Fibonacci
Quarterly
4 (1966):265-69.
3. M. Avriel & D. J. Wilde. "Golden Block Search for the Maximum of Unimodal Functions." Management Science
14 (1968):307-19.
4. J. H. Beamer & D. J. Wilde. "Minimax Optimization of Unimodal Functions
by Variable Block Search." Management Science
16 (1970):529-41.
5. M. J. Box, D. Davies, & W. H. Swann. Non-Linear
Optimization
Techniques.
Edinburgh: Oliver & Boyd, 1969.
6. J. Kiefer. "Sequential Minimax Search for a Maximum." Proc. Amer. Math.
Soc. 4 (1963):502-06.
7. L. T. Oliver & D. J. Wilde. "Symmetrical Sequential Minimax Search for a
Maximum." The Fibonacci
Quarterly
2 (1964):169-75.
8. H. H. Rosenbrock. "An Automatic Method for Finding the Greatest and Least
Value of a Function." Computer J. 3 (1960):175-84.
9. D. J. Wilde. Optimum Seeking Methods.
Englewood Cliffs, N.J.: PrenticeHall, Inc., 1964.
10. C. Witzgall. "Fibonacci Search with Arbitrary First Evaluation." The Fibonacci Quarterly
10 (1972):113-34.
?*####

2,3 SEQUENCE AS BINARY MIXTURE

Exxon Research

DONALD J. MINTZ
and Engineering
Company,

Linden,

NJ 07036

The integer sequence formed by multiplying integral powers of the numbers

2 and 3 can be viewed as a binary sequence. The numbers 2 and 3 are the component factors of this binary. This paper explores the combination of these
components to form the properties of the integers in the binary. Properties
considered are: value, ordinality (position in the sequence), and exponents of
the factors of each integer in the binary sequence.
Questions related to the properties of integer sequences with irregular
nth differences are notoriously hard to answer [1]. The integers in the 2,3
sequence produce irregular nth differences. These integers can be related to
the graphs constructed in the study of 2,3 trees [2, 3]. It is shown in this
paper that the ordinality property of the integers in the 2,3 sequence can be
derived from the irrational number log 3/log 2. This number also finds application in the derivation of a discontinuous spatial pattern found in the study
of fractal dimension [4].
In Table 1, the first fifty-one numbers in the 2,3 sequence are listed according to their ordinality with respect to value. Since the 2,3 sequence consists of numbers which are integral multiples of the factors 2 and 3, it is
convenient to plot the information in Table 1 in the form of a two-dimensional
lattice, as shown in Figure 1. In this figure, the horizontal axis represents
integral powers of 2 and the vertical axis represents integral powers of 3.
The ordinality of each number is printed next to its corresponding lattice
point. For example, the number 2592 = 2 5 3 4 and 0rd(253Lf) = 50; therefore, at
the coordinates 2 5 , 3 4 , the number "50" is printed.