Some basic topics in analysis

Stephen Semmes Rice University

Preface

These informal notes deal with some possible topics for a second course in analysis. In particular, the reader is assumed to be familiar with metric spaces, sequences and series, and continuous functions. Some topics may be mentioned in a first course, with some review or elaboration here.

Of course, there are many textbooks in analysis, a few of which are mentioned in the bibliography. The aim here is to complement these textbooks, while trying to look ahead a bit to more advanced courses.

Although some basic notions are used frequently throughout the text, there is a fair amout of independence between the various sections and chapters. Thus the reader may wish to focus more on some parts, at least initially.

Some aspects of history related to topics like those considered here may be found in [3, 15, 17, 18, 19, 20, 32, 33, 41, 42, 45, 66, 67, 68, 71, 72, 73, 74, 75, 76, 77, 79, 80, 96, 113, 152, 164], for instance. Some songs related to some topics like those considered here may be found in [118, 119, 120]. Some remarks concerning the clarity of explanations in mathematics may be found in [78].

Contents

1 Some mappings, metrics, and norms 1 1.1 Lipschitz mappings . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Lipschitz conditions on R . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Norms on R^{n} . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 Norms on C^{n} . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.5 Some basic inequalities . . . . . . . . . . . . . . . . . . . . . . . . 6 1.6 Functions with finite support . . . . . . . . . . . . . . . . . . . . 8 1.7 Cauchy sequences and completeness . . . . . . . . . . . . . . . . 10 1.8 Pointwise and uniform convergence . . . . . . . . . . . . . . . . . 11 1.9 Bounded sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.10 Some remarks and examples . . . . . . . . . . . . . . . . . . . . . 14 1.11 Bounded functions . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.12 Completeness of B(X, Y) . . . . . . . . . . . . . . . . . . . . . . 17 1.13 More on bounded functions . . . . . . . . . . . . . . . . . . . . . 18 1.14 Continuous functions . . . . . . . . . . . . . . . . . . . . . . . . . 19 1.15 Continuous functions on [0, 1] . . . . . . . . . . . . . . . . . . . . 20

2 Basic ℓ^{1} and ℓ^{2} spaces 23 2.1 Infinite series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.2 Basic ℓ^{1} spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.3 Basic ℓ^{2} spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.4 Completeness of ℓ^{1}, ℓ^{2} . . . . . . . . . . . . . . . . . . . . . . . . 29 2.5 Vanishing at infinity . . . . . . . . . . . . . . . . . . . . . . . . . 30

3 Some more metric spaces 33 3.1 Functions on intervals . . . . . . . . . . . . . . . . . . . . . . . . 33 3.2 The square norm . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.3 Riemann–Stieltjes integrals . . . . . . . . . . . . . . . . . . . . . 36 3.4 Riemann–Stieltjes integrals and seminorms . . . . . . . . . . . . 37 3.5 Square seminorms . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.6 Some more monotone functions . . . . . . . . . . . . . . . . . . . 41 3.7 Compact support . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.8 Functions on R . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.9 Dense sets and Weierstrass’ theorem . . . . . . . . . . . . . . . . 46 3.10 Functions on sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 3.11 The Stone–Weierstrass theorem . . . . . . . . . . . . . . . . . . . 48 3.12 Algebras of bounded functions . . . . . . . . . . . . . . . . . . . 49 3.13 Some remarks about closed subalgebras . . . . . . . . . . . . . . 50 3.14 Local compactness . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.15 Another vanishing condition . . . . . . . . . . . . . . . . . . . . . 52

4 Compactness and completeness 55 4.1 Diameters of sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 4.2 Totally bounded sets . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.3 Separable metric spaces . . . . . . . . . . . . . . . . . . . . . . . 58 4.4 Lindelöf’s theorem . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.5 The limit point property . . . . . . . . . . . . . . . . . . . . . . . 61 4.6 Sequential compactness . . . . . . . . . . . . . . . . . . . . . . . 63 4.7 A criterion for compactness . . . . . . . . . . . . . . . . . . . . . 64 4.8 The Baire category theorem . . . . . . . . . . . . . . . . . . . . . 66 4.9 The interior of a set . . . . . . . . . . . . . . . . . . . . . . . . . 68

5 Equicontinuity and sequences of functions 69 5.1 Pointwise convergent subsequences . . . . . . . . . . . . . . . . . 69 5.2 Equicontinuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 5.3 Uniformly Cauchy sequences . . . . . . . . . . . . . . . . . . . . 72 5.4 Equicontinuity and uniform convergence . . . . . . . . . . . . . . 74 5.5 Equicontinuity and Cauchy sequences . . . . . . . . . . . . . . . 75 5.6 Equicontinuity and subsequences . . . . . . . . . . . . . . . . . . 76 5.7 Pointwise and uniform boundedness . . . . . . . . . . . . . . . . 77 5.8 Total boundedness in B(X, Y) . . . . . . . . . . . . . . . . . . . . 79 5.9 Equicontinuity and total boundedness . . . . . . . . . . . . . . . 80 5.10 Equiconvergence of limits . . . . . . . . . . . . . . . . . . . . . . 82 5.11 Equiconvergence and differentiability . . . . . . . . . . . . . . . . 83

6 More on sums and norms 85 6.1 Weierstrass’ criterion . . . . . . . . . . . . . . . . . . . . . . . . . 85 6.2 Radius of convergence . . . . . . . . . . . . . . . . . . . . . . . . 86 6.3 Termwise differentiation . . . . . . . . . . . . . . . . . . . . . . . 88 6.4 Cauchy products . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 6.5 Rearrangements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 6.6 Comparing norms on R^{n}, C^{n} . . . . . . . . . . . . . . . . . . . . 93 6.7 Another comparison . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.8 Inner products on R^{n}, C^{n} . . . . . . . . . . . . . . . . . . . . . . 96 6.9 Sums and inner products . . . . . . . . . . . . . . . . . . . . . . 97 6.10 Integral inner products . . . . . . . . . . . . . . . . . . . . . . . . 99 6.11 Some remarks about n-dimensional volume . . . . . . . . . . . . 101 6.12 Volumes and Lipschitz mappings . . . . . . . . . . . . . . . . . . 102 6.13 Bounded vector-valued functions . . . . . . . . . . . . . . . . . . 103 7 Matrix norms and Lipschitz conditions 106

7.1 Real and complex matrices . . . . . . . . . . . . . . . . . . . . . 106 7.2 Matrices and linear mappings . . . . . . . . . . . . . . . . . . . . 107 7.3 Some related estimates . . . . . . . . . . . . . . . . . . . . . . . . 109 7.4 Spaces of linear mappings . . . . . . . . . . . . . . . . . . . . . . 110 7.5 Operator norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 7.6 Determinants and volume . . . . . . . . . . . . . . . . . . . . . . 113 7.7 Lipschitz constants . . . . . . . . . . . . . . . . . . . . . . . . . . 114 7.8 Compositions and isometries . . . . . . . . . . . . . . . . . . . . 116 7.9 Bilipschitz embeddings . . . . . . . . . . . . . . . . . . . . . . . . 117 7.10 Linear mappings and seminorms . . . . . . . . . . . . . . . . . . 119 7.11 Small perturbations . . . . . . . . . . . . . . . . . . . . . . . . . 120 7.12 The contraction mapping theorem . . . . . . . . . . . . . . . . . 121 7.13 A localized condition . . . . . . . . . . . . . . . . . . . . . . . . . 123 7.14 Open mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

8 Some topics related to differentiability 126 8.1 An integral triangle inequality . . . . . . . . . . . . . . . . . . . . 126 8.2 A basic Lipschitz estimate . . . . . . . . . . . . . . . . . . . . . . 127 8.3 Some partial Lipschitz conditions . . . . . . . . . . . . . . . . . . 128 8.4 Partial derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . 130 8.5 Using the mean value theorem . . . . . . . . . . . . . . . . . . . 131 8.6 Directional derivatives . . . . . . . . . . . . . . . . . . . . . . . . 131 8.7 Differentiable mappings . . . . . . . . . . . . . . . . . . . . . . . 133 8.8 Pointwise Lipschitz conditions . . . . . . . . . . . . . . . . . . . . 134 8.9 The chain rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 8.10 Continuous differentiability . . . . . . . . . . . . . . . . . . . . . 138 8.11 Another basic Lipschitz estimate . . . . . . . . . . . . . . . . . . 140 8.12 Some remarks about connectedness . . . . . . . . . . . . . . . . . 141 8.13 Locally constant mappings . . . . . . . . . . . . . . . . . . . . . . 142 8.14 Some local Lipschitz conditions . . . . . . . . . . . . . . . . . . . 144

9 More on differentiable mappings 146 9.1 Continuously-differentiable mappings . . . . . . . . . . . . . . . . 146 9.2 Invertible linear mappings . . . . . . . . . . . . . . . . . . . . . . 147 9.3 The inverse function theorem . . . . . . . . . . . . . . . . . . . . 149 9.4 Proving the inverse function theorem . . . . . . . . . . . . . . . . 150 9.5 Some remarks about R^{n+m} . . . . . . . . . . . . . . . . . . . . . 152 9.6 The implicit function theorem . . . . . . . . . . . . . . . . . . . . 153 9.7 Local embeddings . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 9.8 Ranks of linear mappings . . . . . . . . . . . . . . . . . . . . . . 156 9.9 Some remarks about determinants . . . . . . . . . . . . . . . . . 158 9.10 Some remarks about linear subspaces . . . . . . . . . . . . . . . . 159 9.11 Complementary linear subspaces . . . . . . . . . . . . . . . . . . 160 9.12 The rank theorem . . . . . . . . . . . . . . . . . . . . . . . . . . 161 9.13 Proving this part . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

9.14 Some more properties of F ∘ H . . . . . . . . . . . . . . . . . . . 164

10 Product spaces and related matters 167 10.1 Products of metric spaces . . . . . . . . . . . . . . . . . . . . . . 167 10.2 Open and closed sets . . . . . . . . . . . . . . . . . . . . . . . . . 168 10.3 Sequences and bounded sets . . . . . . . . . . . . . . . . . . . . . 170 10.4 Products of compact sets . . . . . . . . . . . . . . . . . . . . . . 172 10.5 Some mappings on products . . . . . . . . . . . . . . . . . . . . . 173 10.6 Uniform continuity . . . . . . . . . . . . . . . . . . . . . . . . . . 175 10.7 Continuity and integration . . . . . . . . . . . . . . . . . . . . . . 176 10.8 Iterated integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 10.9 Partitions of intervals . . . . . . . . . . . . . . . . . . . . . . . . 178 10.10Partitions and products . . . . . . . . . . . . . . . . . . . . . . . 180 10.11Partitions and integrals . . . . . . . . . . . . . . . . . . . . . . . 182 10.12A simpler approximation . . . . . . . . . . . . . . . . . . . . . . . 183 10.13Another approximation . . . . . . . . . . . . . . . . . . . . . . . 185 10.14Partitions of unity . . . . . . . . . . . . . . . . . . . . . . . . . . 186 10.15Another approximation argument . . . . . . . . . . . . . . . . . . 188 10.16Some continuous functions . . . . . . . . . . . . . . . . . . . . . . 190 10.17Products and reciprocals . . . . . . . . . . . . . . . . . . . . . . . 192 10.18Graphs of mappings . . . . . . . . . . . . . . . . . . . . . . . . . 192 10.19Semicontinuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 10.20Homeomorphisms between metric spaces . . . . . . . . . . . . . . 196 10.21Graphs and homeomorphisms . . . . . . . . . . . . . . . . . . . . 198

11 Summable functions 200 11.1 Extended real numbers . . . . . . . . . . . . . . . . . . . . . . . . 200 11.2 Nonnegative sums . . . . . . . . . . . . . . . . . . . . . . . . . . 201 11.3 Compositions and subsets . . . . . . . . . . . . . . . . . . . . . . 202 11.4 Nonnegative summable functions . . . . . . . . . . . . . . . . . . 204 11.5 Some linearity properties . . . . . . . . . . . . . . . . . . . . . . 205 11.6 ℓ^{1} Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 11.7 Real-valued summable functions . . . . . . . . . . . . . . . . . . 209 11.8 Complex-valued summable functions . . . . . . . . . . . . . . . . 210 11.9 Some properties of the sum . . . . . . . . . . . . . . . . . . . . . 211 11.10Generalized convergence . . . . . . . . . . . . . . . . . . . . . . . 213 11.11Compositions and sums . . . . . . . . . . . . . . . . . . . . . . . 215 11.12Completeness of ℓ^{1} spaces . . . . . . . . . . . . . . . . . . . . . . 216 11.13Monotone convergence . . . . . . . . . . . . . . . . . . . . . . . . 218 11.14Dominated convergence . . . . . . . . . . . . . . . . . . . . . . . 219 11.15Nonnegative sums of sums . . . . . . . . . . . . . . . . . . . . . . 220 11.16Real and complex sums . . . . . . . . . . . . . . . . . . . . . . . 222 11.17Square-summable functions . . . . . . . . . . . . . . . . . . . . . 224 11.18ℓ^{2} Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 11.19Completeness of ℓ^{2} spaces . . . . . . . . . . . . . . . . . . . . . . 227 11.20Inner products on ℓ^{2} spaces . . . . . . . . . . . . . . . . . . . . . 228 12 Some additional topics 231

12.1 Lebesgue measure and integration . . . . . . . . . . . . . . . . . 231 12.2 Banach spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 12.3 Hilbert spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 12.4 Infinite series in Banach spaces . . . . . . . . . . . . . . . . . . . 235 12.5 Bounded linear mappings . . . . . . . . . . . . . . . . . . . . . . 237

A Some more on mappings, metrics, and norms 240 A.1 A nice inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . 240 A.2 Some more Lipschitz conditions . . . . . . . . . . . . . . . . . . . 241 A.3 Another nice inequality . . . . . . . . . . . . . . . . . . . . . . . 242 A.4 Some functions on [0, 1] . . . . . . . . . . . . . . . . . . . . . . . 243 A.5 Some Cauchy sequences . . . . . . . . . . . . . . . . . . . . . . . 245 A.6 Norms and convexity . . . . . . . . . . . . . . . . . . . . . . . . . 246 A.7 Path-connected sets . . . . . . . . . . . . . . . . . . . . . . . . . 247

Bibliography 249

Index 260 viii CONTENTS

Chapter 1

Some mappings, metrics, and norms

1.1 Lipschitz mappings

Let (X, d_{X}(·, ·)) and (Y, d_{Y} (·, ·)) be metric spaces. A mapping f from X into Y is said to be Lipschitz if there is a nonnegative real number C such that

d_{Y} (f(x), f(w)) ≤ C d_{X}(x, w) (1.1.1)

for every x, w ∈ X. It is easy to see that Lipschitz mappings are uniformly continuous. Note that (1.1.1) holds with C = 0 if and only if f is constant on X.

Let X be a set, and let d(·, ·) and d′(·, ·) be metrics on X. Consider the condition that there be a nonnegative real number C such that

d′(x, w) ≤ C d(x, w) (1.1.2)

for every x, w ∈ X. This is the same as (1.1.1), with d_{X} = d, Y = X, d_{Y} = d′, and f taken to be the identity mapping on X.

Let (X, d(·, ·)) be a metric space, and put

ρ(x, w) = \sqrt{d(x, w)} (1.1.3)

for every x, w ∈ X. One can check that this defines a metric on X as well. This corresponds to the second part of Exercise 11 at the end of Chapter 2 in [155], when we start with the standard Euclidean metric on the real line.

One can check that the identity mapping on X is uniformly continuous as a mapping from X equipped with d(·, ·) into X equipped with ρ(x, w). Similarly, one can check that the identity mapping on X in uniformly continuous as a mapping from X equipped with ρ(·, ·) into X equipped with d(·, ·).

1.2 Lipschitz conditions on R

Let E be a nonempty subset of the real line R, and let f be a real-valued function on E. Note that f is Lipschitz on E with respect to the standard Euclidean metric on R and its restriction to E if and only if there is a nonnegative real number C such that |f(x) − f(w)| ≤ C |x − w| (1.2.1)

for every x, w ∈ E. Here |t| denotes the usual absolute value of a real number t.

Of course, (1.2.1) holds automatically when x = w. If x ≠ w, then (1.2.1) is the same as saying that

\frac{|f(x) − f(w)|}{|x − w|} ≤ C. (1.2.2)

Suppose that x ∈ E is a limit point of E. The derivative of f at x is defined as usual by

f′(x) = lim_{{w∈E \atop w→x}} \frac{f(w) − f(x)}{w − x}, (1.2.3)

when the limit on the right exists. In this case, f is said to be differentiable at x, as a function on E. If f is differentiable at x, and (1.2.1) holds for all w ∈ E, or at least when w ∈ E is sufficiently close to x, then one can check that

|f′(x)| ≤ C. (1.2.4)

If f is differentiable at x, then f is continuous at x, as a function on E, by a standard argument. More precisely, if C is a real number such that

|f′(x)| < C, (1.2.5)

then one can verify that (1.2.1) holds for all w ∈ E that are sufficiently close to x.

Let a and b be real numbers with a < b, and suppose for the moment that E is the corresponding open interval (a, b) in R. We may also allow a = −∞ or b = +∞ here, so that E could be the real line, or an open half-line in R. Suppose that f is differentiable at every point in E, and that there is a nonnegative real number C such that (1.2.4) holds for every x ∈ E. Under these conditions, the mean value theorem implies that (1.2.1) holds for every x, w ∈ E.

Let a and b be real numbers with a < b again, and suppose now that E is the corresponding closed interval [a, b] in R. Suppose that f is continuous on [a, b], and differentiable at every point in (a, b). If there is a nonnegative real number C such that (1.2.4) holds for every x ∈ (a, b), then the mean-value theorem implies that (1.2.1) holds for every x, w ∈ E.

Of course, there are analogous statements when E is a half-open, half-closed interval in R, or a closed half-line in R.

See [13, 34, 39] for some related perspectives on the mean value theorem. Some additional results related to the mean value theorem can be found in [99, 181]. Some aspects of calculus on the rationals are discussed in [100, 102]. Some more connections between Lipschitz conditions and derivatives will be considered in Chapter 8.

1.3. NORMS ON R^{N} 3

1.3 Norms on R^{n}

Let n be a positive integer. Remember that R^{n} is the space of ordered n-tuples x = (x_{1}, …, x_{n}) such that x_{j} ∈ R for each j = 1, …, n. Addition can be defined on R^{n} coordinatewise, so that

x + y = (x_{1} + y_{1}, …, x_{n} + y_{n}) (1.3.1)

for every x, y ∈ R^{n}. Similarly, if t ∈ R and x ∈ R^{n}, then

t x = (t x_{1}, …, t x_{n}) (1.3.2)

defines another element of R^{n}. Using these definitions of addition and scalar multiplication, R^{n} becomes a vector space over the real numbers. Although we shall not discuss the formal definition of a vector space here, the relevant notions will hopefully be clear in the examples. Note that we shall use 0 to refer to the element of R^{n} whose coordinates are equal to the real number 0, which will hopefully also be clear from the context.

A nonnegative real-valued function N on R^{n} is said to define a norm on R^{n} if it satisfies the following three conditions. First, N(x) = 0 if and only if x = 0. Second, N(t x) = |t| N(x) (1.3.3)

for every t ∈ R and x ∈ R^{n}. Third,

N(x + y) ≤ N(x) + N(y) (1.3.4)

for every x, y ∈ R^{n}, which is the triangle inequality for norms.

The standard Euclidean norm is defined by

∥x∥_{2} = ( ∑_{j=1}^{n} x_{j}^{2})^{1/2} (1.3.5)

for every x ∈ R^{n}. Of course, this uses the nonnegative square root on the right side of the equation. It is easy to see that this satisfies the first two requirements of a norm mentioned in the preceding paragraph. The triangle inequality is more complicated, and can be obtained from the Cauchy–Schwarz inequality. See Theorem 1.37 on p16 of [155].

One can check directly that

∥x∥_{1} = ∑_{j=1}^{n} |x_{j}| (1.3.6)

defines a norm on R^{n}. (Exercise.) Similarly,

∥x∥_{∞} = max(|x_{1}|, …, |x_{n}|) (1.3.7) defines a norm on R^{n}, where more precisely the right side is the maximum of |x_{1}|, …, |x_{n}|. In particular, if x, y ∈ R^{n}, then

∥x + y∥_{∞} = max(|x_{1} + y_{1}|, …, |x_{n} + y_{n}|)
(1.3.8)  ≤ max(|x_{1}| + |y_{1}|, …, |x_{n}| + |y_{n}|) ≤ ∥x∥_{∞} + ∥y∥_{∞},

using the triangle inequality for the absolute value function on R in the second step.

If N is any norm on R^{n}, then

d_{N}(x, y) = N(x − y) (1.3.9)

defines a metric on R^{n}. Indeed, the first requirement of a norm ensures that (1.3.9) is equal to 0 if and only if x = y. The homogeneity condition (1.3.3) implies that (1.3.9) is symmetric in x and y, by taking t = −1 in (1.3.3). The triangle inequality for (1.3.9) as a metric on R^{n} follows from the triangle inequality (1.3.4) for N as a norm on R^{n}.

The metric d_{2}(x, y) = ∥x − y∥_{2} (1.3.10)

associated to the standard Euclidean norm (1.3.5) is the standard Euclidean metric on R^{n}. Let d_{1}(x, y) = ∥x − y∥_{1} (1.3.11)

and d_{∞}(x, y) = ∥x − y∥_{∞} (1.3.12)

be the metrics on R^{n} corresponding to the norms (1.3.6) and (1.3.7), respectively. If n = 1, then the norms (1.3.5), (1.3.6), and (1.3.7) reduce to the absolute value function on R, and the corresponding metrics are the same as the standard Euclidean metric on R.

If a_{1}, …, a_{n}, b_{1}, …, b_{n} are nonnegative real numbers, then

∑_{j=1}^{n} a_{j} b_{j} ≤ ( ∑_{j=1}^{n} a_{j}^{2})^{1/2} ( ∑_{j=1}^{n} b_{j}^{2})^{1/2}. (1.3.13)

This is a version of the Cauchy–Schwarz inequality. This is often formulated a bit differently for arbitrary real or complex numbers, as in Theorem 1.35 on p15 of [155]. This formulation is included in the other one, by restricting one’s attention to nonnegative real numbers. The other formulation can also be obtained from this one, by applying (1.3.13) to the absolute values of the given real or complex numbers.

1.4 Norms on C^{n}

Let C be the complex plane, as usual, and let n be a positive integer again. As before, C^{n} is the set of ordered n-tuples of complex numbers. Addition can be 1.4. NORMS ON C^{N} 5

defined on C^{n} coordinatewise, as in (1.3.1). One can also multiply an element of C^{n} by a complex number coordinatewise, as in (1.3.2). In this way, C^{n} becomes a vector space over the complex numbers.

A nonnegative real-valued function N on C^{n} is said to be a norm on C^{n} if it satisfies the same type of conditions as in the previous section. More precisely, the first and third conditions conditions are the same as before. In this situation, the homogeneity condition (1.3.3) should hold for all complex numbers t and elements of C^{n}, where |t| is the usual absolute value function on C.

The standard Euclidean norm is defined on C^{n} by

∥z∥_{2} = ( ∑_{j=1}^{n} |z_{j}|^{2})^{1/2}, (1.4.1)

where |z_{j}| is the absolute value of z_{j} ∈ C for each j = 1, …, n. The triangle inequality for (1.4.1) can be reduced to the real case, by taking the absolute values of the coordinates of elements of C^{n} to get n-tuples of nonnegative real numbers. This argument uses the triangle inequality for the absolute value function on C, which is the same as the n = 1 case. The triangle inequality for (1.4.1) on C^{n} can also be obtained from the Cauchy–Schwarz inequality, using an argument analogous to the one in the real case. As before, it is easy to verify the other two requirements for (1.4.1) to be a norm on C^{n} directly from the definition.

Alternatively, the complex plane can be identified with R^{2}, using the real and imaginary parts of a complex number. Using this identification, the absolute value of a complex number corresponds to the standard Euclidean norm on R^{2}. Similarly, C^{n} can be identified with R^{2n}, using the real and imaginary parts of the n coordinates of an element of C^{n}. Using this identification, the standard Euclidean norm (1.4.1) on C^{n} corresponds exactly to the standard Euclidean norm on R^{2n}. This permits one to get the triangle inequality for (1.4.1) on C^{n} from the triangle inequality for the standard Euclidean norm on R^{2n}, because addition on C^{n} corresponds exactly to addition on R^{2n} with respect to this identification.

As before, one can verify directly that

∥z∥_{1} = ∑_{j=1}^{n} |z_{j}| (1.4.2)

and ∥z∥_{∞} = max(|z_{1}|, …, |z_{n}|) (1.4.3)

define norms on C^{n} as well. Of course, R^{n} may be considered as a subset of C^{n}, because R is contained in C. The restrictions of (1.4.2) and (1.4.3) to z ∈ R^{n} are the same as the corresponding norms defined on R^{n} in the previous section. Similarly, the restriction of (1.4.1) to z ∈ R^{n} is the same as the standard Euclidean norm on R^{n}.

If N is any norm on C^{n}, then

d_{N}(z, w) = N(z − w) (1.4.4) defines a metric on C^{n}, for the same reasons as in the real case. The standard Euclidean metric on C^{n} is the metric

d_{2}(z, w) = ∥z − w∥_{2} (1.4.5)

associated to the standard Euclidean norm (1.4.1). Similarly, let

d_{1}(z, w) = ∥z − w∥_{1} (1.4.6)

and d_{∞}(z, w) = ∥z − w∥_{∞} (1.4.7)

be the metrics on C^{n} associated to the norms (1.4.2) and (1.4.3), respectively. If n = 1, then the norms (1.4.1), (1.4.2), and (1.4.3) reduce to the absolute value function on C, so that the corresponding metrics are the same as the standard Euclidean metric on C.

The restriction of any norm N on C^{n} to R^{n} defines a norm on R^{n}. In this case, the restriction of (1.4.4) to z, w ∈ R^{n} is the same as the metric on R^{n} associated to the restriction of N to R^{n}. In particular, the restrictions of (1.4.5), (1.4.6), and (1.4.7) to z, w ∈ R^{n} are the same as the corresponding metrics defined on R^{n} in the previous section.

Let p be a positive real number, and put

∥z∥_{p} = ( ∑_{j=1}^{n} |z_{j}|^{p})^{1/p} (1.4.8)

for every z ∈ C^{n}. It is easy to see that this satisfies that positivity and homogeneity requirements of a norm. If p ≥ 1, then it is well known that (1.4.8) satisfies the triangle inequality, and hence defines a norm on C^{n}. This is a version of Minkowski’s inequality for sums. Of course, (1.4.8) is the same as (1.4.1) when p = 2, and it is the same as (1.4.2) when p = 1. If n = 1, then (1.4.8) reduces to the absolute value function on C. If 0 < p < 1 and n ≥ 2, then one can show that (1.4.8) does not satisfy the triangle inequality, because the corresponding balls in C^{n} are not convex. There are analogous statements for the restriction of (1.4.8) to R^{n}.

1.5 Some basic inequalities

Let n be a positive integer, and let z ∈ C^{n} be given. It is easy to see that

∥z∥_{∞} ≤ ∥z∥_{2}, ∥z∥_{1}, (1.5.1)

directly from the definitions of these norms in the previous section. Similarly,

∥z∥_{2}^{2} = ∑_{j=1}^{n} |z_{j}|^{2} ≤ ∥z∥_{∞} ∑_{j=1}^{n} |z_{j}| = ∥z∥_{∞} ∥z∥_{1} ≤ ∥z∥_{1}^{2}, (1.5.2)

so that ∥z∥_{2} ≤ ∥z∥_{1}. (1.5.3)

It follows that the corresponding metrics satisfy

d_{∞}(z, w) ≤ d_{2}(z, w) ≤ d_{1}(z, w) (1.5.4)

for every z, w ∈ C^{n}.

In the other direction, we have that

∥z∥_{2} ≤ n^{1/2} ∥z∥_{∞} (1.5.5)

and ∥z∥_{1} ≤ n ∥z∥_{∞} (1.5.6)

for every z ∈ C^{n}. One can also check that

∥z∥_{1} ≤ n^{1/2} ∥z∥_{2} (1.5.7)

for every z ∈ C^{n}, using the Cauchy-Schwarz inequality (1.3.13). This implies that

(1.5.8)  d_{2}(z, w) ≤ n^{1/2} d_{∞}(z, w),
(1.5.9)  d_{1}(z, w) ≤ n d_{∞}(z, w),

and d_{1}(z, w) ≤ n^{1/2} d_{2}(z, w) (1.5.10)

for every z, w ∈ C^{n}.

Using these simple relationships, we get that d_{1}(z, w), d_{2}(z, w), and d_{∞}(z, w) have many of the same properties on C^{n}. They determine the same collections of open sets, closed sets, compact sets, and bounded sets, for instance. They also determine the same limit points of subsets of C^{n}, convergent sequences in C^{n}, and Cauchy sequences. Using (1.5.4), it is easy to see that the identity mapping on C^{n} is Lipschitz as a mapping from C^{n} equipped with d_{1}(z, w) into C^{n} equipped with d_{2}(z, w), and from C^{n} equipped with d_{2}(z, w) into C^{n} equipped with d_{∞}(z, w). Similarly, the identity mapping on C^{n} is Lipschitz as a mapping from C^{n} equipped with d_{1}(z, w) into C^{n} equipped with d_{∞}(z, w). We can use (1.5.8) to get that the identity mapping on C^{n} is Lipschitz as a mapping from C^{n} equipped with d_{∞}(z, w) into C^{n} equipped with d_{2}(z, w), and (1.5.10) implies that the identity mapping on C^{n} is Lipschitz as a mapping from C^{n} equipped with d_{2}(z, w) into C^{n} equipped with d_{1}(z, w). One can use (1.5.9) to get that the identity mapping on C^{n} is Lipschitz as a mapping from C^{n} equipped with d_{∞}(z, w) into C^{n} equipped with d_{1}(z, w). Of course, there are analogous statements for the restrictions of these metrics to R^{n}.

If N is any norm on R^{n} or C^{n}, then one can show that N is bounded by a constant times the standard Euclidean norm, or equivalently by a constant times either of the norms ∥ · ∥_{1}, ∥ · ∥_{∞}. To see this, one can express any element of R^{n} or C^{n} as a linear combination of the standard basis vectors, to estimate N in terms of the absolute values of the coodinates of the given vector. One can use this to show that N is continuous as a real-valued function on R^{n} or C^{n}, as appropriate, with respect to the standard Euclidean metric. It follows that N attains its minimum on the unit sphere in R^{n} or C^{n}, as appropriate, with respect to the standard Euclidean metric again. Because the infimum is positive, by definition of a norm, one can verify that the standard Euclidean norm is bounded by a constant times N on R^{n} or C^{n}, as appropriate.

1.6 Functions with finite support

Let X be a nonempty set, and let f be a real or complex-valued function on X. The support of f is defined to be the subset of X given by

supp f = {x ∈ X : f(x) ≠ 0}. (1.6.1)

Let c_{00}(X, R) be the space of real-valued functions on X whose support has only finitely many elements, and let c_{00}(X, C) be the space of complex-valued functions on X with finite support. If f and g are real or complex-valued functions on X with finite support, then their sum f(x) + g(x) also defines a real or complex-valued function on X with finite support. Similarly, if f is a real or complex-valued function on X with finite support, and t is a real or complex number, as appropriate, then t f(x) has finite support in X. More precisely, c_{00}(X, R) and c_{00}(X, C) are vector spaces over the real and complex numbers, respectively, with respect to pointwise addition and scalar multiplication of functions. These may be considered as linear subspaces of the spaces of all real or complex-valued functions on X, respectively.

Of course, if X has only finitely many elements, then every real or complexvalued function on X automatically has finite support. Let n be a positive integer, and suppose for the moment that

X = {1, …, n} (1.6.2)

is the set of positive integers from 1 to n. In this case, c_{00}(X, R) and c_{00}(X, C) can be identified with R^{n} and C^{n}, respectively. Similarly, if X is the set Z_{+} of positive integers, then a real or complex-valued function on X corresponds to an infinite sequence of real or complex numbers. Thus c_{00}(Z_{+}, R) and c_{00}(Z_{+}, C) can be identified with the spaces of infinite sequences of real or complex numbers for which all but finitely many terms are equal to 0, respectively.

As usual, a nonnegative real-valued function N on c_{00}(X, R) or c_{00}(X, C) is said to be a norm if it satisfies the following three conditions. First, N(f) = 0 if and only if f = 0. Second, if f ∈ c_{00}(X, R) or c_{00}(X, C) and t ∈ R or C, as appropriate, then N(t f) = |t| N(f). (1.6.3)

Third, N(f + g) ≤ N(f) + N(g) (1.6.4) for every f, g ∈ c_{00}(X, R) or c_{00}(X, C), as appropriate. In this case,

d_{N}(f, g) = N(f − g) (1.6.5)

defines a metric on c_{00}(X, R) or c_{00}(X, C), as appropriate.

Let f be a real or complex-valued function on X. If A is a nonempty finite subset of X, then

∑_{x∈A} f(x) (1.6.6)

can be defined as a real or complex number, as appropriate. Suppose that f has finite support in X, and observe that the finite sums (1.6.6) are all the same when supp f ⊆ A. This permits us to define the sum

∑_{x∈X} f(x) (1.6.7)

as a real or complex number, as appropriate, as the value of (1.6.6) when A is a nonempty finite subset of X that contains the support of f.

If f is a real or complex-valued function on X with finite support, then

∥f∥_{1} = ∑_{x∈X} |f(x)| (1.6.8)

is defined as a nonnegative real number, as in the previous paragraph. Similarly,

∥f∥_{2} = ( ∑_{x∈X} |f(x)|^{2})^{1/2} (1.6.9)

is defined as a nonnegative real number, where the sum on the right is defined as before. We can also put

∥f∥_{∞} = max_{x∈X} |f(x)|, (1.6.10)

where the maximum of |f(x)| over x ∈ X is clearly attained in this situation. One can check that these define norms on c_{00}(X, R) and c_{00}(X, C). In particular, the triangle inequality for (1.6.9) reduces to the analogous statement for R^{n} and C^{n}, mentioned in Sections 1.3 and 1.4.

Using these norms, we get metrics

(1.6.11)  d_{1}(f, g) = ∥f − g∥_{1},
(1.6.12)  d_{2}(f, g) = ∥f − g∥_{2},

and d_{∞}(f, g) = ∥f − g∥_{∞} (1.6.13)

on c_{00}(X, R) and c_{00}(X, C). If X is as in (1.6.2) for some positive integer n, then the norms mentioned in the previous paragraph correspond to the analogous norms defined on R^{n} and C^{n} in Sections 1.3 and 1.4. Similarly, these metrics correspond to the analogous metrics defined earlier on R^{n} and C^{n} in this case.

If f is a real or complex-valued function on X with finite support, then

∥f∥_{∞} ≤ ∥f∥_{2} ≤ ∥f∥_{1}. (1.6.14)

This follows from (1.5.1) and (1.5.3), since it is enough to look at the finitely many elements of X in the support of f. This implies that

d_{∞}(f, g) ≤ d_{2}(f, g) ≤ d_{1}(f, g) (1.6.15)

for all real and complex-valued functions f and g on X with finite support.

It follows that the identity mappings on c_{00}(X, R) and c_{00}(X, C) are Lipschitz with respect to d_{2}(f, g) on the domain and d_{∞}(f, g) on the range, as before. Similarly, the identity mappings on c_{00}(X, R) and c_{00}(X, C) are Lipschitz with respect to d_{1}(f, g) on the domain and d_{2}(f, g) on the range. The identity mappings on c_{00}(X, R) and c_{00}(X, C) are also Lipschitz with respect to d_{1}(f, g) on the domain and d_{∞}(f, g) on the range.

1.7 Cauchy sequences and completeness

Let (X, d_{X}(·, ·)) be a metric space. Remember that a sequence {x_{j}}_{j=1}^{∞} of elements of X is said to be a Cauchy sequence with respect to d_{X}(·, ·) if for every ϵ > 0 there is a positive integer L such that

d_{X}(x_{j}, x_{l}) < ϵ (1.7.1)

for every j, l ≥ L. It is not difficult to verify that

(1.7.2) convergent sequences in X are Cauchy sequences.

If (1.7.3) every Cauchy sequence in X converges to an element of X,

then X is said to be complete with respect to d_{X}(·, ·). It is well known that R and C are complete with respect to their standard Euclidean metrics.

Let E be a subset of X, and remember that the restriction of d_{X}(x, w) to x, w ∈ E defines a metric on E. If {x_{j}}_{j=1}^{∞} is a sequence of elements of E, then it is easy to see that

(1.7.4) {x_{j}}_{j=1}^{∞} is a Cauchy sequence as a sequence of elements of E

if and only if

(1.7.5) {x_{j}}_{j=1}^{∞} is a Cauchy sequence as a sequence of elements of X,

with respect to d_{X}(·, ·) and its restriction to E. If X is complete with respect to d_{X}(·, ·), and if E is a closed set in X, then

(1.7.6) E is complete with respect to the restriction of d_{X}(·, ·) to E.

More precisely, if {x_{j}}_{j=1}^{∞} is a Cauchy sequence of elements of E, then {x_{j}}_{j=1}^{∞} is a Cauchy sequence in X, which converges to some x ∈ X, because X is complete. We also have that x ∈ E, because E is a closed set in X, so that {x_{j}}_{j=1}^{∞} converges to x in E, with respect to the restriction of d_{X}(·, ·) to E.

Suppose now that E is complete with respect to the restriction of d_{X}(·, ·) to E, and let us check that

(1.7.7) E is a closed set in X.

Let x ∈ X be a limit point of E, which implies that there is a sequence {x_{j}}_{j=1}^{∞} of elements of E that converges to x in X. Note that {x_{j}}_{j=1}^{∞} is a Cauchy sequence as a sequence of elements of X, and hence as a sequence of elements of E. Because E is complete, there is an x′ ∈ E such that {x_{j}}_{j=1}^{∞} converges to x′ with respect to the restriction of d_{X}(·, ·) to E. Of course, {x_{j}}_{j=1}^{∞} converges to x′ in X as well, so that x = x′, and thus x ∈ E.

Let (Y, d_{Y} (·, ·)) be another metric space, and let f be a uniformly continuous mapping from X into Y. If {x_{j}}_{j=1}^{∞} is a Cauchy sequence of elements of X, then one can check that

(1.7.8) {f(x_{j})}_{j=1}^{∞} is a Cauchy sequence in Y.

This is the first part of Exercise 11 at the end of Chapter 4 in [155]. If Y is complete, then it follows that {f(x_{j})}_{j=1}^{∞} converges in Y. If {x_{j}}_{j=1}^{∞} converges to an element x of X, then continuity of f at x implies that {f(x_{j})}_{j=1}^{∞} converges to f(x) in Y.

1.8 Pointwise and uniform convergence

Let X be a set, and let (Y, d_{Y} (·, ·)) be a metric space. Also let {f_{j}}_{j=1}^{∞} be a sequence of mappings from X into Y, and let f be another mapping from X into Y. We say that {f_{j}}_{j=1}^{∞} converges to f pointwise on X if for every x ∈ X, {f_{j}(x)}_{j=1}^{∞} converges to f(x) in Y. This means that for every x ∈ X and ϵ > 0 there is a positive integer L such that

d_{Y} (f_{j}(x), f(x)) < ϵ (1.8.1)

for every j ≥ L. We say that {f_{j}}_{j=1}^{∞} converges to f uniformly on X if for every ϵ > 0 there is a positive integer L such that (1.8.1) holds for every x ∈ X and j ≥ L. Note that uniform convergence implies pointwise convergence. If X has only finitely many elements, and {f_{j}}_{j=1}^{∞} converges to f pointwise on X, then one can check that {f_{j}}_{j=1}^{∞} converges to f uniformly on X.

As an example, let us take X to be the closed unit interval [0, 1] in the real line, and Y = R with the standard metric. Put

f_{j}(x) = x^{j} (1.8.2)

12 CHAPTER 1. SOME MAPPINGS, METRICS, AND NORMS

for each positive integer j and 0 ≤ x ≤ 1. In this case,

(1.8.3) lim_{j→∞} f_{j}(x) = 0 when 0 ≤ x < 1
= 1 when x = 1.

However, {f_{j}}_{j=1}^{∞} does not converge uniformly on [0, 1], because for each positive integer j we have that x^{j} is as close to 1 as we want when x is sufficiently close to 1. If r is a positive real number with r < 1, then {f_{j}}_{j=1}^{∞} does converge to 0 uniformly on [0, r].

Now let (X, d_{X}) be a metric space, and let (Y, d_{Y}) be a metric space again too. Also let (1.8.4) {f_{j}}_{j=1}^{∞} be a sequence of mappings from X into Y

that converges uniformly to a mapping f from X into Y,

and let x ∈ X be given. If (1.8.5) f_{j} is continuous at x for every j ≥ 1, then (1.8.6) f is continuous at x as well. To see this, let ϵ > 0 be given. Because {f_{j}}_{j=1}^{∞} converges uniformly to f on X, there is an L ∈ Z_{+} such that d_{Y} (f_{j}(w), f(w)) < ϵ/3 (1.8.7) for every j ≥ L and w ∈ X. In particular, this holds at x, so that d_{Y} (f_{j}(x), f(x)) < ϵ/3 (1.8.8) for every j ≥ L. Because f_{L} is continuous at x, there is a δ_{L} > 0 such that d_{Y} (f_{L}(x), f_{L}(w)) < ϵ/3 (1.8.9) for every w ∈ X with d_{X}(x, w) < δ_{L}. Observe that

(1.8.10) d_{Y} (f(x), f(w)) ≤ d_{Y} (f(x), f_{L}(x)) + d_{Y} (f_{L}(x), f_{L}(w))
+d_{Y} (f_{L}(w), f(w))

for every w ∈ X, by the triangle inequality. It follows that d_{Y} (f(x), f(w)) < ϵ/3 + ϵ/3 + ϵ/3 = ϵ (1.8.11) for every w ∈ X with d_{X}(x, w) < δ_{L}, as desired.

Similarly, if {f_{j}}_{j=1}^{∞} is a sequence of uniformly continuous mappings from X into Y that converges uniformly to a mapping f from X into Y, then (1.8.12) f is uniformly continuous on X.

As before, we let ϵ > 0 be given, and let L be a positive integer such that (1.8.8) holds for every j ≥ L and w ∈ X. In this case, the uniform continuity of f_{L} implies that there is a δ_{L} > 0 such that (1.8.9) holds for every x, w ∈ X with d_{X}(x, w) < δ_{L}. This implies that (1.8.11) holds for every x, w ∈ X with d_{X}(x, w) < δ_{L}, as before.

Note that there are related statements about limits of functions at a given point, instead of continuity at a point.

1.9 Bounded sets

Let (X, d_{X}) be a metric space. If x ∈ X and r is a positive real number, then the open ball in X centered at x with radius r is defined as usual by

B(x, r) = B_{X}(x, r) = {w ∈ X : d_{X}(x, w) < r}. (1.9.1)

If x′ is another element of X, then it is easy to see that

B(x, r) ⊆ B(x′, r + d_{X}(x, x′)), (1.9.2)

using the triangle inequality. It is well known that open balls in X are open sets.

A subset E of X is said to be bounded in X if there is an x ∈ X and an r > 0 such that E ⊆ B(x, r). (1.9.3)

This implies that for every x′ ∈ X there is an r′ > 0 such that

E ⊆ B(x′, r′), (1.9.4)

because of (1.9.2). Of course, this condition implies the previous one when X ≠ ∅. To avoid minor technicalities, the empty set will be considered as a bounded set even when X = ∅.

If K is a compact subset of X, then K is bounded in X. This is trivial when X = ∅, because the empty set is automatically considered to be a bounded set, and so we may suppose that X ≠ ∅. If x is any element of X, then the collection of open balls B(x, j) with j ∈ Z_{+} is an open covering of K, because

⋃_{j=1}^{∞} B(x, j) = X. (1.9.5)

If K is compact, then there are finitely many positive integers j_{1}, …, j_{n} such that

K ⊆ ⋃_{l=1}^{n} B(x, j_{l}). (1.9.6)

This implies that K ⊆ B(x, r), with r = max(j_{1}, …, j_{n}).

Note that subsets of bounded sets are bounded. Let E_{1}, …, E_{n} be finitely many bounded subsets of X, and let us check that their union ⋃_{j=1}^{n} E_{j} is bounded. As before, this is trivial when X = ∅, and so we may suppose that X ≠ ∅. If x is any element of X, then for each j = 1, …, n there is a positive real number r_{j} such that E_{j} ⊆ B(x, r_{j}). (1.9.7)

This implies that

⋃_{j=1}^{n} E_{j} ⊆ B(x, max_{1≤j≤n} r_{j}), (1.9.8)

as desired.

If x ∈ X and r is a nonnegative real number, then the closed ball in X centered at x with radius r is defined by

\overline{B}(x, r) = \overline{B}_{X}(x, r) = \{w ∈ X : d_{X}(x, w) ≤ r\}. (1.9.9)

If x′ is another element of X, then

\overline{B}(x, r) ⊆ \overline{B}(x′, r + d_{X}(x, x′)), (1.9.10)

as in (1.9.2). One can check that closed balls in X are closed sets.

If X ≠ ∅, then a subset E of X is bounded if and only if it is contained in a closed ball in X. In this case, E is contained in a closed ball centered at any point in X, as before. In particular, if E is bounded, then the closure \overline{E} of E in X is bounded too.

A sequence {x_{j}}_{j=1}^{∞} of elements of X is said to be bounded in X if the set of x_{j}’s, j ∈ Z_{+}, is bounded in X. One can check that

(1.9.11) convergent sequences in X are bounded.

Similarly, one can verify that

(1.9.12) Cauchy sequences in X are bounded.

Totally bounded subsets of metric spaces are discussed in Section 4.2. In particular, compact sets are totally bounded, and totally bounded sets are bounded.

1.10 Some remarks and examples

Let n be a positive integer, and let E be a subset of R^{n} or C^{n}. If E is bounded with respect to any of the metrics d_{1}, d_{2}, or d_{∞} defined in Sections 1.3 or 1.4, as appropriate, then it is easy to see that E is bounded with respect to the other two metrics, using the inequalities in Section 1.5. Similarly, if a sequence of elements of R^{n} or C^{n} converges with respect to any of these three metrics, then it converges with respect to the other two metrics, and with the same limit.

Let X be a nonempty set, and let E be a subset of c_{00}(X, R) or c_{00}(X, C). If E is bounded with respect to the metric d_{2} defined in Section 1.6, then E is bounded with respect to d_{∞}. Similarly, if E is bounded with respect to d_{1}, then E is bounded with respect to d_{2} and d_{∞}. If X has only finitely many elements, and E is bounded with respect to d_{∞}, then E is bounded with respect to d_{1} and d_{2}.

Let {f_{j}}_{j=1}^{∞} be a sequence of elements of c_{00}(X, R) or c_{00}(X, C), and let f be another element of the same space. If {f_{j}}_{j=1}^{∞} converges to f with respect to d_{2}, then {f_{j}}_{j=1}^{∞} converges to f with respect to d_{∞}. If {f_{j}}_{j=1}^{∞} converges to f with respect to d_{1}, then {f_{j}}_{j=1}^{∞} converges to f with respect to d_{2} and d_{∞}. If X has only finitely many elements, and {f_{j}}_{j=1}^{∞} converges to f with respect to d_{∞}, then {f_{j}}_{j=1}^{∞} converges to f with respect to d_{1} and d_{2}.

One can check that {f_{j}}_{j=1}^{∞} converges to f with respect to d_{∞} if and only if {f_{j}}_{j=1}^{∞} converges to f uniformly on X. This uses the standard metric on R or C, as appropriate.

If x ∈ X, then let δ_{x} be the real-valued function on X equal to 1 at x, and to 0 at every other element of X. It is easy to see that the collection of δ_{x}’s, x ∈ X, is a basis for each of c_{00}(X, R) and c_{00}(X, C), as vector spaces over R and C, respectively, if one is familiar with these notions from linear algebra. Basically, this means that every element f of c_{00}(X, R) or c_{00}(X, C) can be expressed in a unique way as a linear combination of the δ_{x}’s, x ∈ X, with coefficients in R or C, as appropriate. In fact, the coefficient of δ_{x} is equal to f(x) for each x ∈ X.

Observe that ∥δ_{x}∥_{1} = ∥δ_{x}∥_{2} = ∥δ_{x}∥_{∞} = 1 (1.10.1) for every x ∈ X. If x, y ∈ X and x ≠ y, then

(1.10.2)  ∥δ_{x} − δ_{y}∥_{∞} = 1,
(1.10.3)  ∥δ_{x} − δ_{y}∥_{2} = \sqrt{2},
(1.10.4)  ∥δ_{x} − δ_{y}∥_{1} = 2.

Let us now take X = Z_{+}, and let δ_{j} be as before for each positive integer j. It is easy to see that {δ_{j}}_{j=1}^{∞} converges to 0 pointwise on Z_{+}. Note that {δ_{j}}_{j=1}^{∞} is bounded with respect to each of d_{1}, d_{2}, and d_{∞}, and that {δ_{j}}_{j=1}^{∞} does not converge to 0 with respect to any of these three metrics.

Similarly, {j δ_{j}}_{j=1}^{∞} converges to 0 pointwise on Z_{+}. However,

∥j δ_{j}∥_{1} = ∥j δ_{j}∥_{2} = ∥j δ_{j}∥_{∞} = j (1.10.5)

for each j, so that {j δ_{j}}_{j=1}^{∞} is not bounded with respect to d_{1}, d_{2}, or d_{∞}. If j ∈ Z_{+}, then let f_{j} be the real-valued function on Z_{+} defined by

(1.10.6)  f_{j}(l) = 1 when l ≤ j
= 0 when l > j.

Observe that {f_{j}}_{j=1}^{∞} converges pointwise to the function equal to 1 everywhere on Z_{+}. One can check that {f_{j}}_{j=1}^{∞} does not converge uniformly on Z_{+}. We also have that

(1.10.7)  ∥f_{j}∥_{∞} = 1,
(1.10.8)  ∥f_{j}∥_{2} = \sqrt{j},
(1.10.9)  ∥f_{j}∥_{1} = j

for every j.

If α ∈ R, then ∥j^{−α} f_{j}∥_{∞} = j^{−α} (1.10.10)

for every j. This means that {j^{−α}f_{j}}_{j=1}^{∞} is bounded with respect to d_{∞} exactly when α ≥ 0, and that {j^{−α} f_{j}}_{j=1}^{∞} converges to 0 with respect to d_{∞} exactly when α > 0. Similarly, ∥j^{−α} f_{j}∥_{2} = j^{(1/2)−α} (1.10.11)

for every j, so that {j^{−α} f_{j}}_{j=1}^{∞} is bounded with respect to d_{2} if and only if α ≥ 1/2, and {j^{−α} f_{j}}_{j=1}^{∞} converges to 0 with respect to d_{2} exactly when α > 1/2. In the same way,

∥j^{−α} f_{j}∥_{1} = j^{1−α} (1.10.12)

for every j, so that {j^{−α} f_{j}}_{j=1}^{∞} is bounded with respect to d_{1} if and only if α ≥ 1, and {j^{−α} f_{j}}_{j=1}^{∞} converges to 0 with respect to d_{1} if and only if α > 1.

1.11 Bounded functions

Let X be a set, and let (Y, d_{Y}) be a metric space. A mapping f from X into Y is said to be bounded if

(1.11.1) the image f(X) of X under f is a bounded subset of Y.

Let B(X, Y) be the space of bounded mappings from X into Y.

Let {f_{j}}_{j=1}^{∞} be a sequence of bounded mappings from X into Y that converges uniformly to a mapping f from X into Y. This implies that there is an L ∈ Z_{+} such that d_{Y} (f_{j}(x), f(x)) < 1 (1.11.2)

for every j ≥ L and x ∈ X. One can use this to check that

f is bounded, (1.11.3)

because f_{L} is bounded.

Suppose that X ≠ ∅, and let f, g be bounded mappings from X into Y. It is easy to see that d_{Y} (f(x), g(x)) (1.11.4)

is bounded as a nonnegative real-valued function of x on X, using the triangle inequality. Put θ(f, g) = sup{d_{Y} (f(x), g(x)) : x ∈ X}. (1.11.5)

If f = g, then f(x) = g(x) for every x ∈ X, so that

d_{Y} (f(x), g(x)) = 0 (1.11.6)

for every x ∈ X, and hence θ(f, g) = 0. (1.11.7)

1.12. COMPLETENESS OF B(X, Y) 17 Conversely, if (1.11.7) holds, then (1.11.6) holds for every x ∈ X, so that f(x) = g(x) for every x ∈ X, which means that f = g. We also have that θ(f, g) = θ(g, f), (1.11.8) because d_{Y} (f(x), g(x)) = d_{Y} (g(x), f(x)) for every x ∈ X. If h is another bounded mapping from X into Y, then

(1.11.9)  d_{Y} (f(x), h(x)) ≤ d_{Y} (f(x), g(x)) + d_{Y} (g(x), h(x))
≤ θ(f, g) + θ(g, h)

for every x ∈ X. This implies that θ(f, h) ≤ θ(f, g) + θ(g, h). (1.11.10) Thus (1.11.5) defines a metric on B(X, Y), which is known as the supremum metric.

Let {f_{j}}_{j=1}^{∞} be a sequence of bounded mappings from X into Y, and let f be another bounded mapping from X into Y. If {f_{j}}_{j=1}^{∞} converges to f with respect to the supremum metric, then for each ϵ > 0 there is an L(ϵ) ∈ Z_{+} such that θ(f_{j}, f) < ϵ (1.11.11) for every j ≥ L(ϵ). It follows that d_{Y} (f_{j}(x), f(x)) < ϵ (1.11.12) for every j ≥ L(ϵ) and x ∈ X, so that {f_{j}}_{j=1}^{∞} converges to f uniformly on X.

Conversely, if {f_{j}}_{j=1}^{∞} converges to f uniformly on X, then for each ϵ > 0 there is an L′(ϵ) ∈ Z_{+} such that (1.11.12) holds for every j ≥ L′(ϵ) and x ∈ X. This implies that θ(f_{j}, f) = sup{d_{Y} (f_{j}(x), f(x)) : x ∈ X} ≤ ϵ (1.11.13) for every j ≥ L′(ϵ), and hence that {f_{j}}_{j=1}^{∞} converges to f with respect to the supremum metric.

1.12 Completeness of B(X, Y)

Let us continue with the same notation and hypotheses as in the previous section. Suppose that (1.12.1) Y is complete with respect to d_{Y} , and let us check that (1.12.2) B(X, Y) is complete with respect to the supremum metric.

Let {f_{j}}_{j=1}^{∞} be a sequence of bounded mappings from X into Y that is a Cauchy sequence with respect to the supremum metric. This means that for each ϵ > 0 there is an L(ϵ) ∈ Z_{+} such that

θ(f_{j}, f_{l}) < ϵ (1.12.3)

for every j, l ≥ L(ϵ). Thus

d_{Y} (f_{j}(x), f_{l}(x)) < ϵ (1.12.4)

for every j, l ≥ L(ϵ) and x ∈ X. In particular,

(1.12.5) {f_{j}(x)}_{j=1}^{∞} is a Cauchy sequence in Y

for every x ∈ X. Because Y is complete,

(1.12.6) {f_{j}(x)}_{j=1}^{∞} converges in Y

for every x ∈ X, and we put

f(x) = lim_{j→∞} f_{j}(x). (1.12.7)

This defines a mapping f from X into Y, and one can check that

d_{Y} (f_{j}(x), f(x)) ≤ ϵ (1.12.8)

for every j ≥ L(ϵ) and x ∈ X, using (1.12.4). Indeed,

(1.12.9) d_{Y} (f_{j}(x), f(x)) ≤ d_{Y} (f_{j}(x), f_{l}(x)) + d_{Y} (f_{l}(x), f(x))
< ϵ + d_{Y} (f_{l}(x), f(x))

for all j, l ≥ L(ϵ) and x ∈ X, because of (1.12.4) and the triangle inequality. This implies (1.12.8), because {f_{l}(x)}_{l=1}^{∞} converges to f(x) in Y.

It follows that {f_{j}}_{j=1}^{∞} converges to f uniformly on X, and hence that f is bounded on X, as in the previous section. This implies that {f_{j}}_{j=1}^{∞} converges to f with respect to the supremum metric, as before.

One can also verify that (1.12.2) implies (1.12.1). Indeed, every element of Y corresponds to a constant function on X with values in Y. It is easy to see that a Cauchy sequence in Y corresponds to a Cauchy sequence in B(X, Y) with respect to the supremum metric in this way. If a sequence of constant mappings from X to Y converges with respect to the supremum metric, then it converges uniformly on X, and thus pointwise on X, and it is easy to see that the limit is a constant mapping as well.

1.13 More on bounded functions

The spaces of bounded real and complex-valued functions on a empty set X are also denoted ℓ^{∞}(X, R) and ℓ^{∞}(X, C), (1.13.1) respectively. This implicitly uses the standard Euclidean metrics on R and C. If f and g are bounded real or complex-valued functions on X, then it is easy to see that (1.13.2) f + g is bounded on X as well.

Similarly, if f is a bounded real or complex-valued function on X, and t is a real or complex number, as appropriate, then

(1.13.3) t f is bounded on X

too. Thus ℓ^{∞}(X, R) and ℓ^{∞}(X, C) are linear subspaces of the real and complex vector spaces of all real and complex-valued functions on X, respectively.

If f is a bounded real or complex-valued function on X, then put

∥f∥_{∞} = sup{|f(x)| : x ∈ X}. (1.13.4)

Note that ∥f∥_{∞} = 0 if and only if f = 0 on X. If t ∈ R or C, as appropriate, then one can check that ∥t f∥_{∞} = |t| ∥f∥_{∞}. (1.13.5)

If g is another bounded real or complex-valued function on X, then

|f(x) + g(x)| ≤ |f(x)| + |g(x)| ≤ ∥f∥_{∞} + ∥g∥_{∞} (1.13.6)

for every x ∈ X. This implies that

∥f + g∥_{∞} ≤ ∥f∥_{∞} + ∥g∥_{∞}. (1.13.7)

It follows that (1.13.4) defines a norm on each of ℓ^{∞}(X, R) and ℓ^{∞}(X, C), which is known as the supremum norm. The corresponding metric

d_{∞}(f, g) = ∥f − g∥_{∞} (1.13.8)

is the same as the supremum metric on these spaces, associated to the standard Euclidean metric on R or C, as appropriate.

1.14 Continuous functions

Let (X, d_{X}) and (Y, d_{Y}) be metric spaces, and let C(X, Y) be the space of continuous mappings from X into Y. Also let

C_{b}(X, Y) = B(X, Y) ∩ C(X, Y) (1.14.1)

be the space of bounded continuous mappings from X into Y. If f is a continuous mapping from X into Y and X is compact, then it is well known that f(X) is compact in Y, so that f(X) is bounded in Y in particular. Thus C_{b}(X, Y) is the same as C(X, Y) when X is compact.

Suppose that X ≠ ∅, so that the supremum metric can be defined on B(X, Y) as in Section 1.11. Note that

(1.14.2) C_{b}(X, Y) is a closed set in B(X, Y), with respect to the supremum metric. More precisely, if {f_{j}}_{j=1}^{∞} is a sequence of bounded continuous mappings from X into Y that converges to a bounded mapping f from X into Y with respect to the supremum metric, then we have seen that {f_{j}}_{j=1}^{∞} converges to f uniformly on X, and hence that f is continuous on X. Alternatively, if f ∈ B(X, Y) is a limit point of C_{b}(X, Y) with respect to the supremum metric, then one can show that f is continuous on X. This is very similar to the argument used to show that uniform limits of continuous mappings are continuous, as in Section 1.8.

Of course, C_{b}(X, Y) may be considered as a metric space, using the restriction of the supremum metric on B(X, Y) to C_{b}(X, Y). Suppose that Y is complete with respect to d_{Y}, so that B(X, Y) is complete with respect to the supremum metric, as in Section 1.12. Under these conditions, we get that

(1.14.3) C_{b}(X, Y) is complete as a metric space

with respect to the supremum metric, as in Section 1.7. This also uses (1.14.2).

Let us now take Y = R or C, with their standard Euclidean metrics. If f and g are continuous real or complex-valued functions on X, then it is well known that their sum f + g is continuous on X too. Similarly, if f is a continuous real or complex-valued function on X, and t ∈ R or C, as appropriate, then t f is continuous on X. This means that C(X, R) and C(X, C) are linear subspaces of the real and complex vector spaces of all real and complex-valued functions on X, respectively. We may also consider

C_{b}(X, R) = ℓ^{∞}(X, R) ∩ C(X, R) (1.14.4)

as a linear subspace of both ℓ^{∞}(X, R) and C(X, R), and

C_{b}(X, C) = ℓ^{∞}(X, C) ∩ C(X, C) (1.14.5)

as a linear subspace of both ℓ^{∞}(X, C) and C(X, C).

1.15 Continuous functions on [0, 1]

In this section, we take X to be the closed unit interval [0, 1] in the real line, equipped with the restriction of the standard Euclidean metric on R to [0, 1]. It is well known that [0, 1] is compact as a subset of R, and thus as a subset of itself. This means that every continuous real or complex-valued function f on [0, 1] is bounded, as before.

A nonnegative real-valued function N on C([0, 1], R) or C([0, 1], C) is said to be a norm if it satisfies the usual three conditions, as follows. First, N(f) = 0 if and only if f = 0. Second, if f is a continuous real or complex-valued function on [0, 1] and t ∈ R or C, as appropriate, then

N(t f) = |t| N(f). (1.15.1)

Third, if f and g are continuous real or complex-valued functions on [0, 1], then

N(f + g) ≤ N(f) + N(g). (1.15.2)

1.15. CONTINUOUS FUNCTIONS ON [0, 1] 21

In this case, d_{N}(f, g) = N(f − g) (1.15.3)

defines a metric on C([0, 1], R) or C([0, 1], C), as appropriate. The supremum norm (1.13.4) defines a norm on each of C([0, 1], R) and C([0, 1], C), for which the corresponding metric (1.13.8) is the supremum metric, as in Section 1.13.

If f is a continuous real or complex-valued function on [0, 1], then put

∥f∥_{1} = ∫_{0}^{1} |f(x)| dx. (1.15.4)

More precisely, it is well known and not difficult to verify that |f(x)| is also continuous on [0, 1], so that the Riemann integral on the right side of (1.15.4) exists. If f(x_{0}) ≠ 0 for some 0 ≤ x_{0} ≤ 1, then

|f(x)| ≥ |f(x_{0})|/2 > 0 (1.15.5)

when 0 ≤ x ≤ 1 is sufficiently close to x_{0}, because f is continuous at x_{0}. This implies that ∥f∥_{1} > 0, so that (1.15.4) satisfies the first condition in the definition of a norm. It is easy to see that (1.15.4) satisfies (1.15.1) and (1.15.2), so that (1.15.4) defines a norm on C([0, 1], R) and C([0, 1], C).

Similarly, put

∥f∥_{2} = ( ∫_{0}^{1} |f(x)|^{2} dx)^{1/2} (1.15.6)

for every continuous real or complex-valued function f on [0, 1]. One can check that (1.15.6) is equal to 0 exactly when f = 0 on [0, 1], using the same type of argument as in the preceding paragraph. It is easy to see that (1.15.6) satisfies the homogeneity condition (1.15.1). If g is another continuous real or complexvalued function on [0, 1], then it is well known that

∥f g∥_{1} ≤ ∥f∥_{2} ∥g∥_{2}, (1.15.7)

which is an integral version of the Cauchy–Schwarz inequality. This can be used to show that (1.15.6) satisfies the triangle inequality (1.15.2), as usual, so that (1.15.6) defines a norm on C([0, 1], R) and C([0, 1], C).

Clearly ∥f∥_{1}, ∥f∥_{2} ≤ ∥f∥_{∞} (1.15.8)

for every continuous real or complex-valued function f on [0, 1]. One can can also get that ∥f∥_{1} ≤ ∥f∥_{2}, (1.15.9)

using (1.15.7). Let d_{1}(f, g) = ∥f − g∥_{1} (1.15.10)

and d_{2}(f, g) = ∥f − g∥_{2} (1.15.11) be the metrics on C([0, 1], R) and C([0, 1], C) associated to (1.15.4) and (1.15.6) as in (1.15.3), respectively. Using (1.15.8) and (1.15.9), we get that

d_{1}(f, g) ≤ d_{2}(f, g) ≤ d_{∞}(f, g) (1.15.12)

for all continuous real and complex-valued functions f and g on [0, 1].

It follows that the identity mappings on C([0, 1], R) and C([0, 1], C) are Lipschitz with respect to d_{2}(f, g) on the domain and d_{1}(f, g) on the range. Similarly, the identity mappings on C([0, 1], R) and C([0, 1], C) are Lipschitz with respect to d_{∞}(f, g) on the domain and d_{2}(f, g) on the range. The identity mappings on C([0, 1], R) and C([0, 1], C) are also Lipschitz with respect to d_{∞}(f, g) on the domain and d_{1}(f, g) on the range.

Basic ℓ^{1} and ℓ^{2} spaces

This chapter deals with classical ℓ^{1} and ℓ^{2} spaces, of absolutely summable and square-summable sequences of real or complex numbers, respectively. We also consider c_{0} spaces of real or complex-valued functions on arbitrary nonempty sets that vanish at infinity. Sums over arbitrary nonempty sets and corresponding ℓ^{1}, ℓ^{2} spaces will be discussed in Chapter 11.

2.1 Infinite series

Remember that an infinite series

∑_{j=1}^{∞} a_{j} (2.1.1)

of real or complex numbers is said to converge if the corresponding sequence of partial sums

∑_{j=1}^{n} a_{j} (2.1.2)

converges with respect to the standard Euclidean metric on R or C, as appropriate. Of course, the value of the sum (2.1.1) is defined to be the limit of the sequence of partial sums (2.1.2) in this case. If (2.1.1) converges, and if ∑_{j=1}^{∞} b_{j} is another convergent series of real or complex numbers, as appropriate, then ∑_{j=1}^{∞}(a_{j} + b_{j}) converges, with

∑_{j=1}^{∞}(a_{j} + b_{j}) = ∑_{j=1}^{∞} a_{j} + ∑_{j=1}^{∞} b_{j}. (2.1.3)

24 CHAPTER 2. BASIC ℓ^{1} AND ℓ^{2} SPACES

This reduces to the corresponding statement for sums of convergent sequences of real or complex numbers, applied to the partial sums of these series. Similarly, if (2.1.1) converges, and t ∈ R or C, as appropriate, then ∑_{j=1}^{∞} t a_{j} converges, with

∑_{j=1}^{∞} t a_{j} = t ∑_{j=1}^{∞} a_{j}. (2.1.4)

If a_{j} is a nonnegative real number for each j ≥ 1, then the partial sums (2.1.2) increase monotonically. It is well known that a monotonically increasing sequence of real numbers converges with respect to the standard Euclidean metric on R if and only if the sequence has an upper bound in R, in which case the sequence converges to the supremum of the set of its terms. If a_{j} is a nonnegative real number for each j, but the partial sums (2.1.2) do not have an upper bound in R, then it is sometimes convenient to consider the value of the sum (2.1.1) to be +∞. Note that the partial sums (2.1.2) tend to +∞ as n → ∞ in this situation. If a_{j} and b_{j} are nonnegative real numbers for each j ≥ 1, then one can check that (2.1.3) holds, where the right side of (2.1.3) is considered to be +∞ when either of the individual sums is +∞. Similarly, (2.1.4) holds for every positive real number t, where the right side is considered to be +∞ when (2.1.1) is +∞. If a_{j} and b_{j} are nonnegative real numbers with

a_{j} ≤ b_{j} (2.1.5)

for every j ≥ 1, then

∑_{j=1}^{∞} a_{j} ≤ ∑_{j=1}^{∞} b_{j}, (2.1.6)

which is trivial when the right side if +∞.

An infinite series (2.1.1) of real or complex numbers is said to converge absolutely if

∑_{j=1}^{∞} |a_{j}| (2.1.7)

converges as an infinite series of nonnegative real numbers. This means that (2.1.7) is finite, in terms of the conventions for sums of nonnegative real numbers mentioned in the previous paragraph. If (2.1.1) converges absolutely, then it is well known that (2.1.1) converges in the usual sense. One can also check that

|∑_{j=1}^{∞} a_{j}| ≤ ∑_{j=1}^{∞} |a_{j}| (2.1.8)

under these conditions. This uses the fact that

|∑_{j=1}^{n} a_{j}| ≤ ∑_{j=1}^{n} |a_{j}| (2.1.9)

for every positive integer n, by the triangle inequality.

If (2.1.1) converges, then it is well known that {a_{j}}_{j=1}^{∞} converges to 0 as a sequence of real or complex numbers, as appropriate. In particular, this holds when (2.1.1) converges absolutely.

2.2. BASIC ℓ^{1} SPACES 25

2.2 Basic ℓ^{1} spaces

Let ℓ^{1}(Z_{+}, R) be the space of real-valued functions f on the set Z_{+} of positive integers such that

∥f∥_{1} = ∑_{j=1}^{∞} |f(j)| (2.2.1)

is finite, which is to say that the right side converges as an infinite series of nonnegative real numbers. Similarly, let ℓ^{1}(Z_{+}, C) be the space of complex-valued functions f on Z_{+} such that (2.2.1) is finite. In both cases, the convergence of the series on the right side of (2.2.1) implies that

lim_{j→∞} f(j) = 0, (2.2.2)

as mentioned in the previous section.

If f ∈ ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C), then (2.2.1) is a nonnegative real number, which is equal to 0 exactly when f(j) = 0 for every j ∈ Z_{+}. If t ∈ R or C, as appropriate, then t f(j) defines another real or complex-valued function on Z_{+}, as appropriate, and

∥tf∥_{1} = ∑_{j=1}^{∞} |t f(j)| = |t| ∥f∥_{1}. (2.2.3)

In particular, t f ∈ ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C), as appropriate.

Let g be another element of ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C), as appropriate. Thus f + g is a real or complex-valued function on Z_{+}, as appropriate, and

∥f + g∥_{1} = ∑_{j=1}^{∞} |f(j) + g(j)| ≤ ∑_{j=1}^{∞}(|f(j)| + |g(j)|) = ∥f∥_{1} + ∥g∥_{1}. (2.2.4)

This implies that f + g ∈ ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C), as appropriate. Hence ℓ^{1}(Z_{+}, R) and ℓ^{1}(Z_{+}, C) are linear subspaces of the real and complex vector spaces of all real and complex-valued functions on Z_{+}, respectively.

This means that ℓ^{1}(Z_{+}, R) and ℓ^{1}(Z_{+}, C) are vector spaces over the real and complex numbers, respectively, with respect to pointwise addition and scalar multiplication. We also get that (2.2.1) defines a norm on each of these spaces, by the remarks in the preceding paragraphs. Using (2.2.3) and (2.2.4), one can check that d_{1}(f, g) = ∥f − g∥_{1} (2.2.5)

defines a metric on each of ℓ^{1}(Z_{+}, R) and ℓ^{1}(Z_{+}, C), as usual.

If f ∈ ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C), then

∑_{j=1}^{∞} f(j) (2.2.6)

converges as an infinite series of real or complex numbers, as in the previous section. If g is another element of ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C), as appropriate, then

(2.2.7)  |∑_{j=1}^{∞} f(j) − ∑_{j=1}^{∞} g(j)| = |∑_{j=1}^{∞}(f(j) − g(j))|

≤ ∑_{j=1}^{∞} |f(j) − g(j)| = ∥f − g∥_{1},

using (2.1.8) in the second step. This implies that the mapping from f to the sum (2.2.6) is uniformly continuous as a mapping from ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z, C) into R or C, respectively, using the ℓ^{1} metric (2.2.5) on ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C), and the standard Euclidean metric on R or C.

If f ∈ ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C), then it is easy to see directly that f is bounded on Z_{+}, with ∥f∥_{∞} ≤ ∥f∥_{1}. (2.2.8)

Here ∥f∥_{∞} is the supremum norm of f on Z_{+}, as in (1.13.4). If g is another element of ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C), as appropriate, then we get that

d_{∞}(f, g) ≤ d_{1}(f, g), (2.2.9)

where d_{∞}(f, g) is the supremum metric for bounded real or complex-valued functions on Z_{+}, as in (1.13.8).

Remember that c_{00}(Z_{+}, R), c_{00}(Z_{+}, C) are the spaces of real and complexvalued functions on Z_{+} with finite support, respectively, as in Section 1.6. Clearly (2.2.10) c_{00}(Z_{+}, R) ⊆ ℓ^{1}(Z_{+}, R), c_{00}(Z_{+}, C) ⊆ ℓ^{1}(Z_{+}, C),

since an infinite series automatically converges when all but finitely many terms are equal to 0. One can also check that c_{00}(Z_{+}, R) and c_{00}(Z_{+}, C) are dense subsets of ℓ^{1}(Z_{+}, R) and ℓ^{1}(Z_{+}, C), respectively, with respect to the ℓ^{1} metric (2.2.5).

2.3 Basic ℓ^{2} spaces

Let ℓ^{2}(Z_{+}, R) and ℓ^{2}(Z_{+}, C) be the spaces of real and complex-valued functions f on Z_{+} such that

∑_{j=1}^{∞} |f(j)|^{2} (2.3.1)

converges as an infinite series of nonnegative real numbers, respectively. In both cases, we put

∥f∥_{2} = ( ∑_{j=1}^{∞} |f(j)|^{2})^{1/2}, (2.3.2)

using the nonnegative square root on the right side. Note that this is equal to 0 if and only if f(j) = 0 for every j ≥ 1. The convergence of the series (2.3.1) implies that

lim_{j→∞} |f(j)|^{2} = 0, (2.3.3)

and hence that f(j) → 0 as j → ∞.

Let f ∈ ℓ^{2}(Z_{+}, R) or ℓ^{2}(Z_{+}, C) be given. If t ∈ R or C, as appropriate, then

∑_{j=1}^{∞} |t f(j)|^{2} = |t|^{2} ∑_{j=1}^{∞} |f(j)|^{2}, (2.3.4)

and in particular the series on the left converges. This means that t f is an element of ℓ^{2}(Z_{+}, R) or ℓ^{2}(Z_{+}, C), as appropriate, with

∥t f∥_{2} = |t| ∥f∥_{2}. (2.3.5)

If a and b are nonnegative real numbers, then it is well known that

a b ≤ \frac{1}{2} (a^{2} + b^{2}), (2.3.6)

because 0 ≤ (a − b)^{2} = a^{2} − 2 a b + b^{2}. Suppose that g is another element of ℓ^{2}(Z_{+}, R) or ℓ^{2}(Z_{+}, C), as appropriate. Using (2.3.6), we get that

|f(j)| |g(j)| ≤ \frac{1}{2} (|f(j)|^{2} + |g(j)|^{2}) (2.3.7)

for every j ∈ Z_{+}. Hence

∑_{j=1}^{∞} |f(j)| |g(j)| ≤ ∑_{j=1}^{∞} \frac{1}{2} (|f(j)|^{2} + |g(j)|^{2})

(2.3.8)  = \frac{1}{2} ∑_{j=1}^{∞} |f(j)|^{2} + \frac{1}{2} ∑_{j=1}^{∞} |g(j)|^{2} = \frac{1}{2} ∥f∥_{2}^{2} + \frac{1}{2} ∥g∥_{2}^{2}.

In particular, the series on the left converges, so that f g ∈ ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C), as appropriate.

In fact, we have that

∑_{j=1}^{∞} |f(j)| |g(j)| ≤ ∥f∥_{2} ∥g∥_{2} (2.3.9)

under these conditions, which is another version of the Cauchy–Schwarz inequality. This follows from (2.3.8) when ∥f∥_{2} = ∥g∥_{2} = 1, and otherwise one can reduce to that case using (2.3.5).

Observe that

|f(j) + g(j)|^{2} ≤ (|f(j)| + |g(j)|)^{2} ≤ |f(j)|^{2} + 2 |f(j)| |g(j)| + |g(j)|^{2} (2.3.10) for every j ≥ 1, so that

∑_{j=1}^{∞} |f(j) + g(j)|^{2} ≤ ∑_{j=1}^{∞} |f(j)|^{2} + 2 ∑_{j=1}^{∞} |f(j)| |g(j)| + ∑_{j=1}^{∞} |g(j)|^{2}. (2.3.11)

This implies that the series on the left converges, so that f + g ∈ ℓ^{2}(Z_{+}, R) or ℓ^{2}(Z_{+}, C), as appropriate. Combining (2.3.9) and (2.3.11), we get that

∥f + g∥_{2}^{2} ≤ ∥f∥_{2}^{2} + 2 ∥f∥_{2} ∥g∥_{2} + ∥g∥_{2}^{2} = (∥f∥_{2} + ∥g∥_{2})^{2}, (2.3.12)

so that ∥f + g∥_{2} ≤ ∥f∥_{2} + ∥g∥_{2}. (2.3.13)

This shows that ℓ^{2}(Z_{+}, R) and ℓ^{2}(Z_{+}, C) are linear subspaces of the real and complex vector spaces of all real and complex-valued functions on Z_{+}, respectively. We also get that (2.3.2) defines a norm on each of ℓ^{2}(Z_{+}, R) and ℓ^{2}(Z_{+}, C), by (2.3.5) and (2.3.13). It follows that

d_{2}(f, g) = ∥f − g∥_{2} (2.3.14)

defines a metric on each of ℓ^{2}(Z_{+}, R) and ℓ^{2}(Z_{+}, C), as usual.

If f ∈ ℓ^{2}(Z_{+}, R) or ℓ^{2}(Z_{+}, C), then one can check directly that f is bounded on Z_{+}, with ∥f∥_{∞} ≤ ∥f∥_{2}. (2.3.15)

If g is another element of ℓ^{2}(Z_{+}, R) or ℓ^{2}(Z_{+}, C), as appropriate, then we get that d_{∞}(f, g) ≤ d_{2}(f, g). (2.3.16)

Suppose now that f ∈ ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C). Observe that

∑_{j=1}^{∞} |f(j)|^{2} ≤ ∥f∥_{∞} ∑_{j=1}^{∞} |f(j)| = ∥f∥_{∞} ∥f∥_{1} ≤ ∥f∥_{1}^{2}, (2.3.17)

using (2.2.8) in the third step. This implies that f ∈ ℓ^{2}(Z_{+}, R) or ℓ^{2}(Z_{+}, C), as appropriate, with ∥f∥_{2} ≤ ∥f∥_{1}. (2.3.18)

In particular, we get that

(2.3.19) ℓ^{1}(Z_{+}, R) ⊆ ℓ^{2}(Z_{+}, R), ℓ^{1}(Z_{+}, C) ⊆ ℓ^{2}(Z_{+}, C).

If g is another element of ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C), as appropriate, then it follows that d_{2}(f, g) ≤ d_{1}(f, g). (2.3.20)

If f is a real or complex-valued function on Z_{+} with finite support, then f is clearly an element of ℓ^{2}(Z_{+}, R) or ℓ^{2}(Z_{+}, C), as appropriate. One can verify that c_{00}(Z_{+}, R) and c_{00}(Z_{+}, C) are dense subsets of ℓ^{2}(Z_{+}, R) and ℓ^{2}(Z_{+}, C), respectively, with respect to the ℓ^{2} metric (2.3.14).

2.4. COMPLETENESS OF ℓ^{1}, ℓ^{2} 29

2.4 Completeness of ℓ^{1}, ℓ^{2}

Let {f_{l}}_{l=1}^{∞} be a sequence of elements of ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C), and suppose that the f_{l}’s have bounded ℓ^{1} norms, so that there is a nonnegative real number C_{1} such that

∑_{j=1}^{∞} |f_{l}(j)| ≤ C_{1} (2.4.1)

for every l ≥ 1. Suppose also that {f_{l}}_{l=1}^{∞} converges to a real or complex-valued function f pointwise on Z_{+}, as appropriate. Let us verify that f ∈ ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C), as appropriate, with

∑_{j=1}^{∞} |f(j)| ≤ C_{1}. (2.4.2)

If n ∈ Z_{+}, then

∑_{j=1}^{n} |f(j)| = lim_{l→∞} ∑_{j=1}^{n} |f_{l}(j)| ≤ C_{1}, (2.4.3)

using pointwise convergence in the first step, and (2.4.1) in the second step. This implies (2.4.2), since this estimate holds for all n ≥ 1.

We would like to show that ℓ^{1}(Z_{+}, R) and ℓ^{1}(Z_{+}, C) are complete with respect to the ℓ^{1} metric (2.2.5). Let {f_{l}}_{l=1}^{∞} be a sequence of elements of ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C) that is a Cauchy sequence with respect to (2.2.5). This implies that for every ϵ > 0 there is a positive integer L(ϵ) such that

∑_{j=1}^{∞} |f_{k}(j) − f_{l}(j)| = ∥f_{k} − f_{l}∥_{1} < ϵ (2.4.4)

for all k, l ≥ L(ϵ). Hence |f_{k}(j) − f_{l}(j)| < ϵ (2.4.5)

for every j ∈ Z_{+} when k, l ≥ L(ϵ). This means that {f_{l}(j)}_{l=1}^{∞} is a Cauchy sequence of real or complex numbers, as appropriate, for every j ∈ Z_{+}, and with respect to the standard Euclidean metric on R or C. Remember that R and C are complete as metric spaces with respect to their standard Euclidean metrics. It follows that {f_{l}(j)}_{l=1}^{∞} converges in R or C, as appropriate, for every j ∈ Z_{+}, with respect to the standard Euclidean metric. Thus

f(j) = lim_{l→∞} f_{l}(j) (2.4.6)

defines a real or complex-valued function on Z_{+}, as appropriate. We would like to check that ∑_{j=1}^{∞} |f(j)| converges, so that f is an element of ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C), as appropriate. We would also like to verify that {f_{l}}_{l=1}^{∞} converges to f with respect to the ℓ^{1} metric.

Let ϵ > 0 and l ≥ L(ϵ) be given. Note that {f_{k} − f_{l}}_{k=L(ϵ)}^{∞} is a sequence of elements of ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C), as appropriate, that converges to f − f_{l} pointwise on Z_{+}. The remarks at the beginning of the section imply that f − f_{l} is an element of ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C), as appropriate, with

∥f − f_{l}∥_{1} = ∑_{j=1}^{∞} |f(j) − f_{l}(j)| ≤ ϵ, (2.4.7)

because of (2.4.4). Hence f ∈ ℓ^{1}(Z_{+}, R) or ℓ^{1}(Z_{+}, C), as appropriate, because of the corresponding property of f_{l}. It is easy to see that {f_{l}}_{l=1}^{∞} converges to f with respect to the ℓ^{1} metric, because (2.4.7) holds for every l ≥ L(ϵ).

Now let {f_{l}}_{l=1}^{∞} be a sequence of elements of ℓ^{2}(Z_{+}, R) or ℓ^{2}(Z_{+}, C) with bounded ℓ^{2} norms, so that ∥f_{l}∥_{2} ≤ C_{2} (2.4.8)

for some C_{2} ≥ 0 and every l ≥ 1. This is the same as saying that

∑_{j=1}^{∞} |f_{l}(j)|^{2} ≤ C_{2}^{2} (2.4.9)

for every l ≥ 1. Suppose that {f_{l}}_{l=1}^{∞} also converges pointwise to a real or complex-valued function f on Z_{+}, which implies that {|f_{l}|^{2}}_{l=1}^{∞} converges to |f|^{2} pointwise on Z_{+} too. It follows that

∑_{j=1}^{∞} |f(j)|^{2} ≤ C_{2}^{2}, (2.4.10)

by the remarks at the beginning of the section, applied to {|f_{l}|^{2}}_{l=1}^{∞}. This means that f ∈ ℓ^{2}(Z_{+}, R) or ℓ^{2}(Z_{+}, C), as appropriate, with

∥f∥_{2} ≤ C_{2}. (2.4.11)

Using this, one can show that ℓ^{2}(Z_{+}, R) and ℓ^{2}(Z_{+}, C) are complete with respect to the ℓ^{2} metric (2.3.14). The argument is similar to the previous one for ℓ^{1}(Z_{+}, R), ℓ^{1}(Z_{+}, C).

2.5 Vanishing at infinity

Let X be a nonempty set, and let f be a real or complex-valued function on X. We say that f vanishes at infinity on X if for every ϵ > 0,

|f(x)| < ϵ (2.5.1)

for all but finitely many x ∈ X. Equivalently, this means that for each ϵ > 0,

E_{ϵ}(f) = {x ∈ X : |f(x)| ≥ ϵ} (2.5.2)

has only finitely many elements. Let c_{0}(X, R) and c_{0}(X, C) be the spaces of real and complex-valued functions on X that vanish at infinity, respectively.

Remember that c_{00}(X, R) and c_{00}(X, C) denote the spaces of real and complexvalued functions f on X such that the support of f has only finitely many elements, respectively, as in Section 1.6. In this case, f automatically vanishes at infinity on X, so that

c_{00}(X, R) ⊆ c_{0}(X, R), c_{00}(X, C) ⊆ c_{0}(X, C). (2.5.3)

In particular, if X has only finitely many elements, then every real or complexvalued function on X vanishes at infinity.

If f is a real or complex-valued function on the set Z_{+} of positive integers, then f vanishes at infinity on Z_{+} if and only if

lim_{j→∞} f(j) = 0. (2.5.4)

Let X be any nonempty set again, and let {x_{j}}_{j=1}^{∞} be an infinite sequence of distinct elements of X. Also let f be a real or complex-valued function on X, and suppose that the support of f is contained in the set of x_{j}’s, j ∈ Z_{+}. Under these conditions, f vanishes at infinity on X if and only if

lim_{j→∞} f(x_{j}) = 0. (2.5.5)

Let f be any real or complex-valued function on X, and remember that the support of f is the set of x ∈ X such that f(x) ≠ 0. Equivalently,

supp f = ⋃_{n=1}^{∞} E_{1/n}(f), (2.5.6)

where E_{1/n}(f) is as in (2.5.2). If f vanishes at infinity on X, then it follows that the support of f has only finitely or countably many elements.

If f is a real or complex-valued function on X that vanishes at infinity, then it is easy to see that f is bounded on X. In fact, we have that

∥f∥_{∞} = max_{x∈X} |f(x)|, (2.5.7)

which is to say that the maximum on the right is attained. Of course, this is trivial when f ≡ 0 on X. Otherwise, if f(x_{0}) ≠ 0 for some x_{0} ∈ X, then there are only finitely many x ∈ X such that |f(x)| ≥ |f(x_{0})|. In this case, it suffices to take the maximum of |f(x)| over this finite set.

Thus c_{0}(X, R) ⊆ ℓ^{∞}(X, R), c_{0}(X, C) ⊆ ℓ^{∞}(X, C). (2.5.8)

If g is another real or complex-valued function on X that vanishes at infinity, then one can check that f + g vanishes at infinity on X as well. Similarly, if t ∈ R or C, as appropriate, then t f vanishes at infinity on X. This means that c_{0}(X, R) and c_{0}(X, C) are linear subspaces of ℓ^{∞}(X, R) and ℓ^{∞}(X, C), respectively.

Let f be a bounded real or complex-valued function on X. Suppose that f is a limit point of c_{0}(X, R) or c_{0}(X, C) in ℓ^{∞}(X, R) or ℓ^{∞}(X, C), respectively, with respect to the supremum metric. We would like to verify that f also vanishes at infinity on X in this case. Let ϵ > 0 be given. By hypothesis, there is a real or complex-valued function g on X, as appropriate, such that g vanishes at infinity on X, and ∥f − g∥_{∞} < ϵ/2. (2.5.9)

This implies that

|f(x)| ≤ |g(x)| + |f(x) − g(x)| ≤ |g(x)| + ∥f − g∥_{∞} < |g(x)| + ϵ/2 (2.5.10)

for every x ∈ X. Of course, |g(x)| < ϵ/2 for all but finitely many x ∈ X, because g vanishes at infinity on X. It follows that

|f(x)| < ϵ/2 + ϵ/2 = ϵ (2.5.11)

for all but finitely many x ∈ X, as desired.

This shows that c_{0}(X, R) and c_{0}(X, C) are closed sets in ℓ^{∞}(X, R) and ℓ^{∞}(X, C), respectively, with respect to the supremum metric. As a slightly different version of this, let {f_{j}}_{j=1}^{∞} be a sequence of real or complex-valued functions on X that vanish at infinity and converge uniformly to a real or complex-valued function f on X, as appropriate. Under these conditions, f vanishes at infinity on X too. This can be obtained from the previous statement, or using the same argument as in the preceding paragraph.

Let f be a real or complex-valued function on X that vanishes at infinity, and let ϵ > 0 be given. Let f_{ϵ} be the real or complex-valued function, as appropriate, defined on X by

(2.5.12)  f_{ϵ}(x) = f(x) when |f(x)| ≥ ϵ
= 0  when |f(x)| < ϵ.

Note that f_{ϵ} has finite support in X, because f vanishes at infinity on X. By construction, ∥f − f_{ϵ}∥_{∞} < ϵ, (2.5.13)

where the strict inequality uses the hypothesis that f vanish at infinity on X again. This shows that c_{00}(X, R) and c_{00}(X, C) are dense in c_{0}(X, R) and c_{0}(X, C), respectively, with respect to the supremum metric.

Some more metric spaces

3.1 Functions on intervals

Let a, b be real numbers with a < b, and let

[a, b] = {x ∈ R : a ≤ x ≤ b} (3.1.1)

be the usual closed interval in the real line from a to b. It is well known that [a, b] is compact with respect to the standard metric on R. As in Section 1.14, C([a, b], R) and C([a, b], C) are the spaces of continuous real and complex-valued functions on [a, b]. This uses the restriction of the standard metric on R to [a, b], and the standard Euclidean metrics on R and C, as appropriate. As before, these spaces are linear subspaces of the real and complex vector spaces of all real and complex-valued functions on [a, b], as appropriate.

As usual, a nonnegative real-valued function N on C([a, b], R) or C([a, b], C) is said to be a norm if it satisfies the following three conditions. First, N(f) = 0 if and only if f = 0. Second, if f is a continuous real or complex-valued function on [a, b] and t ∈ R of C, as appropriate, then

N(t f) = |t| N(f). (3.1.2)

Third, if f and g are continuous real or complex-valued functions on [a, b], as appropriate, then N(f + g) ≤ N(f) + N(g). (3.1.3)

In this case, d_{N}(f, g) = N(f − g) (3.1.4)

defines a metric on C([a, b], R) or C([a, b], C), as appropriate.

Remember that continuous real and complex-valued functions on [a, b] are bounded, because [a, b] is compact. The supremum norm is defined on each of C([a, b], R) and C([a, b], C) by

∥f∥_{∞} = sup_{a≤x≤b} |f(x)|, (3.1.5)

as in (1.13.4). Note that the supremum is attained in this situation, because of the extreme value theorem. The corresponding metric

d_{∞}(f, g) = ∥f − g∥_{∞} (3.1.6)

is the same as the supremum metric, as before.

If f is a continuous real or complex-valued function on [a, b], then put

∥f∥_{1} = ∫_{a}^{b} |f(x)| dx, (3.1.7)

using the standard Riemann integral on the right side. This is the same as (1.15.4) in the case of the unit interval in R. As before, one can check that (3.1.7) defines a norm on each of C([a, b], R) and C([a, b], C). Let

d_{1}(f, g) = ∥f − g∥_{1} = ∫_{a}^{b} |f(x) − g(x)| dx (3.1.8)

be the metric associated to (3.1.7) on each of C([a, b], R) and C([a, b], C). If f is a continuous real or complex-valued function on [a, b], then

∥f∥_{1} = ∫_{a}^{b} |f(x)| dx ≤ (b − a) ∥f∥_{∞}. (3.1.9)

Hence d_{1}(f, g) = ∥f − g∥_{1} ≤ (b − a) ∥f − g∥_{∞} = (b − a) d_{∞}(f, g) (3.1.10)

for all real or complex-valued continuous functions f and g on [a, b]. This implies that the identity mappings on C([a, b], R) and C([a, b], C) are Lipschitz with respect to d_{∞}(f, g) on the domain and d_{1}(f, g) on the range.

3.2 The square norm

Let a, b be real numbers with a < b again. Put

∥f∥_{2} = ( ∫_{a}^{b} |f(x)|^{2} dx)^{1/2} (3.2.1)

for each continuous real or complex-valued continuous function f on [a, b], using the nonnegative square root on the right side, as usual. Note that this is the same as (1.15.6) in the case of the unit interval. It is easy to see that this satisfies the homogeneity and positivity requirements of a norm, as before. To get the triangle inequality, one can use an integral version of the Cauchy–Schwarz inequality, as follows.

Let f, g be continuous real or complex-valued functions on [a, b]. Observe that

|f(x)| |g(x)| ≤ \frac{1}{2} |f(x)|^{2} + \frac{1}{2} |g(x)|^{2} (3.2.2)

for every a ≤ x ≤ b, as in (2.3.6). This implies that

(3.2.3) ∫_{a}^{b} |f(x)| |g(x)| dx ≤ \frac{1}{2} ∫_{a}^{b} |f(x)|^{2} dx + \frac{1}{2} ∫_{a}^{b} |g(x)|^{2} dx
= \frac{1}{2} ∥f∥_{2}^{2} + \frac{1}{2} ∥g∥_{2}^{2}.

Using this, we can get that

∫_{a}^{b} |f(x)| |g(x)| dx ≤ ∥f∥_{2} ∥g∥_{2}, (3.2.4)

which is the integral version of the Cauchy–Schwarz inequality mentioned in the preceding paragraph. More precisely, (3.2.4) follows from (3.2.3) when ∥f∥_{2} = ∥g∥_{2} = 1. If ∥f∥_{2}, ∥g∥_{2} > 0, then one can reduce to the previous case, using scalar multiplication. Otherwise, (3.2.4) is trivial when f = 0 or g = 0 on [a, b].

Clearly

∥f + g∥_{2}^{2} = ∫_{a}^{b} |f(x) + g(x)|^{2} dx
(3.2.5)  ≤ ∫_{a}^{b}(|f(x)| + |g(x)|)^{2} dx
= ∫_{a}^{b} |f(x)|^{2} dx + 2 ∫_{a}^{b} |f(x)| |g(x)| dx + ∫_{a}^{b} |g(x)|^{2} dx.

It follows that

∥f + g∥_{2}^{2} ≤ ∥f∥_{2}^{2} + 2 ∥f∥_{2} ∥g∥_{2} + ∥g∥_{2}^{2} = (∥f∥_{2} + ∥g∥_{2})^{2}, (3.2.6)

using (3.2.4) in the first step. Thus

∥f + g∥_{2} ≤ ∥f∥_{2} + ∥g∥_{2}, (3.2.7)

so that (3.2.1) defines a norm on each of C([a, b], R) and C([a, b], C). The associated metric is given by

d_{2}(f, g) = ∥f − g∥_{2} = ( ∫_{a}^{b} |f(x) − g(x)|^{2} dx)^{1/2}. (3.2.8)

If f is a continuous real or complex-valued function on [a, b], then

∥f∥_{2}^{2} = ∫_{a}^{b} |f(x)|^{2} dx ≤ (b − a) ∥f∥_{∞}^{2}. (3.2.9)

Equivalently, ∥f∥_{2} ≤ (b − a)^{1/2} ∥f∥_{∞}. (3.2.10)

This implies that d_{2}(f, g) ≤ (b − a)^{1/2} d_{∞}(f, g) (3.2.11) for all real or complex-valued continuous functions f and g on [a, b]. In particular, the identity mappings on C([a, b], R) and C([a, b], C) are Lipschitz with respect to d_{∞}(f, g) on the domain and d_{2}(f, g) on the range.

Observe that

∥f∥_{1} = ∫_{a}^{b} |f(x)| dx ≤ (b − a)^{1/2} ∥f∥_{2} (3.2.12)

for every continuous real or complex-valued function f on [a, b], by (3.2.4). Hence d_{1}(f, g) ≤ (b − a)^{1/2} d_{2}(f, g) (3.2.13)

for all continuous real or complex-valued functions f and g on [a, b]. It follows that the identity mappings on C([a, b], R) and C([a, b], C) are Lipschitz with respect to d_{2}(f, g) on the domain and d_{1}(f, g) on the range.

3.3 Riemann–Stieltjes integrals

Let a, b be real numbers with a < b, and let α be a monotonically increasing real-valued function on [a, b]. If f is a continuous real-valued function on [a, b], then the corresponding Riemann–Stieltjes integral

∫_{a}^{b} f dα = ∫_{a}^{b} f(x) dα(x) (3.3.1)

can be defined as a real number in a natural way. This reduces to the ordinary Riemann integral of f on [a, b] when α(x) = x for every x ∈ [a, b].

To be a bit more precise, let P = {t_{j}}_{j=0}^{l} be a partition of [a, b]. This is a finite sequence of real numbers with

a = t_{0} < t_{1} < ⋯ < t_{l−1} < t_{l} = b, (3.3.2)

as usual. In the definition of the Riemann integral, one considers certain sums associated to such a partition, involving the values of f on each interval [t_{j−1}, t_{j}], and the length of the interval. In the definition of the Riemann–Stieltjes integral, one considers sums of a similar type, but using

α(t_{j}) − α(t_{j−1}) (3.3.3)

instead of the length of [t_{j−1}, t_{j}]. Note that (3.3.3) is greater than or equal to 0 for each j, and that

∑_{j=1}^{l}(α(t_{j}) − α(t_{j−1})) = α(b) − α(a). (3.3.4)

Riemann–Stieltjes integrals are discussed in many textbooks, as well as the article [86]. See [28] for a geometric interpretation of the Riemann–Stieltjes integral, and [153, 177] for some variants of the Riemann–Stieltjes integral.

The Riemann–Stieltjes integral has many of the same properties as the Riemann integral. In particular, (3.3.1) is greater than or equal to 0 when f ≥ 0 on [a, b]. In this case, if f(x_{0}) > 0 for some x_{0} ∈ [a, b], and if α is not constant near x_{0}, then (3.3.1) is positive. If α is strictly increasing on [a, b], then this works for any x_{0} ∈ [a, b]. If α is constant on [a, b], then (3.3.1) is equal to 0 for any function f on [a, b].

If α is continuously-differentiable on [a, b], then it is well known that

∫_{a}^{b} f(x) dα(x) = ∫_{a}^{b} f(x) α′(x) dx. (3.3.5)

In fact, this works when α is differentiable at every point in [a, b], and its derivative α′ is Riemann integrable on [a, b]. Although monotonically increasing functions on [a, b] are not necessarily continuous, it is well known that their only possible discontinuities are jump discontinuities. It is also well known that such a function can have only finitely or countably many discontinuities in [a, b]. It is interesting to consider the Riemann–Stieltjes integral when α is a monotonically increasing step function on [a, b], for instance.

It is well known that the Riemann integral of a bounded real-valued function f on [a, b] can be defined when f is Riemann integrable on [a, b], in a suitable sense, which includes the case where f is continuous on [a, b]. Similarly, (3.3.1) can be defined when f is Riemann–Stieltjes integrable on [a, b] with respect to α, in a suitable sense, which includes the case when f is continuous on [a, b]. If f is a complex-valued function on [a, b], then one can consider the corresponding integrability properties of the real and imaginary parts of f. If the real and imaginary parts of f are Riemann–Stieltjes integrable on [a, b] with respect to α, then one can define the Riemann–Stieltjes integral of f as a complex number, whose real and imaginary parts are the corresponding Riemann–Stieltjes integrals of the real and imaginary parts of f. In particular, this works when f is continuous on [a, b].

3.4 Riemann–Stieltjes integrals and seminorms

Let a, b be real numbers with a < b, and let N be a nonnegative real-valued function on C([a, b], R) or C([a, b], C). As in Section A.6, N is said to be a seminorm if

N(t f) = |t| N(f) (3.4.1)

and

N(f + g) ≤ N(f) + N(g) (3.4.2)

for every f, g ∈ C([a, b], R) or C([a, b], C) and t ∈ R or C, as appropriate. In this case, N(0) = 0, by taking t = 0 in (3.4.1). If we also have that N(f) > 0 when f ≢ 0 on [a, b], then N is a norm on C([a, b], R) or C([a, b], C), as appropriate.

Let α be a monotonically increasing real-valued function on [a, b]. If f is a continuous real or complex-valued function on [a, b], then

∥f∥_{1,α} = ∫_{a}^{b} |f(x)| dα(x) (3.4.3)

is defined as a nonnegative real number, as in the previous section. Of course, this is the same as (3.1.7) when α(x) = x on [a, b]. It is easy to see that

∥t f∥_{1,α} = |t| ∥f∥_{1,α} (3.4.4)

and ∥f + g∥_{1,α} ≤ ∥f∥_{1,α} + ∥g∥_{1,α} (3.4.5)

for all continuous real or complex-valued functions f and g on [a, b] and t ∈ R or C, as appropriate. Thus (3.4.3) defines a seminorm on each of C([a, b], R) and C([a, b], C).

If α is strictly increasing on [a, b], and if f is a continuous real or complexvalued function on [a, b] such that f(x_{0}) ≠ 0 for some x_{0} ∈ [a, b], then one can check that (3.4.3) is positive, as before. This implies that (3.4.3) defines a norm on each of C([a, b], R) and C([a, b], C) in this case. It follows that

d_{1,α}(f, g) = ∥f − g∥_{1,α} (3.4.6)

defines a metric on each of C([a, b], R) and C([a, b], C) under these conditions. If f, g are continuous real or complex-valued function on [a, b], then

|∫_{a}^{b} f dα − ∫_{a}^{b} g dα| = |∫_{a}^{b}(f − g) dα|
(3.4.7)  ≤ ∫_{a}^{b} |f − g| dα = ∥f − g∥_{1,α},

using basic properties of the Riemann–Stieltjes integral in the first two steps. Suppose for the moment that α is strictly increasing on [a, b]. It follows from (3.4.7) that

f ↦ ∫_{a}^{b} f dα (3.4.8)

is Lipschitz as a mapping from C([a, b], R) or C([a, b], C) into R or C, as appropriate, with respect to (3.4.6) on the domain and the standard Euclidean metric on the range.

If f is a continuous real or complex-valued function on [a, b], then

∥f∥_{1,α} ≤ (α(b) − α(a)) ∥f∥_{∞}, (3.4.9)

where ∥f∥_{∞} is the supremum norm of f on [a, b], as in (3.1.5). Combining (3.4.7) and (3.4.9), we get that

|∫_{a}^{b} f dα − ∫_{a}^{b} g dα| ≤ (α(b) − α(a)) ∥f − g∥_{∞}, (3.4.10) for all continuous real or complex-valued functions f, g on [a, b]. This implies that (3.4.8) is Lipschitz as a mapping from C([a, b], R) or C([a, b], C) into R or C, as appropriate, with respect to the supremum metric on the domain and the standard Euclidean metric on the range.

Let {f_{j}}_{j=1}^{∞} be a sequence of real or complex-valued functions on [a, b] that converges uniformly to a real or complex-valued function f on [a, b], as appropriate. If f_{j} is continuous on [a, b] for each j, then f is continuous on [a, b], and ∥f_{j} − f∥_{1,α} → 0 as j → ∞, (3.4.11)

by (3.4.9). This means that {f_{j}}_{j=1}^{∞} converges to f with respect to (3.4.6) when α is strictly increasing on [a, b]. We also get that

lim_{j→∞} ∫_{a}^{b} f_{j}(x) dα(x) = ∫_{a}^{b} f(x) dα(x), (3.4.12)

by (3.4.10).

More precisely, if f_{j} is Riemann–Stieltjes integrable on [a, b] with respect to α for each j, then it is well known that f is Riemann–Stieltjes integrable on [a, b] with respect to α too. In this case, (3.4.12) holds for essentially the same reasons as before. Similarly, one can define (3.4.3) for Riemann–Stieltjes integrable functions on [a, b] with respect to α, and (3.4.11) holds as well.

3.5 Square seminorms

Let a, b be real numbers with a < b, and let α be a monotonically increasing realvalued function on [a, b]. If f is a continuous real or complex-valued function on [a, b], then

∥f∥_{2,α} = ( ∫_{a}^{b} |f(x)|^{2} dα(x))^{1/2} (3.5.1)

is defined as a nonnegative real number. This reduces to (3.2.1) when α(x) = x on [a, b], as before. Note that

∥t f∥_{2,α} = |t| ∥f∥_{2,α} (3.5.2)

for every t ∈ R or C, as appropriate.

Let g be another continuous real or complex-valued function on [a, b]. Observe that

∫_{a}^{b} |f(x)| |g(x)| dα(x) ≤ \frac{1}{2} ∫_{a}^{b} |f(x)|^{2} dα(x) + \frac{1}{2} ∫_{a}^{b} |g(x)|^{2} dα(x)
(3.5.3)  = \frac{1}{2} ∥f∥_{2,α}^{2} + \frac{1}{2} ∥g∥_{2,α}^{2},

using (3.2.2) in the first step. In particular,

∫_{a}^{b} |f(x)| |g(x)| dα(x) ≤ 1 (3.5.4) when ∥f∥_{2,α}, ∥g∥_{2,α} ≤ 1.

Let f, g be arbitrary continuous real or complex-valued functions on [a, b], and let r, t be positive real numbers such that

∥f∥_{2,α} ≤ r, ∥g∥_{2,α} ≤ t. (3.5.5)

Thus ∥r^{−1} f∥_{2,α}, ∥t^{−1} g∥_{2,α} ≤ 1, so that

∫_{a}^{b} |r^{−1} f(x)| |t^{−1} g(x)| dα(x) ≤ 1. (3.5.6)

Equivalently, this means that

∫_{a}^{b} |f(x)| |g(x)| dα(x) ≤ r t. (3.5.7)

One can use this to get that

∫_{a}^{b} |f(x)| |g(x)| dα(x) ≤ ∥f∥_{2,α} ∥g∥_{2,α}, (3.5.8)

which is the integral version of the Cauchy–Schwarz inequality for Riemann– Stieltjes integrals on [a, b].

As in Section 3.2, one can show that

∥f + g∥_{2,α} ≤ ∥f∥_{2,α} + ∥g∥_{2,α}, (3.5.9)

using (3.5.8). This implies that (3.5.1) defines a seminorm on each of C([a, b], R) and C([a, b], C). If α is strictly increasing on [a, b], and if f is a continuous real or complex-valued function on [a, b] that is not equal to 0 everywhere on [a, b], then one can verify that (3.5.1) is positive, as usual. In this case, (3.5.1) defines a norm on each of C([a, b], R) and C([a, b], C). Thus

d_{2,α}(f, g) = ∥f − g∥_{2,α} (3.5.10)

defines a metric on each of these spaces in this situation.

Observe that ∥f∥_{2,α} ≤ (α(b) − α(a))^{1/2} ∥f∥_{∞} (3.5.11)

for every continuous real or complex-valued function f on [a, b]. Of course, this implies that ∥f − g∥_{2,α} ≤ (α(b) − α(a))^{1/2} ∥f − g∥_{∞} (3.5.12)

for all continuous real or complex-valued functions f, g on [a, b]. Similarly,

∥f∥_{1,α} ≤ (α(b) − α(a))^{1/2} ∥f∥_{2,α}, (3.5.13)

by (3.5.8). It follows that

∥f − g∥_{1,α} ≤ (α(b) − α(a))^{1/2} ∥f − g∥_{2,α} (3.5.14)

for every f, g ∈ C([a, b], R) or C([a, b], C).

3.6 Some more monotone functions

Let a, b be real numbers with a < b, and let α, β be monotonically increasing functions real-valued functions on [a, b]. Thus α+β is a monotonically increasing real-valued function on [a, b] as well. If f is a continuous real-valued function on [a, b], then the Riemann–Stieltjes integrals of f on [a, b] with respect to α, β, α + β may be defined in the usual way. It is well known that

∫_{a}^{b} f d(α + β) = ∫_{a}^{b} f dα + ∫_{a}^{b} f dβ. (3.6.1)

In particular, if f ≥ 0 on [a, b], then

∫_{a}^{b} f dα, ∫_{a}^{b} f dβ ≤ ∫_{a}^{b} f d(α + β). (3.6.2)

Suppose now that f is a continuous real or complex-valued function on [a, b], so that ∥f∥_{1,α} and ∥f∥_{2,α} may be defined as in the previous two sections. Of course, the analogous quantities

∥f∥_{1,β}, ∥f∥_{2,β}, ∥f∥_{1,α+β}, ∥f∥_{2,α+β} (3.6.3)

associated to β and α + β may be defined in the same way. Observe that

∥f∥_{1,α+β} = ∥f∥_{1,α} + ∥f∥_{1,β}, (3.6.4)

because of (3.6.1). Similarly,

∥f∥_{2,α+β}^{2} = ∥f∥_{2,α}^{2} + ∥f∥_{2,β}^{2}. (3.6.5)

It follows that ∥f∥_{1,α}, ∥f∥_{1,β} ≤ ∥f∥_{1,α+β} (3.6.6)

and ∥f∥_{2,α}, ∥f∥_{2,β} ≤ ∥f∥_{2,α+β} (3.6.7)

Let t be a nonnegative real number, so that t α is a monotonically increasing real-valued function on [a, b] too. If f is a continuous real-valued function on [a, b], then the Riemann–Stieltjes integral of f on [a, b] with respect to t α may be defined in the usual way. It is well known that

∫_{a}^{b} f d(t α) = t ∫_{a}^{b} f dα. (3.6.8)

If f is a continuous real or complex-valued function on [a, b], then

∥f∥_{1,t α} and ∥f∥_{2,t α} (3.6.9)

may be defined as in the two previous sections again. It is easy to see that

∥f∥_{1,t α} = t ∥f∥_{1,α} (3.6.10)

and ∥f∥_{2,t α} = \sqrt{t} ∥f∥_{2,α}, (3.6.11)

using (3.6.8).

3.7 Compact support

Let (X, d(x, y)) be a metric space, and let f be a real or complex-valued function on X. The support of f in X is defined by

supp f = \overline{\{x ∈ X : f(x) ≠ 0\}}, (3.7.1)

which is to say the closure of the set of x ∈ X with f(x) ≠ 0 with respect to d(·, ·). Note that this is different from the definition of the support used in Section 1.6.

Suppose for the moment that d(x, y) is the discrete metric on X, which is equal to 1 when x ≠ y, and to 0 when x = y. In this case, it is easy to see that every subset of X is a closed set. Similarly, the closure of any subset E of X is the same as E. This means that (3.7.1) is the same as the previous definition of the support in Section 1.6 under these conditions.

If t ∈ R or C, as appropriate, then

(3.7.2) {x ∈ X : t f(x) ≠ 0} = {x ∈ X : f(x) ≠ 0} when t ≠ 0
= ∅  when t = 0.

This implies that

(3.7.3)  supp(t f) = supp f when t ≠ 0
= ∅  when t = 0.

If g is another real or complex-valued function on X, then it is easy to see that

{x ∈ X : f(x) + g(x) ≠ 0} ⊆ {x ∈ X : f(x) ≠ 0} ∪ {x ∈ X : g(x) ≠ 0}. (3.7.4)

This implies that (3.7.5) supp(f + g) ⊆ (supp f) ∪ (supp g).

More precisely, this also uses the well-known fact that

\overline{(E_{1} ∪ E_{2})} ⊆ \overline{E_{1}} ∪ \overline{E_{2}} (3.7.6)

for any two subsets E_{1}, E_{2} of X.

Of course, f is said to have compact support in X if the support of f is a compact subset of X. Suppose that there is a compact subset K of X such that

{x ∈ X : f(x) ≠ 0} ⊆ K. (3.7.7)

This implies that supp f ⊆ K, (3.7.8)

because compact subsets of metric spaces are closed sets. If E is a closed subset of X such that E ⊆ K, then it is well known that E is also compact in X. Thus (3.7.8) implies that supp f is compact, because supp f is a closed set in X, by construction.

If f has compact support in X, then t f has compact support in X for every t ∈ R or C, as appropriate, because of (3.7.3). Let g be another real or complexvalued function on X, as appropriate, with compact support. It is well known and easy to see that the union of any two compact subsets of X is also compact, so that the union of supp f and supp g is compact. Thus (3.7.5) implies that the support of f + g is contained in a compact subset of X. It follows that the support of f + g is compact, as in the preceding paragraph.

Suppose that X ≠ ∅, and let C_{com}(X, R) and C_{com}(X, C) be the spaces of continuous real and complex-valued functions on X with compact support, respectively. These are linear subspaces of the real and complex vector spaces C(X, R) and C(X, C) of continuous real and complex-valued functions on X, respectively, by the remarks in the previous paragraph. Of course, if X is compact, then every real or complex-valued function on X has compact support.

Suppose that f is a continuous real or complex-valued function on X with compact support. This implies that

f(supp f) (3.7.9)

is a compact subset of R or C, as appropriate, because continuous mappings send compact sets to compact sets. In particular, this means that f is bounded on supp f, because compact subsets of metric spaces are bounded. It follows that f is bounded on X, because f is equal to 0 on the complement of supp f, by construction. Thus

C_{com}(X, R) ⊆ C_{b}(X, R), C_{com}(X, C) ⊆ C_{b}(X, C), (3.7.10)

where C_{b}(X, Y) is as in Section 1.14.

Suppose for the moment again that X is equipped with the discrete metric. In this case, the only compact subsets of X are those with only finitely many elements. If Y is any metric space, then every mapping from X into Y is continuous. Hence C_{com}(X, R) and C_{com}(X, C) are the same as the spaces c_{00}(X, R) and c_{00}(X, C) defined in Section 1.6 in this situation.

3.8 Functions on R

In this section, we take the real line to be equipped with the standard Euclidean metric, as usual. Let f be a real or complex-valued function on R. If f has compact support in R, then there are real numbers a, b with a < b and

supp f ⊆ [a, b], (3.8.1)

because compact subsets of R are bounded. Conversely, if (3.8.1) holds for some a, b ∈ R with a ≤ b, then supp f is compact, because [a, b] is a compact subset of R. Remember that the spaces C_{com}(R, R) and C_{com}(R, C) of continuous real and complex-valued functions on R with compact support are linear subspaces of the real and complex vector spaces of all continuous real and complex-valued functions on R, respectively, as in the previous section.

44 CHAPTER 3. SOME MORE METRIC SPACES

A nonnegative real-valued function N on C_{com}(R, R) or C_{com}(R, C) is said to be a norm if it satisfies the following three conditions, as usual. First, N(f) = 0 if and only if f = 0. Second, if f is a continuous real or complex-valued function on R with compact support and t ∈ R or C, as appropriate, then N(t f) = |t| N(f). (3.8.2) Third, if f and g are continuous real or complex-valued functions on R with compact support, as appropriate, then N(f + g) ≤ N(f) + N(g). (3.8.3) This implies that d_{N}(f, g) = N(f − g) (3.8.4) defines a metric on C_{com}(R, R) or C_{com}(R, C), as appropriate.

Remember that continuous real and complex-valued functions on R with compact support are bounded, as in the previous section. The supremum norm is defined on each of C_{com}(R, R) and C_{com}(R, C) by

∥f∥_{∞} = sup_{x∈R} |f(x)|, (3.8.5)

as in (1.13.4). Equivalently, ∥f∥_{∞} = sup{|f(x)| : x ∈ supp f} (3.8.6) when supp f ≠ ∅, in which case the supremum is attained, by the extreme value theorem. If (3.8.1) holds for some a, b ∈ R with a ≤ b, then

∥f∥_{∞} = sup_{a≤x≤b} |f(x)|. (3.8.7)

As before, the corresponding metric d_{∞}(f, g) = ∥f − g∥_{∞} (3.8.8) is the same as the supremum metric.

Let f be a continuous real or complex-valued function on R with compact support, and let a, b be real numbers such that a ≤ b and (3.8.1) holds. Under these conditions, one can define the integral ∫_{−∞}^{∞} f(x) dx (3.8.9) to be the Riemann integral ∫_{a}^{b} f(x) dx. (3.8.10)

More precisely, one can check that (3.8.10) does not depend on the particular choices of a and b, as long as (3.8.1) holds. If g is another continuous real or complex-valued function on R, as appropriate, with compact support, then

∫_{−∞}^{∞}(f(x) + g(x)) dx = ∫_{−∞}^{∞} f(x) dx + ∫_{−∞}^{∞} g(x) dx. (3.8.11)

To see this, one can choose a, b ∈ R such that a < b and the supports of both f and g are contained in [a, b], so that the support of f + g is contained in [a, b] too. In this case, (3.8.11) reduces to the well-known fact that

∫_{a}^{b}(f(x) + g(x)) dx = ∫_{a}^{b} f(x) dx + ∫_{a}^{b} g(x) dx. (3.8.12)

Of course, other basic properties of (3.8.9) follow from the corresponding statements for (3.8.10).

If f is a continuous real or complex-valued function on R with compact support, then put

∥f∥_{1} = ∫_{−∞}^{∞} |f(x)| dx, (3.8.13)

where the integral on the right is defined as in the preceding paragraph. One can check that this defines a norm on each of C_{com}(R, R) and C_{com}(R, C), in essentially the same way as for functions on an interval. This leads to a metric

d_{1}(f, g) = ∥f − g∥_{1} = ∫_{−∞}^{∞} |f(x) − g(x)| dx (3.8.14)

on each of C_{com}(R, R) and C_{com}(R, C), as in (3.8.8).

Similarly, put

∥f∥_{2} = ( ∫_{−∞}^{∞} |f(x)|^{2} dx)^{1/2} (3.8.15)

for every continuous real or complex-valued function f on R with compact support, using the nonnegative square root on the right side. One can verify that this defines a norm on each of C_{com}(R, R) and C_{com}(R, C), using the analogous statements for functions on an interval. Thus

d_{2}(f, g) = ( ∫_{−∞}^{∞} |f(x) − g(x)|^{2} dx)^{1/2} (3.8.16)

defines a metric on each of C_{com}(R, R) and C_{com}(R, C).

Let f be any continuous real or complex-valued function on R with compact support. Observe that

∥f∥_{2}^{2} = ∫_{−∞}^{∞} |f(x)|^{2} dx ≤ ∥f∥_{∞} ∫_{−∞}^{∞} |f(x)| dx = ∥f∥_{1} ∥f∥_{∞}. (3.8.17)

3.9 Dense sets and Weierstrass’ theorem

Let (M, d(x, y)) be a metric space, and let E be a subset of M. Remember that E is said to be dense in X if every element of M is an element of E, or a limit point of E, or both. This is the same as saying that the closure \overline{E} of E in M is equal to M. Equivalently, it is easy to see that E is dense in M if and only if for every x ∈ M and ϵ > 0 there is a y ∈ E such that

d(x, y) < ϵ. (3.9.1)

Alternatively, one can verify that E is dense in M if and only if for every x ∈ M there is a sequence {x_{j}}_{j=1}^{∞} of elements of E that converges to x in M.

Let (X, d_{X}) and (Y, d_{Y}) be nonempty metric spaces, and let θ(·, ·) be the corresponding supremum metric on the space C_{b}(X, Y) of bounded continuous mappings from X into Y, as in Sections 1.11 and 1.14. A subset E of C_{b}(X, Y) is dense in C_{b}(X, Y) with respect to θ(·, ·) if and only if for every f ∈ C_{b}(X, Y) and ϵ > 0 there is a g ∈ E such that

θ(f, g) < ϵ, (3.9.2)

as in the preceding paragraph. Note that (3.9.2) implies that

(3.9.3) d_{Y} (f(x), g(x)) < ϵ for every x ∈ X.

If (3.9.3) holds, then θ(f, g) ≤ ϵ, (3.9.4)

by the definition of the supremum metric. Using this, we get that E is dense in C_{b}(X, Y) with respect to θ(·, ·) if and only if for every f ∈ C_{b}(X, Y) and ϵ > 0 there is a g ∈ E such that (3.9.3) holds.

We also have that E is dense in C_{b}(X, Y) with respect to θ(·, ·) if and only if for every f ∈ C_{b}(X, Y) there is a sequence {f_{j}}_{j=1}^{∞} of elements of E that converges to f uniformly on X. This uses the equivalence of uniform convergence and convergence with respect to θ(·, ·) for sequences of bounded mappings from X into Y, as in Section 1.11.

Let a, b be real numbers with a < b, and let f be a continuous complexvalued function on [a, b]. Of course, this uses the restriction of the standard Euclidean metric on R to [a, b], and the standard Euclidean metric on C. A famous theorem of Weierstrass states that there is a sequence of polynomials on R with complex coefficients that converges to f uniformly on [a, b]. If f is a continuous real-valued function on [a, b], then one can use polynomials on R with real coefficients. This means that the set of functions on [a, b] corresponding to polynomials with complex coefficients is dense in the space C([a, b], C) of continuous complex-valued functions on [a, b], with respect to the supremum metric. Similarly, the set of functions on [a, b] corresponding to polynomials with real coefficients is dense in the space C([a, b], R) of continuous real-valued functions on [a, b], with respect to the supremum metric.

Suppose now that (X, d_{X}) is a nonempty compact metric space. The Stone– Weierstrass theorem gives a criterion for a set of continuous real or complexvalued functions on X to be dense in C(X, R) or C(X, C), as appropriate, with respect to the supremum metric. This will be discussed in Section 3.11, after some preliminaries in the next section.

3.10 Functions on sets

Let X be a nonempty set, and let

c(X, R), c(X, C) (3.10.1)

be the spaces of all real and complex-valued functions on X, respectively. Of course, if f and g are real or complex-valued functions on X, then f + g is a real or complex-valued function on X, as appropriate. Similarly, if f is a real or complex-valued function on X, and t is a real or complex number, as appropriate, then t f is a real or complex-valued function on X, as appropriate. These are basic (classes of) examples of vector spaces over the real or complex numbers.

A subset W of c(X, R) or c(X, C) is said to be a linear subspace of c(X, R) or c(X, C), as appropriate, if 0 ∈ W, and W satisfies the following two additional conditions. First, if f, g ∈ W, then

f + g ∈ W. (3.10.2)

Second, if f ∈ W and t ∈ R or C, as appropriate, then

t f ∈ W. (3.10.3)

If f and g are real or complex-valued functions on X, then their product f(x) g(x) defines a real or complex-valued function on X, as appropriate, too. In fact, c(X, R), c(X, C) are basic (classes of) examples of commutative rings, although we shall not discuss this in detail here. More precisely, these are commutative algebras over the real and complex numbers, respectively.

Let A be a linear subspace of c(X, R) or c(X, C). Suppose that for every f, g ∈ A, we have that f g ∈ A. (3.10.4)

Under these conditions, A is said to be a subalgebra of c(X, R) or c(X, C), as appropriate.

If X is equipped with a metric, then the corresponding spaces C(X, R) and C(X, C) of continuous real and complex-valued functions on X are subalgebras of c(X, R) and c(X, C), respectively. If X is equipped with the discrete metric, then any mapping from X into another metric space is continuous. In particular, this means that

C(X, R) = c(X, R) and C(X, C) = c(X, C) (3.10.5) in this case.

Let W be a subset of c(X, R) or c(X, C). We say that W is nowhere vanishing on X if for every x ∈ X there is an f ∈ W such that

f(x) ≠ 0. (3.10.6)

In particular, this holds when W contains a nonzero constant function on X. We say that W separates points in X if for every x, w ∈ X with x ≠ w there is an f ∈ W such that f(x) ≠ f(w). (3.10.7)

Of course, at least one of f(x) and f(w) is not equal to 0 in this case.

Suppose now that W is a subset of c(X, C). We shall sometimes be concerned with situations where W contains the complex-conjugate of each of its elements. If W is a linear subspace of c(X, C), then this is equivalent to the condition that W contain the real and imaginary parts of all of its elements.

If W is a linear subspace of c(X, R) or c(X, C), then a subset of W may be called a linear subspace of W if it is a linear subspace of c(X, R) or c(X, C), as appropriate. Similarly, if A is a subalgebra of c(X, R) or c(X, C), then a subset of A may be called a subalgebra of A if it is a subalgebra of c(X, R) or c(X, C), as appropriate. In particular, if X is equipped with a metric, then the notions of linear subspaces and subalgebras of C(X, R) and C(X, C) may be defined in this way.

3.11 The Stone–Weierstrass theorem

Let (X, d(x, y)) be a metric space, and suppose that X is nonempty and compact. Also let A be a subalgebra of the algebra C(X, R) of continuous realvalued functions on X, and suppose that

(3.11.1) A is nowhere vanishing on X,

and that (3.11.2) A separates points in X.

Under these conditions, the Stone–Weierstrass theorem states that

(3.11.3) A is dense in C(X, R),

with respect to the supremum metric. It is easy to see that (3.11.1) and (3.11.2) are necessary for (3.11.3) to hold.

Let a, b be real numbers with a < b. Observe that the collection of functions on [a, b] that can be expressed as a polynomial with real coefficients is a subalgebra of the algebra of continuous real-valued functions on [a, b]. It is easy to see that this subalgebra is nowhere vanishing on [a, b] and that it separates points in [a, b]. Thus Weierstrass’ original approximation theorem for approximating real-valued continuous functions on [a, b] by polynomials with real coefficients uniformly on [a, b] can be obtained from the Stone-Weierstrass theorem. However, Weierstrass’ original theorem, or at least an interesting case of it, is typically used to show the Stone–Weierstrass theorem.

Let (X, d(x, y)) be any nonempty compact metric space again, and now let A be a subalgebra of the algebra C(X, C) of continuous complex-valued functions on X. Suppose that A is nowhere vanishing on X, that A separates points in X, and that

(3.11.4) A contains the complex conjugate of each of its elements.

In this case, another version of the Stone–Weierstrass theorem states that

(3.11.5) A is dense in C(X, C),

with respect to the supremum metric.

This version of the Stone–Weierstrass theorem can be obtained from the previous one, as follows. Note that

(3.11.6) A contains the real and imaginary parts of all of its elements,

as in the previous section. One can check that

(3.11.7) the collection of real-valued functions in A

is a subalgebra of C(X, R) that satisfies (3.11.1) and (3.11.2). Thus the previous version of the theorem implies that every continuous real-valued function on X can be approximated by elements of (3.11.7), uniformly on X. Using this, it is easy to see that every continuous complex-valued function on X can be approximated by elements of A, uniformly on X.

It is well known that the version of the Stone–Weierstrass theorem for complex-valued functions does not always work without the hypothesis (3.11.6). One can get a counterexample by taking X to be the closed unit disk in the complex plane, and A to be the algebra of continuous complex-valued functions on the closed unit disk that are holomorphic or complex-analytic on the open unit disk. Alternatively, one can take X to be the unit circle in the complex plane, and A to be the algebra of continuous complex-valued functions on the circle that have a continuous extension to the closed unit disk that is holomorphic on the open unit disk. There are examples related to these that may be described without using complex analysis.

3.12 Algebras of bounded functions

Let X be a nonempty set, and remember that ℓ^{∞}(X, R) and ℓ^{∞}(X, C) denote the spaces of bounded real and complex-valued functions on X, respectively, as in Section 1.13. If f, g are bounded real or complex-valued functions on X, then it is easy to see that their product f g is bounded on X as well. More precisely, one can check that the supremum norm of f g on X satisfies

∥f g∥_{∞} ≤ ∥f∥_{∞} ∥g∥_{∞}. (3.12.1)

Thus ℓ^{∞}(X, R) and ℓ^{∞}(X, C) are subalgebras of the algebras c(X, R) and c(X, C) of all real and complex-valued functions on X, respectively, as in Section 3.10.

If W is a subset of ℓ^{∞}(X, R) or ℓ^{∞}(X, C), then let \overline{W} be the closure of W in ℓ^{∞}(X, R) or ℓ^{∞}(X, C), as appropriate, with respect to the supremum metric. If W is a linear subspace of ℓ^{∞}(X, R) or ℓ^{∞}(X, C), then one can check that

(3.12.2) \overline{W} is a linear subspace of ℓ^{∞}(X, R) or ℓ^{∞}(X, C),

as appropriate. Similarly, if W is a subalgebra of ℓ^{∞}(X, R) or ℓ^{∞}(X, C), then one can verify that

(3.12.3) \overline{W} is a subalgebra of ℓ^{∞}(X, R) or ℓ^{∞}(X, C),

as appropriate.

Suppose now that (X, d(x, y)) is a nonempty metric space. The spaces C_{b}(X, R) and C_{b}(X, C) of bounded continuous real and complex-valued functions on X are subalgebras of ℓ^{∞}(X, R) and ℓ^{∞}(X, C), respectively. Remember that C_{b}(X, R) and C_{b}(X, C) are also closed sets in ℓ^{∞}(X, R) and ℓ^{∞}(X, C), respectively, with respect to the supremum metric, as in Section 1.14.

Let A be a subalgebra of C_{b}(X, R) or C_{b}(X, C), so that A may also be considered as a subalgebra of ℓ^{∞}(X, R) or ℓ^{∞}(X, C), as appropriate. The closure \overline{A} of A in C_{b}(X, R) or C_{b}(X, C), as appropriate, is the same as the closure of A in ℓ^{∞}(X, R) or ℓ^{∞}(X, C), as appropriate. In particular, \overline{A} is a subalgebra of C_{b}(X, R) or C_{b}(X, C), as appropriate, as before.

If X is compact, then one can use these remarks to reduce to the case of closed subalgebras of C(X, R) or C(X, C), with respect to the supremum metric, in the Stone-Weierstrass theorem.

3.13 Some remarks about closed subalgebras

Let a be a positive real number. It is well known that the absolute value function on R can be uniformly approximated on [−a, a] by polynomials p with real coefficients such that p(0) = 0, (3.13.1)

as in Corollary 7.27 on p161 of [155]. Of course, Weierstrass’ approximation theorem implies that | · | may be approximated uniformly on [−a, a] by polynomials q with real coefficients. The constant term of such an approximation should be small, because |0| = 0, and so one can replace q with q − q(0) to get approximations with the additional condition (3.13.1). This type of approximation can also be obtained more directly, without using Weierstrass’ theorem, as in Exercise 23 on p169 of [155].

Let X be a nonempty set, and let A be a subalgebra of the algebra c(X, R) of all real-valued functions on X. If f ∈ A, and if p is a polynomial with real coefficients that satisfies (3.13.1), then it is easy to see that

p ∘ f ∈ A. (3.13.2)

Now let A be a subalgebra of the algebra ℓ^{∞}(X, R) of all bounded real-valued functions on X. Suppose that A is a closed set in ℓ^{∞}(X, R), with respect to the supremum metric. If f ∈ A, then it is well known that

|f| ∈ A (3.13.3)

too. This corresponds to Step 1 in the proof of Theorem 7.32 on p162 of [155], which is the Stone–Weierstrass theorem.

To see this, note that there is a nonnegative real number a such that |f| ≤ a on X, which means that f(X) ⊆ [−a, a]. (3.13.4)

The absolute value function on R can be uniformly approximated on [−a, a] by polynomials p with real coefficients that satisfy (3.13.1), as before, which implies that |f| can be uniformly approximated on X by the corresponding functions p ∘ f. Thus (3.13.3) can be obtained from (3.13.2), because A is a closed set in ℓ^{∞}(X, R), by hypothesis.

If g is another element of A, then it is well known that

max(f, g), min(f, g) ∈ A (3.13.5)

as well, as in Step 2 on p163 of [155]. This can be obtained by expressing the maximum and minimum of two real numbers in terms of the absolute value function, as in [155].

3.14 Local compactness

A metric space (X, d) is said to be locally compact at a point x ∈ X if there is a positive real number r such that \overline{B}(x, r) is a compact subset of X. Remember that \overline{B}(x, r) is the closed ball in X centered at x with radius r, and that this is a closed set in X.

If there is a continuous real or complex-valued function f on X with compact support such that f(x) ≠ 0, then X is locally compact at x. More precisely, if f(x) ≠ 0, then there is an r > 0 such that f is nonzero at every point in \overline{B}(x, r), because f is continuous at x. This implies that

\overline{B}(x, r) ⊆ supp f, (3.14.1)

and hence that \overline{B}(x, r) is compact when supp f is compact, because \overline{B}(x, r) is a closed set in X.

If (X, d) is any metric space and x ∈ X, then it is well known that the distance to x defines a continuous real-valued function on X. If r is a positive real number, then one can use this to get continuous nonnegative real-valued functions f on X such that f(x) > 0 and

supp f ⊆ \overline{B}(x, r). (3.14.2)

More precisely, one can obtain such functions f by composing the distance to x with suitable continuous functions on the real line. If X is locally compact at x, then one can choose r > 0 so that \overline{B}(x, r) is compact in X. In this case, (3.14.2) implies that f has compact support in X, as before.

If X is locally compact at every x ∈ X, then X is said to be locally compact as a metric space. Of course, if X is compact, then X is automatically locally compact. If X is equipped with the discrete metric, then X is clearly locally compact. Note that R^{n} is locally compact with respect to the standard Euclidean metric for every positive integer n.

3.15 Another vanishing condition

Let (X, d(x, y)) be a metric space, and let f be a real or complex-valued function on X. We say that f vanishes at infinity on X if for every ϵ > 0 there is a compact subset K(ϵ) of X such that

|f(x)| < ϵ (3.15.1)

for every x ∈ X \ K(ϵ). Note that this is not the same as the definition used in Section 2.5. However, if X is equipped with the discrete metric, then the two notions are equivalent, because the only compact subsets of X are those with only finitely many elements. If X is a compact metric space, then any real or complex-valued function on X automatically vanishes at infinity in this sense.

Equivalently, f vanishes at infinity on X if for every ϵ > 0 there is a compact set K(ϵ) ⊆ X such that

E_{ϵ}(f) = {x ∈ X : |f(x)| ≥ ϵ} ⊆ K(ϵ). (3.15.2)

This condition implies that \overline{E_{ϵ}(f)} ⊆ K(ϵ), (3.15.3)

where \overline{E_{ϵ}(f)} is the closure of E_{ϵ}(f) in X, because compact subsets of metric spaces are closed sets. It follows from (3.15.3) that \overline{E_{ϵ}(f)} is compact in X, because a closed set in a metric space is compact when it is contained in a compact set. Conversely, if \overline{E_{ϵ}(f)} is compact, then one might as well take it to be K(ϵ). Thus f vanishes at infinity on X if and only if \overline{E_{ϵ}(f)} is compact for every ϵ > 0.

Note that \overline{E_{ϵ}(f)} ⊆ supp f (3.15.4)

for every ϵ > 0, where supp f is as in (3.7.1). If f has compact support in X, then it follows that f vanishes at infinity on X.

If f vanishes at infinity on X, then it is easy to see that t f vanishes at infinity for every t ∈ R or C, as appropriate. Let g be another real or complexvalued function on X, as appropriate. If f and g both vanish at infinity on X, then one can check that f + g vanishes at infinity on X too. This uses the fact that the union of two compact subsets of X is compact as well.

Suppose that X ≠ ∅, and let C_{0}(X, R) and C_{0}(X, C) be the spaces of continuous real and complex-valued functions on X that vanish at infinity, respectively. These are linear subspaces of the real and complex vector spaces C(X, R) and C(X, C) of continuous real and complex-valued functions on X, respectively, by the remarks in the preceding paragraph. We also have that

C_{com}(X, R) ⊆ C_{0}(X, R), C_{com}(X, C) ⊆ C_{0}(X, C) (3.15.5)

because functions on X with compact support automatically vanish at infinity, as before. If X is equipped with the discrete metric, then C_{0}(X, R) and C_{0}(X, C) are the same as the spaces c_{00}(X, R) and c_{00}(X, C) defined in Section 2.5, respectively. This uses the fact that all functions on X are continuous in this case.

Let f be a continuous real or complex-valued function on X. Note that E_{ϵ}(f) is a closed set in X for each ϵ > 0. Thus f vanishes at infinity on X if and only if E_{ϵ}(f) is compact for every ϵ > 0. Of course, the restriction of f to any compact set K ⊆ X is bounded, because f(K) is compact, and hence bounded. If E_{ϵ}(f) is compact for any ϵ > 0, then it follows that f is bounded on X. In particular, if f vanishes at infinity on X, then f is bounded on X. This means that

C_{0}(X, R) ⊆ C_{b}(X, R), C_{0}(X, C) ⊆ C_{b}(X, C), (3.15.6)

where C_{b}(X, Y) is as in Section 1.14.

Let f be a continuous real or complex-valued function on X again, and let x be an element of X such that f(x) ≠ 0. Thus |f(x)| > ϵ for some ϵ > 0, and the continuity of f at x implies that there is an r > 0 such that |f(y)| ≥ ϵ for every y ∈ X with d(x, y) ≤ r. Equivalently, this means that

\overline{B}(x, r) ⊆ E_{ϵ}(f), (3.15.7)

where \overline{B}(x, r) is the closed ball in X centered at x with radius r, as before. If f vanishes at infinity on X, then it follows that \overline{B}(x, r) is compact, because \overline{B}(x, r) is a closed set in X. This implies that X is locally compact at x under these conditions.

Let {f_{j}}_{j=1}^{∞} be a sequence of real or complex-valued functions on X that converges uniformly to a real or complex-valued function f on X, as appropriate. Suppose that f_{j} vanishes at infinity on X for each j, and let us show that f vanishes at infinity on X too. Let ϵ > 0 be given. Because {f_{j}}_{j=1}^{∞} converges to f uniformly on X, there is an L ∈ Z_{+} such that

|f_{j}(x) − f(x)| < ϵ/2 (3.15.8)

for every x ∈ X and j ≥ L. There is also a compact set K ⊆ X such that

|f_{L}(x)| < ϵ/2 (3.15.9)

for every x ∈ X \ K, because f_{L} vanishes at infinity on X. Combining (3.15.8) and (3.15.9), we get that

|f(x)| ≤ |f(x) − f_{L}(x)| + |f_{L}(x)| < ϵ/2 + ϵ/2 = ϵ (3.15.10) for every x ∈ X \ K, as desired. In particular, this implies that C_{0}(X, R) and C_{0}(X, C) are closed sets in C_{b}(X, R) and C_{b}(X, C) with respect to the supremum metric.

Let f be a real or complex-valued function on X that vanishes at infinity. Also let ϕ be a mapping from R or C into itself such that ϕ(0) = 0 and ϕ is continuous at 0. Under these conditions, one can check that ϕ ∘ f vanishes at infinity on X too. If ϕ vanishes on a neighborhood of 0 in R or C, as appropriate, then one can verify that ϕ ∘ f has compact support in X. Of course, if f and ϕ are continuous, then ϕ ∘ f is continuous on X too. It is easy to see that the identity mappings on R and C can be approximated uniformly by continuous functions that vanish on a neighborhood of 0. If f is a continuous function on X that vanishes at infinity, then one can use this to approximate f uniformly by continuous functions on X with compact support. This means that C_{com}(X, R) and C_{com}(X, C) are dense in C_{0}(X, R) and C_{0}(X, C), respectively, with respect to the supremum metric.

Compactness and completeness

4.1 Diameters of sets

Let (X, d(x, y)) be a metric space, and let E be a subset of X. If E ≠ ∅, then the diameter of E can be defined as a nonnegative extended real number by

diam E = sup{d(x, y) : x, y ∈ E}. (4.1.1)

It is easy to see that this is finite exactly when E is bounded in X. It is sometimes convenient to define the diameter of the empty set to be 0. One can check that (4.1.2) diam \overline{E} = diam E

for every E ⊆ X, where \overline{E} is the closure of E in X, as usual.

If E_{1} ⊆ E_{2} ⊆ X, then

(4.1.3) diam E_{1} ≤ diam E_{2}.

Observe that diam \overline{B}(x, r) ≤ 2 r (4.1.4)

for every x ∈ X and nonnegative real number r, by the triangle inequality. Here \overline{B}(x, r) is the closed ball in X centered at x with radius r, as before. If E ⊆ X is a bounded set, then E ⊆ \overline{B}(x, diam E) (4.1.5)

for every x ∈ E.

Let E_{1}, E_{2}, E_{3}, … be an infinite sequence of nonempty subsets of X such that (4.1.6) E_{j} is closed and bounded

for each j ≥ 1, E_{j+1} ⊆ E_{j} (4.1.7)

56 CHAPTER 4. COMPACTNESS AND COMPLETENESS for every j ≥ 1, and

lim_{j→∞} diam E_{j} = 0. (4.1.8)

Let x_{j} be an element of E_{j} for each j ≥ 1. If 1 ≤ j ≤ l, then it follows that x_{l} ∈ E_{l} ⊆ E_{j}. (4.1.9) It is easy to see that (4.1.10) {x_{j}}_{j=1}^{∞} is a Cauchy sequence in X. Suppose for the moment that X is complete with respect to d, so that (4.1.11) {x_{j}}_{j=1}^{∞} converges to an element x of X. If l is any positive integer, then {x_{j}}_{j=l}^{∞} is a sequence of elements of E_{l} that converges to x. This implies that x ∈ E_{l} (4.1.12) for every l ≥ 1, because E_{l} is a closed set, by hypothesis. Thus

⋂_{j=1}^{∞} E_{j} ≠ ∅ (4.1.13)

under these conditions.

Now let {x_{j}}_{j=1}^{∞} be a Cauchy sequence in X, and put E_{j} = \overline{\{x_{l} : l ≥ j\}} (4.1.14) for each j ≥ 1, which is the closure in X of the set of x_{l} with j ≥ l. Clearly E_{j} ≠ ∅ and (4.1.7) holds for every j ≥ 1, and the E_{j}’s are closed sets, by construction. It is well known that the E_{j}’s are bounded in X, because {x_{j}}_{j=1}^{∞} is a Cauchy sequence, and in fact we have that (4.1.8) holds. If

x ∈ ⋂_{j=1}^{∞} E_{j}, (4.1.15)

then one can check that (4.1.16) {x_{j}}_{j=1}^{∞} converges to x in X. This means that completeness of X is necessary in order to get (4.1.13) under the conditions described in the preceding paragraph.

4.2 Totally bounded sets

Let (X, d(x, y)) be a metric space again. A subset E of X is said to be totally bounded if for every r > 0 there are finitely many elements x_{1}, …, x_{l} of X such that

E ⊆ ⋃_{j=1}^{l} B(x_{j}, r). (4.2.1)

If E = ∅, then this may be considered to hold trivially with l = 0 for each r > 0, even when X = ∅. If (4.2.1) holds for some r > 0 and finitely many points x_{1}, …, x_{l} in X, then E is a bounded set in X. Thus totally bounded sets are bounded in particular.

If E ⊆ X is compact, then

(4.2.2) E is totally bounded.

More precisely, for each r > 0, E can be covered by open balls in X of radius r. Compactness implies that this open covering can be reduced to a finite subcovering, as desired.

Suppose that E is a bounded subset of R^{n} for some positive integer n, with respect to the standard Euclidean metric on R^{n}. In this case, one can check directly that E is totally bounded.

Of course, finite subsets of X are totally bounded. If X is equipped with the discrete metric and E ⊆ X is totally bounded, then

(4.2.3) E has only finitely many elements.

This can be seen by taking r = 1 in (4.2.1).

A subset E of X is totally bounded if and only if for every t > 0 there are finitely many elements y_{1}, …, y_{l} of X such that

E ⊆ ⋃_{j=1}^{l} \overline{B}(y_{j}, t). (4.2.4)

More precisely, the previous formulation implies this one, because B(x, r) is contained in \overline{B}(x, r) for every x ∈ X and r > 0. To get the converse, one can use the fact that \overline{B}(y, t) is contained in B(y, r) when t < r. If E is totally bounded, then it follows from this reformulation that

(4.2.5) the closure \overline{E} of E in X is totally bounded

too.

As another reformulation, a subset E of X is totally bounded if and only if for every r_{1} > 0,

(4.2.6) E is contained in the union of finitely many subsets of X with diameter strictly less than r_{1}.

More precisely, this formulation can be obtained from (4.2.1) using (4.1.4). Similarly, one can get (4.2.1) from this formulation using (4.1.5). Equivalently, E ⊆ X is totally bounded if and only if for every t_{1} > 0,

(4.2.7) E is contained in the union of finitely many subsets of X with diameter less than or equal to t_{1}.

Suppose that E ⊆ X is not totally bounded in X. This means that there is an r > 0 such that E cannot be covered by finitely many open balls in X of radius r. In particular, E ≠ ∅, and we let x_{1} be an element of E. By hypothesis, E is not contained in B(x_{1}, r), and so there is an element x_{2} of E that is not in B(x_{1}, r). If elements x_{1}, …, x_{l} of E have been chosen in this way for some positive integer l, then E is not contained in ⋃_{j=1}^{l} B(x_{j}, r), by hypothesis. This permits us to choose x_{l+1} ∈ E such that x_{l+1} is not in B(x_{j}, r) when 1 ≤ j ≤ l. Continuing in this way, we get an infinite sequence {x_{j}}_{j=1}^{∞} of elements of E such that x_{l} is not in B(x_{j}, r) when j < l. Equivalently,

d(x_{j}, x_{l}) ≥ r (4.2.8)

when j < l.

4.3 Separable metric spaces

A metric space (X, d(x, y)) is said to be separable if

(4.3.1) there is a dense subset of X with only finitely or countably many elements.

It is well known that (4.3.2) R^{n} is separable

with respect to the standard Euclidean metric for each positive integer n, as in Exercise 22 at the end of Chapter 2 of [155]. More precisely, one can check that the set of points in R^{n} with rational coordinates is a countable dense set in R^{n}, as in the hint given in [155].

If X is any set equipped with the discrete metric, then

(4.3.3) X is the only dense subset of itself.

Thus X is separable with respect to the discrete metric if and only if

(4.3.4) X has only finitely or countably many elements.

Let (X, d(x, y)) be any metric space again. One can check that X is separable if and only if for each ϵ > 0 there is a subset A(ϵ) of X such that

(4.3.5) A(ϵ) has only finitely or countably many elements and

⋃_{x∈A(ϵ)} B(x, ϵ) = X. (4.3.6)

Indeed, if X is separable, then there is a dense set A ⊆ X with only finitely or countably many elements, and one can take

A(ϵ) = A (4.3.7)

for every ϵ > 0. Conversely, if A(1/j) ⊆ X satisfies (4.3.6) with ϵ = 1/j for each positive integer j, then it is easy to see that

A = ⋃_{j=1}^{∞} A(1/j) (4.3.8)

is dense in X. If A(1/j) also has only finitely or countably many elements for each j ≥ 1, then (4.3.8) has only finitely or countably many elements as well.

Note that X is totally bounded if and only if for every ϵ > 0 there is a finite set A(ϵ) ⊆ X such that (4.3.6) holds. This implies that

(4.3.9) X is separable,

as before.

A collection B of open subsets of X is said to be a base for the topology of X if for every x ∈ X and r > 0 there is a U ∈ B such that x ∈ U and

U ⊆ B(x, r). (4.3.10)

Equivalently, this means that every nonempty open subset of X can be expressed as a union of elements of B. One may consider the empty set to be the union of the empty collection of sets.

Let E be a dense subset of X, and let B(E) be the collection of subsets of X of the form B(y, 1/j), where y ∈ E and j ∈ Z_{+}. We would like to show that

(4.3.11) B(E) is a base for the topology of X.

To do this, let x ∈ X and r > 0 be given. Let j be a positive integer such that 2/j < r. Because E is dense in X, there is a y ∈ E such that d(x, y) < 1/j. This implies that x ∈ B(y, 1/j), and one can also verify that

B(y, 1/j) ⊆ B(x, r), (4.3.12)

using the triangle inequality, as desired.

If E has only finitely or countably many elements, then

(4.3.13) B(E) has only finitely or countably many elements.

Conversely, if there is a base B for the topology of X with only finitely or countably many elements, then X is separable. More precisely, one can choose an element from every nonempty element of B to get a dense set in X with only finitely or countably many elements.

4.4 Lindelöf’s theorem

Let (X, d(x, y)) be a metric space, and suppose that B is a base for the topology of X with only finitely or countably many elements. Let {U_{α}}_{α∈A} be a family of open subsets of X, indexed by a nonempty set A. Under these conditions, a theorem of Lindelöf says that there is a subset A_{1} of A such that

(4.4.1) A_{1} has only finitely or countably many elements

and

⋃_{α∈A_{1}} U_{α} = ⋃_{α∈A} U_{α}. (4.4.2)

To see this, put B_{α} = {V ∈ B : V ⊆ U_{α}} (4.4.3)

for every α ∈ A. Observe that

⋃_{V ∈B_{α}} V = U_{α} (4.4.4)

for every α ∈ A, because B is a base for the topology of X, and U_{α} is an open set. Put

B′ = ⋃_{α∈A} B_{α}, (4.4.5)

so that

⋃_{V ∈B′} V = ⋃_{α∈A} ( ⋃_{V ∈B_{α}} V ) = ⋃_{α∈A} U_{α}. (4.4.6)

If V ∈ B′, then V ∈ B_{α} for some α ∈ A, and we let α(V) be an element of A with this property. Let A_{1} = {α(V) : V ∈ B′} (4.4.7)

be the collection of elements of A that have been chosen in this way.

Thus

⋃_{V ∈B′} V ⊆ ⋃_{α∈A_{1}} U_{α}, (4.4.8)

because V ⊆ U_{α(V)} for every V ∈ B′, by construction. This implies that

⋃_{α∈A} U_{α} ⊆ ⋃_{α∈A_{1}} U_{α}, (4.4.9)

by (4.4.6). The opposite inclusion holds automatically, because A_{1} ⊆ A, so that (4.4.2) holds. Clearly B′ has only finitely or countably many elements, because B′ ⊆ B, and B has only finitely or countably many elements, by hypothesis. It follows that A_{1} has only finitely or countably many elements too, as desired.

A subset E of X is said to have the Lindelöf property if every open covering of E in X can be reduced to a subcovering with only finitely or countably many elements. More precisely, this means that if {U_{α}}_{α∈A} is an open covering of E in X, then there is a subset A_{1} of A such that A_{1} has only finitely or countably many elements and

E ⊆ ⋃_{α∈A_{1}} U_{α}. (4.4.10)

If there is a base B for the topology of X with only finitely or countably many elements, then Lindelöf’s theorem implies that every subset of X has the Lindelöf property.

Suppose that X has the Lindelöf property, and let ϵ > 0 be given. Of course, X is covered by the collection of all open balls in X of radius ϵ. Because X has the Lindelöf property, there is a subset A(ϵ) of X with only finitely or countably many elements such that (4.3.6) holds. This implies that X is separable, as mentioned in the previous section.

4.5 The limit point property

Let (X, d(x, y)) be a metric space. A subset E of X is said to have the limit point property if for every infinite subset A of E there is an x ∈ E such that

(4.5.1) x is a limit point of A in X.

If E is compact, then it is well known that

(4.5.2) E has the limit point property.

Otherwise, there is an infinite set A ⊆ E which has no limit point in E. This means that for every x ∈ E there is a positive real number r(x) such that B(x, r(x)) does not contain any element of A, except perhaps for x itself. Because E is compact, there are finitely many elements x_{1}, …, x_{n} of E such that

E ⊆ ⋃_{j=1}^{n} B(x_{j}, r(x_{j})). (4.5.3)

In particular, this implies that

A ⊆ ⋃_{j=1}^{n}(A ∩ B(x_{j}, r(x_{j}))). (4.5.4)

This means that A has at most n elements, contradicting the hypothesis that A have infinitely many elements.

If E ⊆ X has the limit point property, then

(4.5.5) E is totally bounded in X.

Otherwise, there is an r > 0 and an infinite sequence {x_{j}}_{j=1}^{∞} of elements of X such that d(x_{j}, x_{l}) ≥ r when j < l, (4.5.6) as in Section 4.2. Let A be the set of x_{j}’s, j ≥ 1, which is an infinite subset of E, because the x_{j}’s are distinct. If x ∈ X, then it is easy to see that

(4.5.7) A ∩ B(x, r/2) has at most two elements,

using the triangle inequality. This implies that x is not a limit point of A, and hence that E does not have the limit point property.

Suppose that E ⊆ X has the limit point property, and let U_{1}, U_{2}, U_{3}, … be an infinite sequence of open subsets of X such that

E ⊆ ⋃_{j=1}^{∞} U_{j}. (4.5.8)

We would like to show that there is a positive integer n such that

E ⊆ ⋃_{j=1}^{n} U_{j}. (4.5.9)

Otherwise, for each n ≥ 1, we can choose x_{n} ∈ E such that x_{n} ∉ ⋃_{j=1}^{n} U_{j}. Let A be the set of points x_{n}, n ≥ 1, that have been chosen in this way. Let us check that A has infinitely many elements.

If y ∈ E, then y ∈ U_{l} for some l ≥ 1, by (4.5.8). This means that x_{n} ≠ y when n ≥ l, because x_{n} ∉ U_{l}, by construction. Thus x_{n} = y for at most finitely many n ≥ 1, which implies that A has infinitely many elements.

Hence there is an x ∈ E such that x is a limit point of A in X, because E has the limit point property. We also have that x ∈ U_{j} for some j ≥ 1, by (4.5.8). Because U_{j} is an open set in X, there is an r > 0 such that

B(x, r) ⊆ U_{j}. (4.5.10)

There are infinitely many elements of A in B(x, r), because x is a limit point of A. This means that there are infinitely many elements of A in U_{j}, by (4.5.10). Thus x_{n} ∈ U_{j} for infinitely many n ≥ 1. This contradicts the fact that x_{n} ∉ U_{j} when n ≥ j, by construction.

Suppose that X has the limit point property, and let us show that

(4.5.11) X is compact.

Remember that X is totally bounded in this case, as mentioned earlier. This implies that X is separable, as in the previous section, and hence that there is a base for the topology of X with only finitely or countably many elements. It follows that X has the Lindelöf property, as in the previous section again. Combining this with the argument in the preceding paragraphs, we get that X is compact, as desired.

Now let E be any subset of X with the limit point property, and let us verify that (4.5.12) E is compact.

Remember that E may also be considered as a metric space, with respect to the restriction of d(x, y) to x, y ∈ E. It is easy to see that E has the limit point property as a subset of itself when E has the limit point property as a subset of X. This implies that E is compact as a subset of itself, as in the previous paragraph. It is well known that this implies that E is compact as a subset of X, as desired.

4.6 Sequential compactness

Let (X, d(x, y)) be a metric space again. A subset E of X is said to be sequentially compact if for every sequence {x_{j}}_{j=1}^{∞} of elements of E there is a subsequence {x_{j_{l}}}_{l=1}^{∞} of {x_{j}}_{j=1}^{∞} and an element x of E such that

(4.6.1) {x_{j_{l}}}_{l=1}^{∞} converges to x in X.

If E has the limit point property, then it is well known that

(4.6.2) E is sequentially compact.

More precisely, let {x_{j}}_{j=1}^{∞} be a sequence of elements of E, and let A be the set of x_{j}’s, j ≥ 1. If A has only finitely many elements, then there is a subsequence {x_{j_{l}}}_{l=1}^{∞} of {x_{j}}_{j=1}^{∞} and an element x of E such that x_{j_{l}} = x for every l ≥ 1. In particular, this implies that {x_{j_{l}}}_{l=1}^{∞} converges to x in X. Otherwise, if A has infinitely many elements, then there is an x ∈ E such that x is a limit point of A in X, because E has the limit point property. In this case, one can show that there is a subsequence {x_{j_{l}}}_{l=1}^{∞} of {x_{j}}_{j=1}^{∞} that converges to x in X.

Conversely, if E ⊆ X is sequentially compact, then E has the limit point property. To see this, let A be an infinite subset of E, and let {x_{j}}_{j=1}^{∞} be a sequence of distinct elements of A. By hypothesis, there is a subsequence {x_{j_{l}}}_{l=1}^{∞} of {x_{j}}_{j=1}^{∞} that converges to an element x of E. It is easy to see that x is a limit point of A in X, as desired.

Let us say that E ⊆ X has the Cauchy subsequence property if every sequence {x_{j}}_{j=1}^{∞} of elements of E has a subsequence {x_{j_{l}}}_{l=1}^{∞} that is a Cauchy sequence in X. Sequential compactness automatically implies the Cauchy subsequence property, because convergent sequences are Cauchy sequences.

Let us say that a sequence {x_{j}}_{j=1}^{∞} of elements of X is an ϵ-sequence for some ϵ > 0 if d(x_{j}, x_{l}) < ϵ (4.6.3)

for every j, l ≥ 1. Let us say that a subset E of X has the small subsequence property if for every sequence {x_{j}}_{j=1}^{∞} of elements of E and every ϵ > 0 there is a subsequence {x_{j_{l}}}_{l=1}^{∞} of {x_{j}}_{j=1}^{∞} such that

(4.6.4) {x_{j_{l}}}_{l=1}^{∞} is an ϵ-sequence.

If E has the Cauchy subsequence property, then it is easy to see that E has the small subsequence property. This uses the fact that every Cauchy sequence in X is an ϵ-sequence for any ϵ > 0 after skipping finitely many terms in the sequence.

If E ⊆ X is totally bounded, then E has the small subsequence property. To see this, let a sequence {x_{j}}_{j=1}^{∞} of elements of E and an ϵ > 0 be given. Because E is totally bounded, E can be covered by finitely many open balls in X of radius ϵ/2. This implies that there is a single open ball of radius ϵ/2 that contains x_{j} for infinitely many j ≥ 1, so that there is a subsequence {x_{j_{l}}}_{l=1}^{∞} of {x_{j}}_{j=1}^{∞} such that all of the x_{j_{l}}’s are contained in that open ball. It follows that {x_{j_{l}}}_{l=1}^{∞} is an ϵ-sequence, as desired, by the triangle inequality.

Conversely, suppose that E ⊆ X has the small subsequence property, and let us check that E is totally bounded. Otherwise, if E is not totally bounded, then there is an r > 0 and a sequence {x_{j}}_{j=1}^{∞} of elements of E such that d(x_{j}, x_{l}) ≥ r when j < l, as in Section 4.2. In this case, {x_{j}}_{j=1}^{∞} has no subsequence that is an r-subsequence, so that E does not have the small subsequence property.

If E ⊆ X has the small subsequence property, then E has the Cauchy subsequence property. To see this, let a sequence {x_{j}}_{j=1}^{∞} of elements of E be given. By hypothesis, there is a subsequence of {x_{j}}_{j=1}^{∞} that is a (1/2)-sequence. We can repeat the process, to get a subsequence of the first subsequence that is a (1/4)-sequence. Continuing in this way, we get an infinite sequence of subsequences of {x_{j}}_{j=1}^{∞}, such that the (n + 1)th subsequence of {x_{j}}_{j=1}^{∞} is also a subsequence of the nth subsequence of {x_{j}}_{j=1}^{∞} for every n ≥ 1, and the nth subsequence is a 2^{−n}-sequence for every n ≥ 1. One can check that the sequence obtained by taking the rth term of the rth subsequence for each r ≥ 1 is a subsequence of {x_{j}}_{j=1}^{∞} as well. This subsequence is a Cauchy sequence too, because the terms with r ≥ n may be considered as a subsequence of the nth subsequence for each n ≥ 1.

4.7 A criterion for compactness

Let (X, d(x, y)) be a complete metric space. If

(4.7.1) E ⊆ X is closed and totally bounded,

then it is well known that E is compact. One way to show this is to use the fact that E has the Cauchy subsequence property, because E is totally bounded, as in the previous section. In this case, Cauchy sequences converge in X, because X is complete. We also have that the limit of a convergent sequence of elements of E is contained in E, because E is a closed set. It follows that E is sequentially compact under these conditions. This implies that E has the limit point property, as in the previous section, and hence that E is compact, as in Section 4.5.

Before describing another proof, let us reformulate the hypothesis that E be totally bounded. If r is a positive real number, then E is contained in the union of finitely many subsets of X with diameter less than or equal to r, as in Section 4.2. We may as well suppose that these are subsets of E, since we can replace them with their intersections with E. We can also take these subsets to be closed sets in X, by replacing them with their closures, which have the same diameters as before. The closures of these subsets of E are contained in E, because E is a closed set, by hypothesis.

Let {U_{α}}_{α∈A} be an open covering of E in X for which there is no finite subcovering, so that E ≠ ∅ in particular. As in the previous paragraph, E can be expressed as the union of finitely many closed sets, each of which has diameter less than or equal to 1/2. At least one of these finitely many sets cannot be covered by finitely many U_{α}’s. This implies that there is a subset E_{1} of E such that E_{1} is a closed set in X, the diameter of E_{1} is less than or equal to 1/2, and E_{1} cannot be covered by finitely many U_{α}’s.

Continuing in this way, we can get an infinite sequence E_{1}, E_{2}, E_{3}, … of subsets of E such that E_{j} is a closed set in X for every j ≥ 1,

E_{j+1} ⊆ E_{j} (4.7.2)

for every j ≥ 1, diam E_{j} ≤ 2^{−j} (4.7.3)

for every j ≥ 1, and E_{j} cannot be covered by finitely many U_{α}’s for any j ≥ 1. More precisely, suppose that E_{j} has been chosen for some j ≥ 1, and let us choose E_{j+1}. By hypothesis, E_{j} is a closed set contained in E, and hence E_{j} is totally bounded. Thus E_{j} can be expressed as the union of finitely many closed sets, each of which has diameter less than or equal to 2^{−j−1}. At least one of these subsets cannot be covered by finitely many U_{α}’s, because E_{j} cannot be covered by finitely many U_{α}’s. Thus we take E_{j+1} to be one of these subsets of E_{j}, in such a way that E_{j+1} cannot be covered by finitely many U_{α}’s. Clearly E_{j+1} satisfies (4.7.2), and the analogues of the other conditions for j + 1.

Note that diam E_{j} → 0 as j → ∞, by (4.7.3). This implies that there is an x ∈ X that is contained in E_{j} for every j ≥ 1, because X is complete, as before. In particular, x ∈ E, so that there is an α_{0} ∈ A such that x ∈ U_{α_{0}}. It follows that there is a positive real number r such that

B(x, r) ⊆ U_{α_{0}}, (4.7.4)

because U_{α_{0}} is an open set. If j is large enough so that 2^{−j} < r, then E_{j} is contained in B(x, r), because of (4.7.3) and the fact that x ∈ E_{j}. This means that E_{j} ⊆ U_{α_{0}}, by (4.7.4). This contradicts the condition that E_{j} cannot be covered by finitely many U_{α}’s, as desired.

Let {x_{j}}_{j=1}^{∞} be a Cauchy sequence of elements of a metric space X, which is not necessarily complete. If there is a subsequence {x_{j_{l}}}_{l=1}^{∞} of {x_{j}}_{j=1}^{∞} that converges to an element x of X, then it is well known and not too difficult to check that (4.7.5) {x_{j}}_{j=1}^{∞} converges to x

in X. In particular, if {x_{j}}_{j=1}^{∞} is a sequence of elements of a sequentially compact set E ⊆ X, then {x_{j}}_{j=1}^{∞} converges to an element of E. If X is sequentially compact, then it follows that X is complete. Remember that compact subsets of metric spaces have the limit point property, and hence are sequentially compact.

We can also use the second proof of this criterion for compactness as another way to show that sequential compactness implies compactness. More precisely, if a metric space X is sequentially compact, then X is complete and totally bounded, and hence compact, by this criterion. Now let E be a subset of an arbitrary metric space X, and remember that E may be considered as a metric space as well, by restricting the metric to elements of E. If E is sequentially compact in X, then it is easy to see that E is sequentially compact as a subset of itself too. This implies that E is compact as a subset of itself, as before, and hence that E is compact as a subset of X.

4.8 The Baire category theorem

Let (X, d(x, y)) be a metric space. One can check that E ⊆ X is dense in X if and only if for every open set V ⊆ X with V ≠ ∅, we have that

E ∩ V ≠ ∅. (4.8.1)

If E is dense in X and U is a dense open set in X, then one can verify that

(4.8.2) E ∩ U is dense in X.

This implies that the intersection of two dense open sets in X is a dense open set in X as well. It follows that the intersection of finitely many dense open subsets of X is a dense open subset of X too.

Let U_{1}, U_{2}, U_{3}, … be an infinite sequence of dense open subsets of X. If X is complete, then the Baire category theorem states that

(4.8.3)  ⋂_{j=1}^{∞} U_{j} is dense in X.

To see this, let x ∈ X and r > 0 be given, and let us show that

\overline{B}(x, r) ∩ ( ⋂_{j=1}^{∞} U_{j}) ≠ ∅. (4.8.4)

Because U_{1} is dense in X, there is a y_{1} ∈ U_{1} such that d(x, y_{1}) < r. Let us choose r_{1} > 0 so that r_{1} ≤ 1,

\overline{B}(y_{1}, r_{1}) ⊆ U_{1}, (4.8.5)

and d(x, y_{1}) + r_{1} ≤ r. (4.8.6)

This uses the hypothesis that U_{1} be an open set to get (4.8.5). Note that

B(y_{1}, r_{1}) ⊆ B(x, r), \overline{B}(y_{1}, r_{1}) ⊆ \overline{B}(x, r), (4.8.7)

by (4.8.6).

Repeating the process, we get an infinite sequence {y_{j}}_{j=1}^{∞} of elements of X and an infinite sequence {r_{j}}_{j=1}^{∞} of positive real numbers with the following properties. First, r_{j} ≤ 1/j (4.8.8)

for each j ≥ 1. Second, \overline{B}(y_{j}, r_{j}) ⊆ U_{j} (4.8.9)

for every j ≥ 1. Third, (4.8.6) holds when j = 1, and otherwise

d(y_{j−1}, y_{j}) + r_{j} ≤ r_{j−1} (4.8.10)

when j ≥ 2. This implies that

B(y_{j}, r_{j}) ⊆ B(y_{j−1}, r_{j−1}), \overline{B}(y_{j}, r_{j}) ⊆ \overline{B}(y_{j−1}, r_{j−1}) (4.8.11)

when j ≥ 2.

More precisely, we can do this when j = 1, as before. Suppose that y_{j} ∈ X and r_{j} > 0 have been chosen with these properties for some j ≥ 1, and let us see how we can choose y_{j+1} and r_{j+1}. Because U_{j+1} is dense in X, there is a y_{j+1} ∈ U_{j+1} such that d(y_{j}, y_{j+1}) < r_{j}. (4.8.12)

We can choose r_{j+1} > 0 so that r_{j+1} ≤ 1/(j + 1),

\overline{B}(y_{j+1}, r_{j+1}) ⊆ U_{j+1}, (4.8.13)

and d(y_{j}, y_{j+1}) + r_{j+1} ≤ r_{j}. (4.8.14)

This uses the hypothesis that U_{j+1} be an open set to get (4.8.13), as before. If 1 ≤ j ≤ l, then

B(y_{l}, r_{l}) ⊆ B(y_{j}, r_{j}), \overline{B}(y_{l}, r_{l}) ⊆ \overline{B}(y_{j}, r_{j}), (4.8.15)

by (4.8.11). In particular,

B(y_{l}, r_{l}) ⊆ B(x, r), \overline{B}(y_{l}, r_{l}) ⊆ \overline{B}(x, r) (4.8.16)

for every l ≥ 1, by (4.8.7). Using (4.8.8) and (4.8.15), we get that

(4.8.17) {y_{l}}_{l=1}^{∞} is a Cauchy sequence in X.

Hence {y_{l}}_{l=1}^{∞} converges to an element y of X, because X is complete. Observe that y_{l} ∈ \overline{B}(x, r) for every l ≥ 1, by (4.8.16), so that

y ∈ \overline{B}(x, r). (4.8.18)

Similarly, for each j ≥ 1, we have that y_{l} ∈ \overline{B}(y_{j}, r_{j}) when l ≥ 1, by (4.8.15). This implies that y ∈ \overline{B}(y_{j}, r_{j}) (4.8.19) for every j ≥ 1. It follows that y ∈ U_{j} (4.8.20) for every j ≥ 1, because of (4.8.9). Thus

y ∈ \overline{B}(x, r) ∩ ( ⋂_{j=1}^{∞} U_{j}), (4.8.21)

as desired.

Somse related results will be mentioned in the next section.

4.9 The interior of a set

Let (X, d(·, ·)) be a metric space. The interior E^{∘} of a subset E of X is the set of x ∈ E for which there is an r > 0 such that B(x, r) ⊆ E. It is easy to see that X \ E^{∘} = \overline{X \ E}. (4.9.1) In particular, E^{∘} = ∅ if and only if X \ E is dense in X.

Let E_{1}, E_{2}, E_{3}, … be an infinite sequence of closed subsets of X such that E_{j}^{∘} = ∅ for every j ≥ 1. If X is complete, then the Baire category theorem implies that

(4.9.2)  ⋃_{j=1}^{∞} E_{j} has empty interior

too. More precisely, X \ E_{j} is a dense open set in X for each j, so that

(4.9.3)  ⋂_{j=1}^{∞}(X \ E_{j}) is dense in X

when X is complete. This means that X \ ( ⋃_{j=1}^{∞} E_{j}) is dense in X, as desired. A subset A of X is said to be nowhere dense in X if

(4.9.4) the closure \overline{A} of A in X has empty interior.

A subset B of X is said to be of first category in X, or meager, if

(4.9.5) B can be expressed as the union of a sequence of nowhere dense sets.

Otherwise, B is said to be of second category in X, or nonmeager. If B ⊆ X is of first category and X is complete, then it follows that

B^{∘} = ∅. (4.9.6)

If B_{1}, B_{2}, B_{3}, … is an infinite sequences of subsets of X of first category, then it is not too difficult to show that

⋃_{l=1}^{∞} B_{l} (4.9.7)

is of first category in X as well.

Equicontinuity and sequences of functions

Let {f_{j}}_{j=1}^{∞} be a sequence of mappings from a set X into a metric space Y. In some situations, we might like to find a subsequence {f_{j_{l}}}_{l=1}^{∞} of {f_{j}}_{j=1}^{∞} that converges to a mapping f from X into Y, at least in some sense. If X is a metric space too, then we may be interested in additional continuity conditions on the f_{j}’s, and on f. In particular, X may be a subset of R^{n} for some positive integer n. These and related matters will be discussed in this chapter.

5.1 Pointwise convergent subsequences

Let E be a nonempty set with only finitely or countably many elements, and let Y be a metric space. Also let {f_{j}}_{j=1}^{∞} be a sequence of mappings from E into Y. Suppose that for each x ∈ E there is a sequentially compact set K(x) ⊆ Y such that f_{j}(x) ∈ K(x) (5.1.1)

for every j ≥ 1. We would like to show that there is a subsequence {f_{j_{l}}}_{l=1}^{∞} of {f_{j}}_{j=1}^{∞} that converges pointwise to a mapping f from E into Y, with

f(x) ∈ K(x) (5.1.2)

for every x ∈ E. Remember that compact subsets of Y have the limit point property, and thus are sequentially compact.

Of course, (5.1.1) implies that

(5.1.3) {f_{j}(x)}_{j=1}^{∞} is a bounded sequence in Y.

If Y is the complex plane, or R^{n} for some positive integer n, with the standard Euclidean metric, and if {f_{j}(x)}_{j=1}^{∞} is a bounded sequence in Y, then there is a sequentially compact set K(x) ⊆ Y such that (5.1.1) holds for each j.

70CHAPTER 5. EQUICONTINUITY AND SEQUENCES OF FUNCTIONS

Suppose first that E has only finitely many elements x_{1}, …, x_{n}. Using sequential compactness of K(x_{1}), we can get a subsequence {f_{j_{l}}}_{l=1}^{∞} of {f_{j}}_{j=1}^{∞} such that {f_{j_{l}}(x_{1})}_{l=1}^{∞} converges to an element f(x_{1}) of K(x_{1}). If n ≥ 2, then we can use sequential compactness of K(x_{2}) to get a subsequence {f_{j_{lr}} }_{r=1}^{∞} of {f_{j_{l}}}_{l=1}^{∞} such that {f_{j_{lr}} (x_{2})}_{r=1}^{∞} converges to an element f(x_{2}) of K(x_{2}). Note that {f_{j_{lr}} (x_{1})}_{r=1}^{∞} converges to f(x_{1}) in Y, because a subsequence of a convergent sequence converges to the same limit. We can repeat the process until we get a subsequence of {f_{j}}_{j=1}^{∞} that converges pointwise on E in this case.

Suppose now that E is countably infinite, and let x_{1}, x_{2}, x_{3}, … be an enumeration of the elements of E. As before, we can use sequential compactness of K(x_{1}) to get a subsequence of {f_{j}}_{j=1}^{∞} that converges pointwise at x_{1}. Repeating the process, we get a sequence of subsequences of {f_{j}}_{j=1}^{∞} with the following properties. First, for each r ∈ Z_{+},

(5.1.4) the rth subsequence converges pointwise at x_{r} to an element of K(x_{r}).

Second, if r ≥ 2, then

(5.1.5) the rth subsequence is a subsequence of the (r − 1)th subsequence.

The second property ensures that the rth subsequence is a subsequence of all the previous subsequences, and of the initial sequence {f_{j}}_{j=1}^{∞}. Combining this with the first property, we get that the rth subsequence converges pointwise at each of the previous points, with the same limits as the previous subsequences.

Let f be the mapping from E into Y such that for each r ≥ 1, f(x_{r}) is the limit of the rth subsequence of {f_{j}}_{j=1}^{∞} at x_{r}. Thus

f(x_{r}) ∈ K(x_{r}) (5.1.6)

for every r ≥ 1, by construction. We would like to show that there is a subsequence of {f_{j}}_{j=1}^{∞} that converges to f pointwise on E.

Consider the sequence of mappings {g_{n}}_{n=1}^{∞} from E into Y obtained by taking g_{n} to be the nth term of the nth subsequence mentioned before for each n ∈ Z_{+}. One can check that

(5.1.7) {g_{n}}_{n=1}^{∞} is a subsequence of {f_{j}}_{j=1}^{∞}.

Similarly, for each r ∈ Z_{+},

(5.1.8) {g_{n}}_{n=r}^{∞} is a subsequence of the rth subsequence mentioned earlier.

This implies that (5.1.9) {g_{n}(x_{r})}_{n=r}^{∞} converges to f(x_{r})

in Y. It follows that

(5.1.10) {g_{n}(x_{r})}_{n=1}^{∞} converges to f(x_{r})

in Y for each r ∈ Z_{+}, as desired.

5.2 Equicontinuity

Let (X, d_{X}) and (Y, d_{Y}) be metric spaces. A collection E of mappings from X into Y is said to be equicontinuous at a point x ∈ X if for every ϵ > 0 there is a δ > 0 such that d_{Y} (f(x), f(w)) < ϵ (5.2.1)

for every f ∈ E and w ∈ X with d_{X}(x, w) < δ. In particular, this implies that every f ∈ E is continuous at x.

Similarly, E is said to be uniformly equicontinuous on X if for every ϵ > 0 there is a δ > 0 such that (5.2.1) holds for every x, w ∈ X with d_{X}(x, w) < δ. This implies that E is equicontinuous at every point in X, and that every element of E is uniformly continuous on X.

If X is compact, then it is well known that every continuous mapping from X into Y is uniformly continuous. In this case, if E is equicontinuous at every point in X, then one can show that

(5.2.2) E is uniformly equicontinuous on X,

using essentially the same argument.

Suppose for the moment that E has only finitely many elements. If x ∈ X and every f ∈ E is continuous at x, then it is easy to see that E is equicontinuous at x. Similarly, if every element of E is uniformly continuous on X, then E is uniformly equicontinuous on X.

Let C be a nonnegative real number, and suppose that

(5.2.3) every element of E is Lipschitz with constant C.

It is easy to see that E is uniformly equicontinuous on X in this case. There is an anlogous statement for Lipschitz or Hölder continuity conditions of any order α > 0, as in Section A.2.

Let {f_{j}}_{j=1}^{∞} be a sequence of mappings from X into Y, and let x be an element of X. We say that {f_{j}}_{j=1}^{∞} is equicontinuous at x if the collection of f_{j}’s, j ∈ Z_{+}, is equicontinuous at x. Suppose that {f_{j}}_{j=1}^{∞} converges pointwise to a mapping f from X into Y, and that {f_{j}}_{j=1}^{∞} is equicontinuous at x. We would like to check that (5.2.4) f is continuous at x

as well under these conditions. Let ϵ > 0 be given, so that there is a δ = δ(x, ϵ) > 0 such that d_{Y} (f_{j}(x), f_{j}(w)) < ϵ (5.2.5)

for every j ∈ Z_{+} and w ∈ X with d_{X}(x, w) < δ. If w ∈ X satisfies d_{X}(x, w) < δ, then one can check that d_{Y} (f(x), f(w)) ≤ ϵ, (5.2.6)

using (5.2.5) and the fact that {f_{j}(x)}_{j=1}^{∞} and {f_{j}(w)}_{j=1}^{∞} converge to f(x) and f(w) in Y, respectively, by hypothesis. More precisely,

d_{Y} (f(x), f(w)) ≤ d_{Y} (f(x), f_{j}(x)) + d_{Y} (f_{j}(x), f_{j}(w)) + d_{Y} (f_{j}(w), f(w))
(5.2.7)  < d_{Y} (f_{j}(x), f(x)) + ϵ + d_{Y} (f_{j}(w), f(w))

for every j ∈ Z_{+} and w ∈ X with d_{X}(x, w) < δ, and the right side is arbitrarily close to ϵ when j is sufficiently large. This implies that f is continuous at x, as desired.

Similarly, a sequence {f_{j}}_{j=1}^{∞} of mappings from X into Y is said to be uniformly equicontinuous on X if the collection of f_{j}’s, j ∈ Z_{+}, is uniformly equicontinuous on X. If {f_{j}}_{j=1}^{∞} is uniformly equicontinuous on X, and if {f_{j}}_{j=1}^{∞} converges pointwise to a mapping f from X into Y, then one can verify that (5.2.8) f is uniformly continuous on X.

This is basically the same as the argument in the preceding paragraph, except that δ = δ(ϵ) does not depend on x ∈ X.

Let C be a nonnegative real number again, and let {f_{j}}_{j=1}^{∞} be a sequence of mappings from X into Y such that f_{j} is Lipschitz with constant C for each j. If {f_{j}}_{j=1}^{∞} converges pointwise to a mapping f from X into Y, then one can check that (5.2.9) f is Lipschitz with constant C

too. There is an analogous statement for Lipschitz or Hölder continuity conditions of any order α > 0, as before.

5.3 Uniformly Cauchy sequences

Let X be a set, and let (Y, d_{Y}) be a metric space. Let us say that a sequence {f_{j}}_{j=1}^{∞} of mappings from X into Y is uniformly Cauchy on X if for every ϵ > 0 there is a positive integer L(ϵ) such that

d_{Y} (f_{j}(x), f_{l}(x)) < ϵ (5.3.1)

for every x ∈ X and j, l ≥ L(ϵ). If X ≠ ∅ and the f_{j}’s are bounded mappings from X into Y, then it is easy to see that this is equivalent to the condition that {f_{j}}_{j=1}^{∞} be a Cauchy sequence with respect to the supremum metric on the space of bounded mappings from X into Y. This is analogous to the equivalence between uniform convergence and convergence with respect to the supremum metric for bounded mappings from X into Y, as in Section 1.11.

If {f_{j}}_{j=1}^{∞} is any sequence of mappings from X into Y that converges uniformly to a mapping f from X into Y, then one can check that

(5.3.2) {f_{j}}_{j=1}^{∞} is uniformly Cauchy on X.

This is analogous to the fact that convergent sequences in a metric space are Cauchy sequences.

Let {f_{j}}_{j=1}^{∞} be a uniformly Cauchy sequence of mappings from X into Y. In particular, this implies that

(5.3.3) {f_{j}(x)}_{j=1}^{∞} is a Cauchy sequence in Y for every x ∈ X. If Y is complete, then it follows that {f_{j}}_{j=1}^{∞} converges pointwise to a mapping f from X into Y. One can verify that

(5.3.4) {f_{j}}_{j=1}^{∞} converges uniformly to f on X

in this situation. This is analogous to the fact that the space B(X, Y) of bounded mappings from X into Y is complete with respect to the supremum metric when X ≠ ∅ and Y is complete, as in Section 1.12.

Suppose now that (X, d_{X}) is a metric space too, and let {f_{j}}_{j=1}^{∞} be a uniformly Cauchy sequence of mappings from X into Y. Let x ∈ X be given, and suppose that f_{j} is continuous at x for each j ∈ Z_{+}. Under these conditions,

(5.3.5) {f_{j}}_{j=1}^{∞} is equicontinuous at x.

To see this, let ϵ > 0 be given, and let L(ϵ/3) ∈ Z_{+} be as before, so that

d_{Y} (f_{j}(w), f_{l}(w)) < ϵ/3 (5.3.6)

for every w ∈ X and j, l ≥ L(ϵ/3). It follows that

d_{Y} (f_{j}(x), f_{j}(w)) ≤ d_{Y} (f_{j}(x), f_{l}(x)) + d_{Y} (f_{l}(x), f_{l}(w)) + d_{Y} (f_{l}(w), f_{j}(w))
(5.3.7)  < d_{Y} (f_{l}(x), f_{l}(w)) + 2 ϵ/3

for every w ∈ X and j, l ≥ L(ϵ/3).

If l ≥ L(ϵ/3), then we can use the continuity of f_{l} at x to get a δ_{l}(x, ϵ/3) > 0 such that d_{Y} (f_{l}(x), f_{l}(w)) < ϵ/3 (5.3.8)

for every w ∈ X with d_{X}(x, w) < δ_{l}(x, ϵ/3). (5.3.9)

Combining this with (5.3.7), we get that

d_{Y} (f_{j}(x), f_{j}(w)) < ϵ (5.3.10)

for every j ≥ L(ϵ/3) and w ∈ X such that (5.3.9) holds. We may as well take l = L(ϵ/3) here.

We would like to find a δ(x, ϵ) > 0 such that (5.3.10) holds for every j ∈ Z_{+} and w ∈ X with d_{X}(x, w) < δ(x, ϵ). (5.3.11)

This can be obtained from the previous statement for j ≥ L(ϵ/3) and the continuity of f_{j} at x when j < L(ϵ/3).

Similarly, if f_{j} is uniformly continuous on X for each j ≥ 1, then

(5.3.12) {f_{j}}_{j=1}^{∞} is uniformly equicontinuous on X.

74CHAPTER 5. EQUICONTINUITY AND SEQUENCES OF FUNCTIONS

5.4 Equicontinuity and uniform convergence

Let (X, d_{X}) and (Y, d_{Y}) be metric spaces again, and let {f_{j}}_{j=1}^{∞} be a sequence of mappings from X into Y. Suppose that (5.4.1) {f_{j}}_{j=1}^{∞} converges pointwise to a mapping f from X into Y, and that (5.4.2) {f_{j}}_{j=1}^{∞} is equicontinuous at every x ∈ X. Let ϵ > 0 be given, so that for each x ∈ X there is a δ(x, ϵ) > 0 such that d_{Y} (f_{j}(x), f_{j}(w)) < ϵ/3 (5.4.3) for every j ∈ Z_{+} and w ∈ X with d_{X}(x, w) < δ(x, ϵ). This implies that d_{Y} (f(x), f(w)) ≤ ϵ/3 (5.4.4) for every x, w ∈ X with d_{X}(x, w) < δ(x, ϵ), as in Section 5.2. It follows that d_{Y} (f_{j}(w), f(w)) (5.4.5)

≤ d_{Y} (f_{j}(w), f_{j}(x)) + d_{Y} (f_{j}(x), f(x)) + d_{Y} (f(x), f(w))
< 2 ϵ/3 + d_{Y} (f_{j}(x), f(x))

for every j ∈ Z_{+} and x, w ∈ X with d_{X}(x, w) < δ(x, ϵ), using the triangle inequality twice in the first step, and (5.4.3), (5.4.4) in the second step.

If x ∈ X, then there is a positive integer L(x, ϵ) such that d_{Y} (f_{j}(x), f(x)) < ϵ/3 (5.4.6) for every j ≥ L(x, ϵ), because of (5.4.1). Combining this with (5.4.5), we obtain that d_{Y} (f_{j}(w), f(w)) < ϵ (5.4.7) for every x ∈ X, j ≥ L(x, ϵ), and w ∈ X with d_{X}(x, w) < δ(x, ϵ).

Let K be a compact subset of X. The collection of open balls B_{X}(x, δ(x, ϵ)) (5.4.8) in X centered at elements x of K is an open covering of K in X. Because K is compact, there are finitely many elements x_{1}, …, x_{n} of K such that

K ⊆ ⋃_{l=1}^{n} B_{X}(x_{l}, δ(x_{l}, ϵ)). (5.4.9)

Put

L_{K}(ϵ) = max_{1≤l≤n} L(x_{l}, ϵ). (5.4.10)

It follows that (5.4.7) holds for every w ∈ K and j ≥ L_{K}(ϵ), so that (5.4.11) {f_{j}}_{j=1}^{∞} converges to f uniformly on K 5.5. EQUICONTINUITY AND CAUCHY SEQUENCES 75 under these conditions.

If {f_{j}}_{j=1}^{∞} is uniformly equicontinuous on X, then we can take δ(x, ϵ) = δ(ϵ) to be independent of x ∈ X. In this case, one can check that (5.4.12) {f_{j}}_{j=1}^{∞} converges to f uniformly on totally bounded subsets of X, using an argument like the one in the preceding paragraph.

5.5 Equicontinuity and Cauchy sequences

Let (X, d_{X}), (Y, d_{Y}) be metric spaces, and let {f_{j}}_{j=1}^{∞} be a sequence of mappings from X into Y. Also let E be a subset of X, and suppose that for every w ∈ E, (5.5.1) {f_{j}(w)}_{j=1}^{∞} is a Cauchy sequence in Y. If x ∈ X is a limit point of E, and if (5.5.2) {f_{j}}_{j=1}^{∞} is equicontinuous at x, then (5.5.3) {f_{j}(x)}_{j=1}^{∞} is a Cauchy sequence in Y.

To see this, let ϵ > 0 be given. Because of (5.5.2), there is a δ > 0 such that d_{Y} (f_{j}(x), f_{j}(w)) < ϵ/3 (5.5.4) for every j ≥ 1 and w ∈ X with d_{X}(x, w) < δ. Thus d_{Y} (f_{j}(x), f_{l}(x)) (5.5.5)

≤ d_{Y} (f_{j}(x), f_{j}(w)) + d_{Y} (f_{j}(w), f_{l}(w)) + d_{Y} (f_{l}(w), f_{l}(x))
< 2 ϵ/3 + d_{Y} (f_{j}(w), f_{l}(w))

for every j, l ≥ 1 and w ∈ X with d_{X}(x, w) < δ. This uses the triangle inequality twice in the first step, and (5.5.4) twice in the second step, applied to j and l.

If x is a limit point of E, then there is a w ∈ E such that d_{X}(x, w) < δ. In this case, (5.5.1) implies that there is a positive integer L such that d_{Y} (f_{j}(w), f_{l}(w)) < ϵ/3 (5.5.6) for every j, l ≥ L. Combining this with (5.5.5), we get that d_{Y} (f_{j}(x), f_{l}(x)) < ϵ (5.5.7) for every j, l ≥ L, as desired. If Y is complete as a metric space, then it follows that {f_{j}(x)}_{j=1}^{∞} converges in Y.

76CHAPTER 5. EQUICONTINUITY AND SEQUENCES OF FUNCTIONS

5.6 Equicontinuity and subsequences

Let (X, d_{X}) and (Y, d_{Y}) be metric spaces again, and let {f_{j}}_{j=1}^{∞} be a sequence of mappings from X into Y. Suppose that for every x ∈ X there is a sequentially compact set K(x) ⊆ Y such that f_{j}(x) ∈ K(x) (5.6.1) for every j ≥ 1. Suppose also that (5.6.2) X is separable as a metric space, and let E be a dense set in X with only finitely or countably many elements. Thus (5.6.3) there is a subsequence {f_{j_{l}}}_{l=1}^{∞} of {f_{j}}_{j=1}^{∞}

that converges pointwise on E,

as in Section 5.1.

Of course, if (5.6.4) {f_{j}}_{j=1}^{∞} is equicontinuous at every x ∈ X, then (5.6.5) {f_{j_{l}}}_{l=1}^{∞} is equicontinuous at every x ∈ X too. In this case, (5.6.6) {f_{j_{l}}(x)}_{l=1}^{∞} is a Cauchy sequence in Y for every x ∈ X, as in the previous section. It follows that (5.6.7) {f_{j_{l}}(x)}_{l=1}^{∞} converges to an element of K(x) for each x ∈ X, because a Cauchy sequence of elements of a sequentially compact set converges to an element of that set, as mentioned near the end of Section 4.7.

Let f(x) be the limit of {f_{j_{l}}(x)}_{l=1}^{∞} for every x ∈ X, so that f is a mapping from X into Y, and (5.6.8) {f_{j_{l}}}_{l=1}^{∞} converges to f pointwise on X. Note that (5.6.9) f is continuous on X, because of (5.6.5), as in Section 5.2. We also get that (5.6.10) {f_{j_{l}}}_{l=1}^{∞} converges to f uniformly on compact subsets of X, as in Section 5.4.

Similarly, if

(5.6.11) {f_{j}}_{j=1}^{∞} is uniformly equicontinuous on X,

then

(5.6.12) {f_{j_{l}}}_{l=1}^{∞} is uniformly equicontinuous on X, (5.6.13) f is uniformly continuous on X,

and

(5.6.14) {f_{j_{l}}}_{l=1}^{∞} converges to f uniformly on totally bounded subsets of X.

Of course, this is all a bit simpler when X is compact. Note that X is automatically separable in this case, as in Section 4.3.

Let us now take X = [0, 1] and Y = R, using the standard Euclidean metric on R and its restriction to [0, 1]. Put f_{j}(x) = x^{j} on [0, 1] for every j ≥ 1, and remember that {f_{j}}_{j=1}^{∞} converges pointwise on [0, 1], as in Section 1.8. One can check that {f_{j}}_{j=1}^{∞} is equicontinuous at x ∈ [0, 1] when x < 1, and not when x = 1. More precisely, if 0 ≤ r < 1, then {f_{j}}_{j=1}^{∞} is uniformly equicontinuous on [0, r], and {f_{j}}_{j=1}^{∞} converges to 0 uniformly on [0, r]. However, there is no subsequence of {f_{j}}_{j=1}^{∞} that converges uniformly on [0, 1].

One can use uniformly convergent subsequences to get existence of solutions to ordinary differential equations under suitable conditions, as in Exercises 25, 26 on p170f of [155], for instance. Note that uniqueness of solutions involves additional conditions on the differential equation, as in Exercises 27, 28 on p119 of [155].

The question of finding more elementary approaches to existence theorems like this was raised in [95]. Some responses to this question can be found in [55, 186, 187].

5.7 Pointwise and uniform boundedness

Let X be a set, let (Y, d_{Y}) be a metric space, and let E be a collection of mappings from X into Y. If x ∈ X, then put

E(x) = {f(x) : f ∈ E}, (5.7.1)

which is a subset of Y. Let us say that E is pointwise bounded on a subset A of X if for every x ∈ A, (5.7.2) E(x) is bounded in Y.

Similarly, put

E(A) = ⋃_{x∈A} E(x) = ⋃_{f∈E} f(A) = {f(x) : x ∈ A, f ∈ E}, (5.7.3)

78CHAPTER 5. EQUICONTINUITY AND SEQUENCES OF FUNCTIONS which is a subset of Y as well. We say that E is uniformly bounded on A when (5.7.4) E(A) is bounded in Y.

If E is uniformly bounded on A, then E is pointwise bounded on A, and the restriction of every f ∈ E to A is bounded as a mapping from A into Y. Suppose that X, Y ≠ ∅, and let E be a collection of bounded mappings from X into Y. One can check that (5.7.5) E is uniformly bounded on X if and only if E is bounded as a subset of the space B(X, Y) of bounded mappings from X into Y, with respect to the supremum metric.

Now let (X, d_{X}) be a metric space too, and let E be a collection of mappings from X into Y that is equicontinuous at a point x ∈ X. Let ϵ > 0 be given, so that there is a δ(x, ϵ) > 0 such that d_{Y} (f(x), f(w)) < ϵ (5.7.6) for every f ∈ E and w ∈ X with d_{X}(x, w) < δ(x, ϵ). If there is a w_{0} ∈ X such that d_{X}(x, w_{0}) < δ(x, ϵ) and (5.7.7) E(w_{0}) is bounded in Y, then it is easy to see that (5.7.2) holds. Similarly, if (5.7.2) holds, then (5.7.8) E is uniformly bounded on the open ball B_{X}(x, δ(x, ϵ)).

Suppose for the moment that (5.7.9) E is equicontinuous at every x ∈ X. One can check that (5.7.10) {u ∈ X : E(u) is bounded in Y } is a closed set in X, using the remarks in the preceding paragraph. If K ⊆ X is compact, and (5.7.11) E is pointwise bounded on K, then one can verify that (5.7.12) E is uniformly bounded on K. If (5.7.13) E is uniformly equicontinuous on X, then we can take δ(x, ϵ) = δ(ϵ) to be independent of x ∈ X in the preceding paragraph. In this case, if (5.7.14) A ⊆ X is totally bounded 5.8. TOTAL BOUNDEDNESS IN B(X, Y) 79

and (5.7.15) E is pointwise bounded on A,

then (5.7.16) E is uniformly bounded on A.

Suppose that (5.7.9) holds again. One can check that the set of u ∈ X such that E(u) is totally bounded in Y is a closed set in X. This uses the equicontinuity condition for all ϵ > 0, while a single ϵ > 0 would suffice for the remarks about boundedness in the previous paragraph. If K ⊆ X is compact, and E(x) is totally bounded in Y for every x ∈ K, then E(K) is totally bounded in Y. If E is uniformly equicontinuous on X, A ⊆ X is totally bounded, and E(x) is totally bounded in Y for every x ∈ A, then E(A) is totally bounded in Y.

5.8 Total boundedness in B(X, Y)

Let X be a nonempty set, and let (Y, d_{Y}) be a nonempty metric space. Remember that B(X, Y) is the space of bounded mappings from X into Y, and that θ(f, g) denotes the supremum metric on B(X, Y), as in Section 1.11. Suppose that (5.8.1) E ⊆ B(X, Y) is totally bounded with respect to θ(·, ·).

Let x ∈ X be given, and remember that E(x) is the subset of Y defined in (5.7.1). It is easy to see that

(5.8.2) E(x) is totally bounded in Y.

Suppose that (X, d_{X}) is a metric space too, and that

(5.8.3) every f ∈ E is continuous at x.

We would like to check that

(5.8.4) E is equicontinuous at x.

Let ϵ > 0 be given. Because E is totally bounded with respect to the supremum metric, there are finitely many elements f_{1}, …, f_{n} of E such that for every f ∈ E there is a positive integer j ≤ n such that

θ(f, f_{j}) ≤ ϵ/3. (5.8.5)

By hypothesis, f_{j} is continuous at x for each j = 1, …, n, so that there is a δ_{j}(x) > 0 such that d_{Y} (f_{j}(x), f_{j}(w)) < ϵ/3 (5.8.6)

for every w ∈ X with d_{X}(x, w) < δ_{j}(x).

Put

δ(x) = min_{1≤j≤n} δ_{j}(x) > 0. (5.8.7)

Let f ∈ E be given, and let j ≤ n be as in (5.8.5). Thus

d_{Y} (f(x), f(w)) (5.8.8)

≤ d_{Y} (f(x), f_{j}(x)) + d_{Y} (f_{j}(x), f_{j}(w)) + d_{Y} (f_{j}(w), f(w))
≤ d_{Y} (f_{j}(x), f_{j}(w)) + 2 ϵ/3

for every w ∈ X, using the triangle inequality in the first step. It follows that

d_{Y} (f(x), f(w)) < ϵ/3 + 2 ϵ/3 = ϵ (5.8.9)

when d_{X}(x, w) < δ(x) ≤ δ_{j}(x), using (5.8.6) in the first step. This shows that (5.8.4) holds, as desired.

Similarly, if

(5.8.10) every f ∈ E is uniformly continuous on X,

then (5.8.11) E is uniformly equicontinuous on X.

This is essentially the same as before, but with δ_{j}(x) and hence δ(x) independent of x.

Let A be a subset of X, and remember that E(A) is the subset of Y defined in (5.7.3). Suppose that

(5.8.12) for each f ∈ E, f(A) is totally bounded in Y.

One can check that (5.8.13) E(A) is totally bounded in Y

as well under these conditions. Of course, (5.8.1) holds automatically when E has only finitely amy elements. However, (5.8.12) does not holds automatically in this case.

5.9 Equicontinuity and total boundedness

Let (X, d_{X}) and (Y, d_{Y}) be nonempty metric spaces again. If f is a uniformly continuous mapping from X into Y and A is a totally bounded subset of X, then it is not difficult to show that

(5.9.1) f(A) is totally bounded in Y.

Suppose for the rest of the section that

(5.9.2) X is totally bounded with respect to d_{X}.

This implies that uniformly continuous mappings from X into Y are bounded, because totally bounded subsets of Y are bounded. Let θ(·, ·) be the supremum metric on the space B(X, Y) of bounded mappings from X into Y, as usual.

5.9. EQUICONTINUITY AND TOTAL BOUNDEDNESS 81

Suppose that (5.9.3) E is a uniformly equicontinuous

collection of mappings from X into Y.

In particular, every f ∈ E is uniformly continuous as a mapping from X into Y, and hence bounded, as in the preceding paragraph. Let ϵ > 0 be given, so that there is a δ > 0 such that d_{Y} (f(x), f(w)) < ϵ/3 (5.9.4) holds for every f ∈ E and x, w ∈ X with d_{X}(x, w) < δ. Because X is totally bounded, there are finitely many elements x_{1}, …, x_{n} of X such that

X = ⋃_{j=1}^{n} B_{X}(x_{j}, δ), (5.9.5)

where B_{X}(x, r) is the open ball in X centered at x ∈ X with radius r > 0, as usual.

Let f, g ∈ E and x ∈ X be given, so that d_{X}(x_{j}, x) < δ for some j ≤ n, by (5.9.5). Under these conditions, we have that d_{Y} (f(x_{j}), f(x)), d_{Y} (g(x_{j}), g(x)) < ϵ/3, (5.9.6) as in (5.9.4). This implies that d_{Y} (f(x), g(x)) (5.9.7)

≤ d_{Y} (f(x), f(x_{j})) + d_{Y} (f(x_{j}), g(x_{j})) + d_{Y} (g(x_{j}), g(x))
< d_{Y} (f(x_{j}), g(x_{j})) + 2 ϵ/3,

using the triangle inequality in the first step. Thus

d_{Y} (f(x), g(x)) < max_{1≤j≤n} d_{Y} (f(x_{j}), g(x_{j})) + 2 ϵ/3 (5.9.8)

for every x ∈ X. It follows that

θ(f, g) ≤ max_{1≤j≤n} d_{Y} (f(x_{j}), g(x_{j})) + 2 ϵ/3 (5.9.9)

for every f, g ∈ E.

Suppose that E(x) = {f(x) : f ∈ E} (5.9.10) is totally bounded in Y for each x ∈ X, in addition to the other conditions mentioned in the previous paragraphs. Using this and (5.9.9), one can check that (5.9.11) E is totally bounded with respect to the supremum metric.

5.10 Equiconvergence of limits

Let (X, d_{X}) and (Y, d_{Y}) be metric spaces, let A be a subset of X, and suppose that x ∈ X is a limit point of A in X. Also let E be a collection of mappings from E into Y, and suppose that for each f ∈ E, the limit

lim_{{w∈A \atop w→x}} f(w) = q_{f} (5.10.1)

exists in Y. Let us say that we have equiconvergence of the limit for f ∈ E if for every ϵ > 0 there is a δ > 0 such that

d_{Y} (f(w), q_{f}) < ϵ (5.10.2)

for every f ∈ E and w ∈ A with d_{X}(x, w) < δ and w ≠ x.

More precisely, this condition includes the existence of the limit as in (5.10.1) for each f ∈ E. If E has only finitely many elements, and the limit as in (5.10.1) exists for each f ∈ E, then one can check that we have equiconvergence of the limit for f ∈ E.

Suppose for the moment that A = X, so that x is a limit point of X. In this case, it is well known and easy to see that a mapping f from X into Y is continuous at x if and only if

lim_{w→x} f(w) = f(x), (5.10.3)

where the existence of the limit is part of this condition. One can verify that we have equiconvergence of the limit for f ∈ E, with q_{f} = f(x) for every f ∈ E, exactly when E is equicontinuous at x.

Let A be any subset of X again, with x ∈ X a limit point of A, and let {f_{j}}_{j=1}^{∞} be a sequence of mappings from A into Y. Suppose that for each positive integer j, the limit

lim_{{w∈A \atop w→x}} f_{j}(w) = q_{j} (5.10.4)

exists in Y. We can define equiconvergence of the limit for {f_{j}}_{j=1}^{∞} in the same way as before, by considering the collection of f_{j}’s, j ∈ Z.

Suppose for the moment that

(5.10.5) for each w ∈ A, {f_{j}(w)}_{j=1}^{∞} is a Cauchy sequence in Y.

If we have equiconvergence of the limit in (5.10.4) for {f_{j}}_{j=1}^{∞}, then one can show that (5.10.6) {q_{j}}_{j=1}^{∞} is a Cauchy sequence in Y.

This is very similar to the argument in Section 5.5. If Y is complete as a metric space, then it follows that

(5.10.7) {q_{j}}_{j=1}^{∞} converges to an element q of Y.

Suppose now that {f_{j}}_{j=1}^{∞} converges pointwise to a mapping f from A into Y. If we have equiconvergence of the limit in (5.10.4) for {f_{j}}_{j=1}^{∞}, and if (5.10.7) holds, then one can show that

lim_{{w∈A \atop w→x}} f(w) = q. (5.10.8)

This is analogous to the argument for equicontinuous sequences of functions that converge pointwise, as in Section 5.2.

Let {f_{j}}_{j=1}^{∞} be a uniformly Cauchy sequence of mappings from A into Y, as in Section 5.3. Suppose that the limit as in (5.10.4) exists for each j ∈ Z_{+}. Under these conditions, one can check that (5.10.6) holds. Using this, one can verify that we have equiconvergence of the limit as in (5.10.4) for {f_{j}}_{j=1}^{∞}. This is analogous to the argument for equicontinuity in Section 5.3.

Suppose that {f_{j}}_{j=1}^{∞} converges uniformly to a mapping f from A into Y, so that {f_{j}}_{j=1}^{∞} is uniformly Cauchy on A in particular. Suppose also that the limit as in (5.10.4) exists for every j, and that (5.10.7) holds. Using the remarks in the previous two paragraphs, we get that (5.10.8) holds too.

5.11 Equiconvergence and differentiability

Let a, b be real numbers with a < b, and let f be a real-valued function on [a, b]. As usual, the derivative of f at x ∈ [a, b] is defined by

f′(x) = lim_{w→x} \frac{f(w) − f(x)}{w − x}, (5.11.1)

when the limit exists. Of course, this is really a one-sided limit and derivative when x = a or b.

Let E be a collection of real-valued functions on [a, b], each of which is differentiable at x ∈ [a, b]. In this case, we may be interested in the equiconvergence of the limit of difference quotients in (5.11.1) for f ∈ E, as in the previous section.

Suppose for the moment that each f ∈ E is differentiable at every point in [a, b]. If the collection E′ = {f′ : f ∈ E} (5.11.2)

of derivatives of elements of E is equicontinuous at x, then one can check that we have equiconvergence of the limit as in (5.11.1) for f ∈ E, using the mean value theorem.

Let {f_{j}}_{j=1}^{∞} be a sequence of real-valued functions on [a, b], and suppose that f_{j} is differentiable at a point x ∈ [a, b] for each j. As before, we may be interested in the equiconvergence of the limit

f′_{j}(x) = lim_{w→x} \frac{f_{j}(w) − f_{j}(x)}{w − x} (5.11.3)

for {f_{j}}_{j=1}^{∞}.

Suppose for the moment that f_{j} is differentiable at every point in [a, b] for each j. If {f_{j}}_{j=1}^{∞} is equicontinuous at x, then we have equiconvergence of the limit as in (5.11.3) for {f_{j}}_{j=1}^{∞}, as before.

Suppose now that {f_{j}}_{j=1}^{∞} converges pointwise to a real-valued function f on [a, b]. If x, w ∈ [a, b] and x ≠ w, then it follows that

lim_{j→∞} \frac{f_{j}(w) − f_{j}(x)}{w − x} = \frac{f(w) − f(x)}{w − x}. (5.11.4)

If f_{j} is differentiable at x for each j, and we have equiconvergence of the limit as in (5.11.3) for {f_{j}}_{j=1}^{∞}, then {f′_{j}(x)}_{j=1}^{∞} is a Cauchy sequence in R, as in the previous section. Of course, this means that {f′_{j}(x)}_{j=1}^{∞} converges in R, because R is complete with respect to the standard Euclidean metric. Using this, we get that f is differentiable at x, with

f′(x) = lim_{j→∞} f′_{j}(x), (5.11.5)

as in the previous section.

Suppose that f_{j} is differentiable at a point x ∈ [a, b] for each j again. Suppose also that the sequence of difference quotients

\frac{f_{j}(w) − f_{j}(x)}{w − x} (5.11.6)

is uniformly Cauchy as a sequence of real-valued functions of w on [a, b] \ {x}. This implies that {f′_{j}(x)}_{j=1}^{∞} is a Cauchy sequence, as in the previous section. This means that {f′_{j}(x)}_{j=1}^{∞} converges in R, because R is complete with respect to the standard Euclidean metric. In this case, we get equiconvergence of the limit as in (5.11.3) for {f_{j}}_{j=1}^{∞} too, as in the previous section.

Note that

\frac{f_{j}(w) − f_{j}(x)}{w − x} − \frac{f_{l}(w) − f_{l}(x)}{w − x} = \frac{(f_{j}(w) − f_{l}(w)) − (f_{j}(x) − f_{l}(x))}{w − x} (5.11.7)

for every w ∈ [a, b]\{x} and j, l ≥ 1. The uniform Cauchy condition for (5.11.6) on [a, b] \ {x} means that if j, l are sufficiently large, then (5.11.7) is as small as we like, uniformly over w ∈ [a, b] \ {x}. Suppose that f_{j} is differentiable at every point in [a, b] for each j, and that {f′_{j}}_{j=1}^{∞} is uniformly Cauchy as a sequence of real-valued functions on [a, b]. In this case, one can use the mean value theorem to get that (5.11.7) is as small as we like when j, l are sufficiently large, uniformly over x, w ∈ [a, b] with x ≠ w. This is related to Theorem 7.17 on p152 of [155].

More on sums and norms

6.1 Weierstrass’ criterion

Let X be a nonempty set, and let a_{1}(x), a_{2}(x), a_{3}(x), … be an infinite sequence of complex-valued functions on X. Also let A_{1}, A_{2}, A_{3}, … be an infinite sequence of nonnegative real numbers such that

|a_{j}(x)| ≤ A_{j} (6.1.1)

for every x ∈ X and j ≥ 1. Suppose that

∑_{j=1}^{∞} A_{j} (6.1.2)

converges, which implies that

∑_{j=1}^{∞} a_{j}(x) (6.1.3)

converges absolutely for every x ∈ X. Under these conditions, the sequence of partial sums

∑_{j=1}^{n} a_{j}(x) (6.1.4)

converges to (6.1.3) uniformly on X. This is a well-known criterion of Weierstrass for uniform convergence.

To see this, observe that

|∑_{j=1}^{∞} a_{j}(x) − ∑_{j=1}^{n} a_{j}(x)| = | ∑_{j=n+1}^{∞} a_{j}(x)| ≤ ∑_{j=n+1}^{∞} |a_{j}(x)| ≤ ∑_{j=n+1}^{∞} A_{j} (6.1.5)

for every x ∈ X and n ≥ 1. The convergence of (6.1.2) implies that the right side of (6.1.5) tends to 0 as n → ∞. It follows that (6.1.4) converges to (6.1.3) uniformly on X, because the right side of (6.1.5) does not depend on x.

Suppose that (X, d(·, ·)) is a metric space, and that a_{j}(x) is a continuous complex-valued function on X for each j ≥ 1, with respect to the standard Euclidean metric on the complex plane C. This implies that the partial sums (6.1.4) are continuous on X as well. If (6.1.2) converges, then it follows that (6.1.3) is continuous on X too.

Now let

∑_{j=0}^{∞} a_{j} z^{j} (6.1.6)

be a power series with complex coefficients. Suppose that

∑_{j=0}^{∞} |a_{j}| r^{j} (6.1.7)

converges for some nonnegative real number r, which implies that (6.1.6) converges absolutely for every z ∈ C with |z| ≤ r. Using Weierstrass’ criterion, we get that the partial sums

∑_{j=0}^{n} a_{j} z^{j} (6.1.8)

converge to (6.1.6) uniformly on the closed disk

{z ∈ C : |z| ≤ r}. (6.1.9)

Of course, (6.1.8) is continuous as a mapping from C into itself for each n ≥ 0, using the standard metric on C. In particular, the restriction of (6.1.8) to (6.1.9) is continuous with respect to the restriction of the standard metric on C to (6.1.9), so that (6.1.6) is continuous on (6.1.9) as well.

Suppose that 0 < ρ ≤ ∞ has the property that (6.1.7) converges when 0 ≤ r < ρ. This implies that (6.1.6) converges absolutely for every z ∈ C with |z| < ρ. Under these conditions, (6.1.6) defines a continuous complex-valued function on {z ∈ C : |z| < ρ}, (6.1.10)

with respect to the restriction of the standard metric on C to (6.1.10). To see this, let z_{0} ∈ C with |z_{0}| < ρ be given, and let us check that (6.1.6) is continuous at z_{0}. Let r be a positive real number such that |z_{0}| < r < ρ. The remarks in the preceding paragraph imply that (6.1.6) is continuous on (6.1.9). One can use this to verify that (6.1.6) is continuous at z_{0} as a complex-valued function on (6.1.10), because |z_{0}| < r.

6.2 Radius of convergence

Let

∑_{j=0}^{∞} a_{j} z^{j} (6.2.1)

be a power series with complex coefficients, and let A be the set of nonnegative real numbers r such that

∑_{j=0}^{∞} |a_{j}| r^{j} (6.2.2)

converges. Of course, 0 ∈ A. If r ∈ A, then

[0, r] ⊆ A, (6.2.3)

by the comparison test. If t is a positive real number such that t ∉ A, then it follows that [t, +∞) ∩ A = ∅. (6.2.4)

The radius of convergence of (6.2.1) can be defined as a nonnegative extended real number by R = sup A. (6.2.5)

If A = [0, ∞), then R = +∞. Otherwise, R < +∞, and A is either [0, R) or [0, R]. If z ∈ C and |z| < R, then (6.2.1) converges absolutely. In fact, (6.2.1) defines a continuous function on

{z ∈ C : |z| < R}, (6.2.6)

as in the previous section.

Let t be a positive real number such that {|a_{j}| t^{j}}_{j=0}^{∞} is a bounded sequence of nonnegative real numbers. This means that there is a nonnegative real number C such that |a_{j}| t^{j} ≤ C (6.2.7)

for every j ≥ 0. If r is any nonnegative real number, then we get that

|a_{j}| r^{j} ≤ C (r/t)^{j} (6.2.8)

for every j ≥ 0. If r < t, then it follows that (6.2.2) converges, by comparison with the convergent geometric series ∑_{j=0}^{∞}(r/t)^{j}. This implies that

[0, t) ⊆ A, (6.2.9)

so that t ≤ R.

If (6.2.1) converges for some z ∈ C, then

lim_{j→∞} a_{j} z^{j} = 0. (6.2.10)

This implies that {a_{j} z^{j}}_{j=0}^{∞} is a bounded sequence of complex numbers, which is the same as saying that {|a_{j}| |z|^{j}}_{j=0}^{∞} is a bounded sequence of nonnegative real numbers. It follows that |z| ≤ R, as in the preceding paragraph.

Suppose that (6.2.2) converges for some positive real number r. If r_{0} is a nonnegative real number strictly less that r, then

lim_{j→∞} j (r_{0}/r)^{j} = 0, (6.2.11)

because r_{0}/r < 1. In particular, {j (r_{0}/r)^{j}}_{j=0}^{∞} is a bounded sequence of nonnegative real numbers, so that there is a nonnegative real number C_{0} such that

j (r_{0}/r)^{j} ≤ C_{0} (6.2.12)

for every j ≥ 0. Thus j |a_{j}| r_{0}^{j} ≤ C_{0} |a_{j}| r^{j} (6.2.13)

for every j ≥ 0, which implies that

∑_{j=0}^{∞} j |a_{j}| r_{0}^{j} (6.2.14)

converges, by the comparison test. It follows that (6.2.14) converges when r_{0} is strictly less than the radius of convergence R.

6.3 Termwise differentiation

Let a, b be real numbers with a < b, and let {f_{j}}_{j=1}^{∞} be a sequence of continuous real or complex-valued functions on [a, b] that converges uniformly to a real or complex-valued function f on [a, b], as appropriate. Thus f is also continuous on [a, b], as in Section 1.8. Of course, we are implicitly using the standard Euclidean metrics on R and C here, and the restriction of the standard Euclidean metric on R to [a, b]. If x ∈ [a, b], then put

F_{j}(x) = ∫_{a}^{x} f_{j}(t) dt (6.3.1)

for every j ∈ Z_{+}, and

F (x) = ∫_{a}^{x} f(t) dt, (6.3.2)

using standard Riemann integrals on the right sides of (6.3.1) and (6.3.2). Observe that

(6.3.3) |F_{j}(x) − F (x)| = |∫_{a}^{x}(f_{j}(t) − f(t)) dt|
≤ ∫_{a}^{x} |f_{j}(t) − f(t)| dt ≤ ∫_{a}^{b} |f_{j}(t) − f(t)| dt

for every j ≥ 1 and x ∈ [a, b]. This implies that {F_{j}}_{j=1}^{∞} converges uniformly to F on [a, b], because the right side of (6.3.3) tends to 0 as j → ∞, and does not depend on x. We also have that F′_{j}(x) = f_{j}(x) for every j ≥ 1 and x ∈ [a, b], and that F′(x) = f(x) for every x ∈ [a, b], using the appropriate one-sided derivative when x = a or b.

Now let {g_{j}}_{j=1}^{∞} be a sequence of continuously-differentiable real or complexvalued functions on [a, b]. Thus, for each j ∈ Z_{+}, the derivative g′_{j}(x) of g_{j} exists at every x ∈ [a, b], using the appropriate one-sided derivative when x = a or b, and g′_{j} is continuous on [a, b]. It follows that

g_{j}(x) = g_{j}(a) + ∫_{a}^{x} g′_{j}(t) dt (6.3.4)

for every j ≥ 1 and x ∈ [a, b], by the fundamental theorem of calculus. Suppose that {g_{j}(a)}_{j=1}^{∞} converges to a real or complex number g(a), as appropriate, and that {g′_{j}}_{j=1}^{∞} converges uniformly to a real or complex-valued function f on [a, b], as appropriate. Note that f is continuous on [a, b], as in Section 1.8. Let g be the real or complex-valued function defined on [a, b] by

g(x) = g(a) + ∫_{a}^{x} f(t) dt (6.3.5)

for each x ∈ [a, b]. Under these conditions, {g_{j}}_{j=1}^{∞} converges uniformly to g on [a, b], as in the preceding paragraph. Of course, g′ = f on [a, b].

Let ∑_{j=0}^{∞} a_{j} x^{j} be a power series with real or complex coefficients, and suppose that

∑_{j=0}^{∞} j |a_{j}| r^{j} (6.3.6)

converges for some positive real number r. In particular, this implies that

∑_{j=0}^{∞} |a_{j}| r^{j} (6.3.7)

converges, and we put

f(x) = ∑_{j=0}^{∞} a_{j} x^{j} (6.3.8)

for every x ∈ R with |x| ≤ r. Similarly, put

ϕ(x) = ∑_{j=1}^{∞} j a_{j} x^{j−1} (6.3.9)

for every x ∈ R with |x| ≤ r, where the series on the right converges absolutely because of the convergence of (6.3.6). Under these conditions, the partial sums of the series on the right sides of (6.3.8) and (6.3.9) converge uniformly on [−r, r], as in Section 6.1. By construction, the partial sums of the right side of (6.3.9) are the same as the first derivatives of the partial sums of the right side of (6.3.8). Using the remarks in the previous paragraph, we get that f is differentiable on [−r, r], with

f′(x) = ϕ(x) (6.3.10)

for every x ∈ [−r, r]. This uses the appropriate one-sided derivatives when x = ±r, as usual.

Suppose that 0 < ρ ≤ +∞ has the property that (6.3.7) converges when 0 ≤ r < ρ. This implies that (6.3.6) converges when 0 ≤ r < ρ too. More precisely, if ∑_{j=0}^{∞} |a_{j}| t^{j} converges for some t > r, then (6.3.6) converges, as in the previous section. In this situation, the series on the right sides of (6.3.8) and (6.3.9) converge absolutely for every x ∈ R with |x| < ρ, so that f(x) and ϕ(x) may be defined on (−ρ, ρ) as before. Using the remarks in the preceding paragraph, we get that f is differentiable on (−ρ, ρ), with derivative given by (6.3.10).

6.4 Cauchy products

Let ∑_{j=0}^{∞} a_{j} and ∑_{l=0}^{∞} b_{l} be infinite seris of complex numbers, and put

c_{n} = ∑_{j=0}^{n} a_{j} b_{n−j} (6.4.1)

for each nonnegative integer n. The infinite series ∑_{n=0}^{∞} c_{n} is the Cauchy product of the series ∑_{j=0}^{∞} a_{j} and ∑_{l=0}^{∞} b_{l}. It is easy to see that

∑_{n=0}^{∞} c_{n} = ( ∑_{j=0}^{∞} a_{j}) ( ∑_{l=0}^{∞} b_{l}) (6.4.2)

formally. In particular, if a_{j} = 0 for all but finitely many j ≥ 0, and b_{l} = 0 for all but finitely many l ≥ 0, then one can check that c_{n} = 0 for all but finitely many n ≥ 0, and that (6.4.2) holds.

Suppose for the moment that the a_{j}’s and b_{l}’s are nonnegative real numbers, so that the c_{n}’s are nonnegative real numbers too. Observe that

∑_{n=0}^{N} c_{n} ≤ ( ∑_{j=0}^{N} a_{j}) ( ∑_{l=0}^{N} b_{l}) (6.4.3)

for every nonnegative integer N. If ∑_{j=0}^{∞} a_{j} and ∑_{l=0}^{∞} b_{l} converge, then

∑_{n=0}^{N} c_{n} ≤ ( ∑_{j=0}^{∞} a_{j}) ( ∑_{l=0}^{∞} b_{l}) (6.4.4)

for every N ≥ 0. This implies that ∑_{n=0}^{∞} c_{n} converges, with

∑_{n=0}^{∞} c_{n} ≤ ( ∑_{j=0}^{∞} a_{j}) ( ∑_{l=0}^{∞} b_{l}). (6.4.5)

We also have that

( ∑_{j=0}^{N} a_{j}) ( ∑_{l=0}^{N} b_{l}) ≤ ∑_{n=0}^{2 N} c_{n} ≤ ∑_{n=0}^{∞} c_{n} (6.4.6)

for every N ≥ 0. If ∑_{j=0}^{∞} a_{j} and ∑_{l=0}^{∞} b_{l} converge, then we get that

( ∑_{j=0}^{∞} a_{j}) ( ∑_{l=0}^{∞} b_{l}) ≤ ∑_{n=0}^{∞} c_{n}. (6.4.7)

Of course, (6.4.2) follows from (6.4.5) and (6.4.7) in this situation.

Suppose now that ∑_{j=0}^{∞} a_{j} and ∑_{l=0}^{∞} b_{l} are absolutely convergent series of complex numbers. Clearly

|c_{n}| ≤ ∑_{j=0}^{n} |a_{j}| |b_{n−j}| (6.4.8)

for each n ≥ 0, by the triangle inequality. The right side of (6.4.8) is the same as the nth term of the Cauchy product of ∑_{j=0}^{∞} |a_{j}| and ∑_{l=0}^{∞} |b_{l}|. These two series converge, by hypothesis, and so their Cauchy product converges as well, as in the previous paragraph. This implies that ∑_{n=0}^{∞} c_{n} converges absolutely, with

∑_{n=0}^{∞} |c_{n}| ≤ ∑_{n=0}^{∞} ( ∑_{j=0}^{n} |a_{j}| |b_{l}|) = ( ∑_{j=0}^{∞} |a_{j}|) ( ∑_{l=0}^{∞} |b_{l}|). (6.4.9)

If the a_{j}’s and b_{l}’s are real numbers, then ∑_{j=0}^{∞} a_{j} and ∑_{l=0}^{∞} b_{l} can be expressed as differences of convergent series of nonnegative real numbers. In this case, (6.4.2) can be obtained from the corresponding statement for nonnegative real numbers, as in the previous paragraph. If the a_{j}’s and b_{l}’s are complex numbers, then one can get (6.4.2) using the analogous statements for the real and imaginary parts of the a_{j}’s and b_{l}’s.

Alternatively, consider

∑_{j,l≥0} a_{j} b_{l}, (6.4.10)

where more precisely the sum is taken over all ordered pairs (j, l) of nonnegative integers j, l. This sum can be identified formally with both sides of (6.4.2). The left side of (6.4.2) corresponds to summing first over (j, l) such that j + l = n, and then summing over n ≥ 0. The right side of (6.4.2) can be obtained by summing over j and l separately. If a_{j} = 0 for all but finitely many j ≥ 0, and b_{l} = 0 for all but finitely many l ≥ 0, then a_{j} b_{l} = 0 for all but finitely many (j, l), and all of these sums can be reduced to finite sums. If the a_{j}’s and b_{l}’s are nonnegative real numbers, then (6.4.10) can be defined as a nonnegative extended real number, as in Section 11.2. This sum can be expressed in terms of iterated sums, as in Section 11.15. In particular, one can check that (6.4.10) is finite when ∑_{j=0}^{∞} a_{j} and ∑_{l=0}^{∞} b_{l} converge, in which case (6.4.10) is the same as both sides of (6.4.2). Similarly, if ∑_{j=0}^{∞} a_{j} and ∑_{l=0}^{∞} b_{l} are absolutely convergent series of complex numbers, then

∑_{j,l≥0} |a_{j}| |b_{l}| = ( ∑_{j=0}^{∞} |a_{j}|) ( ∑_{l=0}^{∞} |b_{l}|). (6.4.11)

Thus a_{j} b_{l} is a summable complex-valued function of (j, l), so that (6.4.10) can be defined as in Section 11.8. In this situation, (6.4.10) can be expressed in terms of iterated sums, as in Section 11.16. This can be used to get that ∑_{n=0}^{∞} c_{n} converges absolutely, and that both sides of (6.4.2) are equal to (6.4.10).

Suppose now that ∑_{j=0}^{∞} a_{j} z^{j} and ∑_{l=0}^{∞} b_{l} z^{l} are power series with complex coefficients. One can check that their Cauchy product is the power series

∑_{n=0}^{∞} c_{n} z^{n}, (6.4.12)

where c_{n} is as in (6.4.1) for each n ≥ 0.

6.5 Rearrangements

Let ∑_{j=1}^{∞} a_{j} be an infinite series of real or complex numbers, and let π be a one-to-one mapping from the set Z_{+} onto itself. Under these conditions, the infinite series

∑_{l=1}^{∞} a_{π(l)} (6.5.1)

is called a rearrangement of ∑_{j=1}^{∞} a_{j}. If a_{j} = 0 for all but finitely many positive integers j, then a_{π(l)} = 0 for all but finitely many l too, and it is easy to see that

∑_{l=1}^{∞} a_{π(l)} = ∑_{j=1}^{∞} a_{j}. (6.5.2)

Suppose for the moment that a_{j} is a nonnegative real number for each j ≥ 1. If n ∈ Z_{+}, then

∑_{l=1}^{n} a_{π(l)} ≤ ∑_{j=1}^{N} a_{j} (6.5.3)

for every N ≥ max_{1≤l≤n} π(l). Similarly,

∑_{j=1}^{n} a_{j} ≤ ∑_{l=1}^{N} a_{π(l)} (6.5.4)

for every N ≥ max_{1≤j≤n} π^{−1}(j). This implies that ∑_{j=1}^{∞} a_{j} converges if and only if ∑_{l=1}^{∞} a_{π(l)} converges, in which case (6.5.2) holds.

If ∑_{j=1}^{∞} a_{j} is an infinite series of real or complex numbers, then it follows that ∑_{j=1}^{∞} a_{j} converges absolutely if and only if ∑_{l=1}^{∞} a_{π(l)} converges absolutely. One can check that (6.5.2) holds in this situation as well. More precisely, if the a_{j}’s are real numbers, then one can reduce to the previous case by expressing ∑_{j=1}^{∞} a_{j} as a difference of convergent series of nonnegative real numbers. If the a_{j}’s are complex numbers, then one can consider their real and imaginary parts.

6.6. COMPARING NORMS ON R^{N}, C^{N} 93

6.6 Comparing norms on R^{n}, C^{n}

Let n be a positive integer, and let

e_{j} = (e_{j,1}, …, e_{j,n}) (6.6.1)

be the jth standard basis vector in R^{n} for each j = 1, …, n. Thus

(6.6.2)  e_{j,l} = 1 when j = l
= 0 when j ≠ l.

If v = (v_{1}, …, v_{n}) ∈ R^{n}, then v can be expressed as

v = ∑_{j=1}^{n} v_{j} e_{j}, (6.6.3)

where the right side is a linear combination of elements of R^{n}. The e_{j}’s may also be considered as standard basis vectors in C^{n}. If v = (v_{1}, …, v_{n}) ∈ C^{n}, then v can be expressed as in (6.6.3) again, where now the right side is a linear combination of elements of C^{n}.

Let N be a norm on R^{n} or C^{n}, as in Sections 1.3 and 1.4. If v = (v_{1}, …, v_{n}) is an element of R^{n} or C^{n}, as appropriate, then

N(v) = N( ∑_{j=1}^{n} v_{j} e_{j}), (6.6.4)

by (6.6.3). This implies that

N(v) ≤ ∑_{j=1}^{n} N(v_{j} e_{j}) = ∑_{j=1}^{n} |v_{j}| N(e_{j}), (6.6.5)

using the triangle inequality for N in the first step, and the homogeneity condition for N in the second step. Note that this argument also works for seminorms instead of norms.

It follows that

N(v) ≤ ( max_{1≤l≤n} N(e_{l})) ∑_{j=1}^{n} |v_{j}| = ( max_{1≤l≤n} N(e_{l})) ∥v∥_{1}, (6.6.6)

where ∥v∥_{1} is as in (1.3.6) or (1.4.2), as appropriate. Remember that

d_{N}(v, w) = N(v − w) (6.6.7)

is the metric on R^{n} or C^{n}, as appropriate, associated to N, as in (1.3.9) and (1.4.4). Similarly, we let d_{1}(v, w) = ∥v − w∥_{1} (6.6.8) be the metric on R^{n} or C^{n}, as appropriate, associated to ∥ · ∥_{1}, as in (1.3.11) and (1.4.6). Thus

d_{N}(v, w) ≤ ( max_{1≤l≤n} N(e_{l})) d_{1}(v, w) (6.6.9)

for every v, w in R^{n} or C^{n}, as appropriate. In particular, the identity mapping on R^{n} or C^{n}, as appropriate, is Lipschitz with respect to (6.6.8) on the domain and (6.6.7) on the range.

Observe that

N(v) ≤ ( ∑_{l=1}^{n} N(e_{j})^{2})^{1/2} ( ∑_{j=1}^{n} |v_{j}|^{2})^{1/2} (6.6.10)

for every v ∈ R^{n} or C^{n}, as appropriate, using the Cauchy–Schwarz inequality (1.3.13) on the right side of (6.6.5). Equivalently, this means that

N(v) ≤ ( ∑_{l=1}^{n} N(e_{j})^{2})^{1/2} ∥v∥_{2}, (6.6.11)

where ∥v∥_{2} is the standard Euclidean norm of v, as in (1.3.5) or (1.4.1), as appropriate. Remember that

d_{2}(v, w) = ∥v − w∥_{2} (6.6.12)

is the standard Euclidean metric on R^{n} or C^{n}, as appropriate, as in (1.3.10) and (1.4.5). It follows that

d_{N}(v, w) ≤ ( ∑_{l=1}^{n} N(e_{l})^{2})^{1/2} d_{2}(v, w) (6.6.13)

for every v, w in R^{n} or C^{n}, as appropriate. This implies that the identity mapping on R^{n} or C^{n}, as appropriate, is Lipschitz with respect to (6.6.12) on the domain and (6.6.7) on the range.

Using (6.6.5), we also get that

N(v) ≤ ( ∑_{j=1}^{n} N(e_{j})) max_{1≤l≤n} |v_{l}| = ( ∑_{j=1}^{n} N(e_{j})) ∥v∥_{∞} (6.6.14)

for every v ∈ R^{n} or C^{n}, as appropriate. Here ∥v∥_{∞} is as in (1.3.7) or (1.4.3), as appropriate. Let d_{∞}(v, w) = ∥v − w∥_{∞} (6.6.15) be the metric on R^{n} or C^{n}, as appropriate, associated to ∥ · ∥_{∞} as in (1.3.12) and (1.4.7). Note that

d_{N}(v, w) ≤ ( ∑_{j=1}^{n} N(e_{j})) d_{∞}(v, w) (6.6.16)

for every v, w in R^{n} or C^{n}, as appropriate. Hence the identity mapping on R^{n} or C^{n}, as appropriate, is Lipschitz with respect to (6.6.15) on the domain and (6.6.7) on the range.

6.7 Another comparison

Let N be a norm on R^{n} or C^{n} for some n ∈ Z_{+} again. Observe that

N(v) ≤ N(w) + N(v − w) (6.7.1)

and N(w) ≤ N(v) + N(v − w) (6.7.2)

for every v, w in R^{n} or C^{n}, as appropriate. This implies that

|N(v) − N(w)| = max(N(v) − N(w), N(w) − N(v)) ≤ N(v − w), (6.7.3)

for every v, w in R^{n} or C^{n}, as appropriate. More precisely, the left side of (6.7.3) is the usual absolute value of N(v) − N(w), as a real number.

Remember that N(v) ≤ C ∥v∥_{2} (6.7.4)

for some nonnegative real number C and every v ∈ R^{n} or C^{n}, as appropriate, as in (6.6.11). Combining this with (6.7.3), we get that

|N(v) − N(w)| ≤ C ∥v − w∥_{2} = C d_{2}(v, w) (6.7.5)

for every v, w in R^{n} or C^{n}, as appropriate. Thus N is Lipschitz as a realvalued function on R^{n} or C^{n}, as appropriate, using the corresponding Euclidean metrics on the domain and range. In particular, N is continuous. This argument works for seminorms instead of norms as well.

The extreme value theorem implies that there is a u ∈ R^{n} or C^{n}, as appropriate, such that ∥u∥_{2} = 1 and

N(u) ≤ N(w) (6.7.6)

for every w ∈ R^{n} or C^{n}, as appropriate, with ∥w∥_{2} = 1. This uses the fact that the unit sphere in R^{n} or C^{n}, as appropriate, is compact with respect to the Euclidean metric. Put c = N(u), (6.7.7)

and note that c > 0, because N is a norm. Let us check that

c ∥v∥_{2} ≤ N(v) (6.7.8)

for every v ∈ R^{n} or C^{n}. Of course, (6.7.8) is trivial when v = 0, and so we may suppose that v ≠ 0. In this case, ∥v∥_{2} > 0, and

w = \frac{v}{∥v∥_{2}} (6.7.9)

satisfies ∥w∥_{2} = 1. Thus

N(v)/∥v∥_{2} = N(v/∥v∥_{2}) = N(w) ≥ N(u) = c, (6.7.10) using (6.7.6) in the third step. This implies (6.7.8), as desired.

Put C′ = 1/c, so that (6.7.8) is the same as saying that

∥v∥_{2} ≤ C′ N(v) (6.7.11)

for every v ∈ R^{n} or C^{n}, as appropriate. It follows that

d_{2}(v, w) = ∥v − w∥_{2} ≤ C′ N(v − w) = C′ d_{N}(v, w) (6.7.12)

for every v, w in R^{n} or C^{n}, as appropriate. In particular, this implies that the identity mapping on R^{n} or C^{n}, as appropriate, is Lipschitz with respect to the metric d_{N}(v, w) associated to N on the domain, and the standard Euclidean metric on the range. Of course, there are analogous arguments using ∥ · ∥_{1} or ∥ · ∥_{∞} here, instead of the Euclidean norm ∥ · ∥_{2}.

6.8 Inner products on R^{n}, C^{n}

Let n be a positive integer, and let x, y ∈ R^{n} be given. Thus

⟨x, y⟩ = ⟨x, y⟩_{R^{n}} = ∑_{j=1}^{n} x_{j} y_{j} (6.8.1)

is defined as a real number. This is the standard inner product on R^{n}. Note that

⟨x, x⟩_{R^{n}} = ∑_{j=1}^{n} x_{j}^{2} = ∥x∥_{2}^{2}, (6.8.2)

where ∥x∥_{2} is the standard Euclidean norm of x, as in (1.3.5). We also have that

|⟨x, y⟩_{R^{n}}| = |∑_{j=1}^{n} x_{j} y_{j}| ≤ ∑_{j=1}^{n} |x_{j}| |y_{j}| ≤ ∥x∥_{2} ∥y∥_{2}, (6.8.3)

using the Cauchy–Schwarz inequality (1.3.13) in the third step.

Let z = x + i y be a complex number, where x and y are real numbers. Remember that the complex conjugate of z is the complex number defined by

\overline{z} = x − i y. (6.8.4)

If w is another complex number, then

\overline{(w + z)} = \overline{w} + \overline{z} (6.8.5)

and \overline{(w z)} = \overline{w} \overline{z}. (6.8.6)

It is easy to see that z \overline{z} = x^{2} + y^{2} = |z|^{2}, (6.8.7) where |z| is the usual absolute value of z. The well-known fact that

|w z| = |z| |w| (6.8.8)

follows from (6.8.6) and (6.8.7).

Clearly \overline{(\overline{z})} = z (6.8.9)

and |\overline{z}| = |z|. (6.8.10)

Let u, v ∈ R be the real and imaginary parts of w, so that w = u + i v. The real part of w \overline{z} is given by Re(w \overline{z}) = u x + v y. (6.8.11)

As usual, w and z correspond to (u, v) and (x, y), respectively, as elements of R^{2}. The right side of (6.8.11) is the same as the inner product of (u, v) and (x, y) as elements of R^{2}.

If w, z ∈ C^{n}, then

⟨w, z⟩ = ⟨w, z⟩_{C^{n}} = ∑_{j=1}^{n} w_{j} \overline{z_{j}} (6.8.12)

is defined as a complex number. This is the standard inner product on C^{n}. Observe that

\overline{⟨w, z⟩_{C^{n}}} = \overline{( ∑_{j=1}^{n} w_{j} \overline{z_{j}})} = ∑_{j=1}^{n} z_{j} \overline{w_{j}} = ⟨z, w⟩_{C^{n}}. (6.8.13)

As before,

⟨z, z⟩_{C^{n}} = ∑_{j=1}^{n} z_{j} \overline{z_{j}} = ∑_{j=1}^{n} |z_{j}|^{2} = ∥z∥_{2}^{2}, (6.8.14)

where ∥z∥_{2} is the standard Euclidean norm of z, as in (1.4.1). Moreover,

|⟨w, z⟩_{C^{n}}| = |∑_{j=1}^{n} w_{j} \overline{z_{j}}| ≤ ∑_{j=1}^{n} |w_{j}| |z_{j}| ≤ ∥w∥_{2} ∥z∥_{2}, (6.8.15)

using the Cauchy–Schwarz inequality (1.3.13) in the third step.

6.9 Sums and inner products

Let X be a nonempty set, and let f, g be real-valued functions on X with finite support in X. Put

⟨f, g⟩ = ⟨f, g⟩_{c_{00}(X,R)} = ∑_{x∈X} f(x) g(x), (6.9.1)

where the sum on the right reduces to a finite sum of real numbers, as in Section 1.6. This may be considered as the standard inner product on c_{00}(X, R). Of course,

⟨f, f⟩_{c_{00}(X,R)} = ∑_{x∈X} f(x)^{2} = ∥f∥_{2}^{2}, (6.9.2)

where ∥f∥_{2} is as in Section 1.6. Observe that

|⟨f, g⟩_{c_{00}(X,R)}| = |∑_{x∈X} f(x) g(x)| ≤ ∑_{x∈X} |f(x)| |g(x)| ≤ ∥f∥_{2} ∥g∥_{2}, (6.9.3)

by the Cauchy–Schwarz inequality.

Similarly, if f, g are complex-valued functions on X with finite support, then

⟨f, g⟩ = ⟨f, g⟩_{c_{00}(X,C)} = ∑_{x∈X} f(x) \overline{g(x)} (6.9.4)

reduces to a finite sum of complex numbers. This may be considered as the standard inner product on c_{00}(X, C). As before,

⟨f, f⟩_{c_{00}(X,C)} = ∑_{x∈X} |f(x)|^{2} = ∥f∥_{2}^{2} (6.9.5)

for every f ∈ c_{00}(X, C). Using the Cauchy–Schwarz inequality again, we get that

|⟨f, g⟩_{c_{00}(X,C)}| = |∑_{x∈X} f(x) \overline{g(x)}| ≤ ∑_{x∈X} |f(x)| |g(x)| ≤ ∥f∥_{2} ∥g∥_{2} (6.9.6)

for every f, g ∈ c_{00}(X, C). We also have that

\overline{⟨f, g⟩_{c_{00}(X,C)}} = \overline{( ∑_{x∈X} f(x) \overline{g(x)})} = ∑_{x∈X} g(x) \overline{f(x)} = ⟨g, f⟩_{c_{00}(X,C)} (6.9.7)

for every f, g ∈ c_{00}(X, C).

Suppose now that f, g ∈ ℓ^{2}(Z_{+}, R), and remember that f g ∈ ℓ^{1}(Z_{+}, R), as in Section 2.3. In this case, we put

⟨f, g⟩ = ⟨f, g⟩_{ℓ^{2}(Z_{+},R)} = ∑_{j=1}^{∞} f(j) g(j), (6.9.8)

where the right side converge absolutely. This may be considered as the standard inner product on ℓ^{2}(Z_{+}, R). Note that

⟨f, f⟩_{ℓ^{2}(Z_{+},R)} = ∑_{j=1}^{∞} f(j)^{2} = ∥f∥_{2}^{2}, (6.9.9)

where ∥f∥_{2} is as in Section 2.3. The version of the Cauchy–Schwarz inequality mentioned in Section 2.3 implies that

|⟨f, g⟩_{ℓ^{2}(Z_{+},R)}| = |∑_{j=1}^{∞} f(j) g(j)| ≤ ∑_{j=1}^{∞} |f(j)| |g(j)| ≤ ∥f∥_{2} ∥g∥_{2}. (6.9.10)

Let f, g ∈ ℓ^{2}(Z_{+}, C) be given, so that |f| |g| ∈ ℓ^{1}(Z_{+}, C), and hence f \overline{g} is an element of ℓ^{1}(Z_{+}, C). Put

⟨f, g⟩ = ⟨f, g⟩_{ℓ^{2}(Z_{+},C)} = ∑_{j=1}^{∞} f(j) \overline{g(j)}, (6.9.11)

which may be considered as the standard inner product on ℓ^{2}(Z_{+}, C). As usual,

⟨f, f⟩_{ℓ^{2}(Z_{+},C)} = ∑_{j=1}^{∞} |f(j)|^{2} = ∥f∥_{2}^{2}. (6.9.12)

We also have that

|⟨f, g⟩_{ℓ^{2}(Z_{+},C)}| = |∑_{j=1}^{∞} f(j) \overline{g(j)}| ≤ ∑_{j=1}^{∞} |f(j)| |g(j)| ≤ ∥f∥_{2} ∥g∥_{2}, (6.9.13)

by the version of the Cauchy–Schwarz inequality in Section 2.3. Moreover,

\overline{⟨f, g⟩_{ℓ^{2}(Z_{+},C)}} = \overline{( ∑_{j=1}^{∞} f(j) \overline{g(j)})} = ∑_{j=1}^{∞} g(j) \overline{f(j)} = ⟨g, f⟩_{ℓ^{2}(Z_{+},C)}. (6.9.14)

6.10 Integral inner products

Let a, b be real numbers with a < b, and let α be a monotonically increasing real-valued function on [a, b]. If f, g are continuous real-valued functions on [a, b], then

⟨f, g⟩ = ⟨f, g⟩_{α} = ∫_{a}^{b} f(x) g(x) dα(x) (6.10.1)

is defined as a real number, using a Riemann–Stieltjes integral on the right side. Clearly

⟨f, f⟩_{α} = ∫_{a}^{b} f(x)^{2} dα(x) = ∥f∥_{2,α}^{2}, (6.10.2)

where ∥f∥_{2,α} is as in (3.5.1). Observe that

(6.10.3)  |⟨f, g⟩_{α}| = |∫_{a}^{b} f(x) g(x) dα(x)|
≤ ∫_{a}^{b} |f(x)| |g(x)| dα(x) ≤ ∥f∥_{2,α} ∥g∥_{2,α},

by the integral version (3.5.8) of the Cauchy–Schwarz inequality for Riemann– Stieltjes integrals. Of course, (6.10.1) is symmetric in f and g, and linear in each of f and g, because of linearity of the integral. If α is strictly increasing on [a, b], then (6.10.2) is positive when f ≢ 0 on [a, b], as before. In this case, (6.10.1) defines an inner product on the space C([a, b], R) of continuous realvalued functions on [a, b].

If f and g are continuous complex-valued functions on [a, b], then

⟨f, g⟩ = ⟨f, g⟩_{α} = ∫_{a}^{b} f(x) \overline{g(x)} dα(x) (6.10.4)

is defined as a complex number, using a Riemann–Stieltjes integral on the right side again. In this situation, we also have that

⟨f, f⟩_{α} = ∫_{a}^{b} |f(x)|^{2} dα(x) = ∥f∥_{2,α}^{2}, (6.10.5)

where ∥f∥_{2,α} is as in (3.5.1). As before,

(6.10.6)  |⟨f, g⟩_{α}| = |∫_{a}^{b} f(x) \overline{g(x)} dα(x)|
≤ ∫_{a}^{b} |f(x)| |g(x)| dα(x) ≤ ∥f∥_{2,α} ∥g∥_{2,α},

using (3.5.8) in the third step. Note that

\overline{⟨f, g⟩_{α}} = \overline{( ∫_{a}^{b} f(x) \overline{g(x)} dα(x))} = ∫_{a}^{b} g(x) \overline{f(x)} dα(x) = ⟨g, f⟩_{α}. (6.10.7)

It is easy to see that (6.10.4) is linear in f, because of linearity of the integral. Similarly, (6.10.4) is conjugate-linear in g, which is to say that it is additive in g, while multiplying g by a complex number t corresponds to multiplying (6.10.4) by \overline{t}. If α is strictly increasing on [a, b], then (6.10.5) is positive when f ≢ 0 on [a, b], and (6.10.4) defines an inner product on the space C([a, b], C) of continuous complex-valued functions on [a, b]. If α(x) = x for every x ∈ [a, b], then (6.10.1) and (6.10.4) are called the standard integral inner products on C([a, b], R) and C([a, b], C), respectively.

Now let f, g be continuous real-valued functions on the real line with compact support. In this case,

⟨f, g⟩ = ⟨f, g⟩_{C_{com}(R,R)} = ∫_{−∞}^{∞} f(x) g(x) dx (6.10.8)

is defined as a real number, where the integral on the right reduces to a Riemann integral over a bounded interval, as in Section 3.8. This is the standard integral inner product on the space C_{com}(R, R) of continuous real-valued functions on R with compact support. As before,

⟨f, f⟩_{C_{com}(R,R)} = ∫_{−∞}^{∞} f(x)^{2} dx = ∥f∥_{2}^{2}, (6.10.9)

6.11. SOME REMARKS ABOUT N-DIMENSIONAL VOLUME 101

where ∥f∥_{2} is as defined in (3.8.15). We also have that

(6.10.10) |⟨f, g⟩_{C_{com}(R,R)}| = |∫_{−∞}^{∞} f(x) g(x) dx|
≤ ∫_{∞}^{∞} |f(x)| |g(x)| dx ≤ ∥f∥_{2} ∥g∥_{2},

where the third step reduces to the integral version (3.2.4) of the Cauchy– Schwarz inequality on bounded intervals. Clearly (6.10.8) is symmetric in f and g. The inner product (6.10.8) is linear in f and g, because of the linearity of the integral.

Similarly, if f and g are continuous complex-valued functions on R with compact support, then

⟨f, g⟩ = ⟨f, g⟩_{C_{com}(R,C)} = ∫_{−∞}^{∞} f(x) \overline{g(x)} dx (6.10.11)

is defined as a complex number. This is the standard integral inner product on the space C_{com}(R, C) of continuous complex-valued functions on R with compact support. As usual,

⟨f, f⟩_{C_{com}(R,C)} = ∫_{−∞}^{∞} |f(x)|^{2} dx = ∥f∥_{2}^{2}, (6.10.12)

where ∥f∥_{2} is as in (3.8.15) again. In addition,

(6.10.13) |⟨f, g⟩_{C_{com}(R,C)}| = |∫_{−∞}^{∞} f(x) \overline{g(x)} dx|
≤ ∫_{−∞}^{∞} |f(x)| |g(x)| dx ≤ ∥f∥_{2} ∥g∥_{2},

using (3.2.4) in the third step. Observe that

(6.10.14) \overline{⟨f, g⟩_{C_{com}(R,C)}} = \overline{( ∫_{−∞}^{∞} f(x) \overline{g(x)} dx)}
= ∫_{−∞}^{∞} g(x) \overline{f(x)} dx = ⟨g, f⟩_{C_{com}(R,C)}.

As before, (6.10.11) is linear in f, because of the linearity of the integral. The inner product (6.10.11) is also conjugate-linear in g.

6.11 Some remarks about n-dimensional volume

Let n be a positive integer. If E is a reasonably nice subset of R^{n}, then the n-dimensional volume Vol_{n}(E) (6.11.1) of E may be defined in the usual way. In fact, if n-dimensional volume on R^{n} is interpreted as n-dimensional Lebesgue measure, then this is defined for Lebesgue measurable subsets of R^{n}. This includes all open and closed subsets of R^{n}, as well as their countable unions or intersections. Note that Vol_{n}(E) may be +∞ when E is not bounded.

If n-dimensional volume on R^{n} is interpreted as Lebesgue outer measure, then Vol_{n}(E) is defined for all subsets E of R^{n}. However, this may not always behave as one might expect in terms of additivity of volumes of unions of disjoint subsets of R^{n}. This is related to the Banach–Tarski paradox, as in [172, 176, 182].

Let a_{1}, …, a_{n} and b_{1}, …, b_{n} be real numbers with a_{j} ≤ b_{j} for each j = 1, …, n. The set

(6.11.2) C = {x ∈ R^{n} : a_{j} ≤ x_{j} ≤ b_{j} for each j = 1, …, n}

may be called a cell in R^{n}, as on p31 of [155]. This is the same as the Cartesian product of the closed intervals [a_{j}, b_{j}], j = 1, …, n. In this case,

Vol_{n}(C) = ∏_{j=1}^{n}(b_{j} − a_{j}). (6.11.3)

In particular, this is equal to 0 when a_{j} = b_{j} for any j.

If a ∈ R^{n} and E ⊆ R^{n}, then put

E + a = {x + a : x ∈ E}, (6.11.4)

which is the translation of E by a in R^{n}. It is well known that

Vol_{n}(E + a) = Vol_{n}(E), (6.11.5)

which is to say that the n-dimensional volume on R^{n} is invariant under translations. It is easy to check directly that this is compatible with (6.11.3).

If t ∈ R and E ⊆ R^{n}, then put

t E = {t x : x ∈ E}, (6.11.6)

which corresponds to dilating E by t in R^{n}. It is also well known that

Vol_{n}(t E) = |t|^{n} Vol_{n}(E). (6.11.7)

One can check directly that this is compatible with (6.11.3) as well.

6.12 Volumes and Lipschitz mappings

Let n be a positive integer, and let N be a norm on R^{n}, so that d_{N}(x, y) = N(x − y) defines a metric on R^{n}, as usual. Observe that

d_{N}(x + a, y + a) = N((x + a) − (y + a)) = N(x − y) = d_{N}(x, y) (6.12.1) for all a, x, y ∈ R^{n}, so that d_{N}(·, ·) is invariant under translations on R^{n}. Similarly,

d_{N}(t x, t y) = N(t x − t y) = N(t (x − y)) = |t| N(x − y) = |t| d_{N}(x, y) (6.12.2)

for every x, y ∈ R^{n} and t ∈ R.

Suppose that f is a Lipschitz mapping from E ⊆ R^{n} into R^{n} with constant C ≥ 0 with respect to d_{N}(·, ·) and its restriction to E, so that

d_{N}(f(x), f(y)) ≤ C d_{N}(x, y) (6.12.3)

for every x, y ∈ E. It is well known that

Vol_{n}(f(E)) ≤ C^{n} Vol_{n}(E). (6.12.4)

One can look at this in terms of the standard way of changing variables in n-dimensional integrals, under suitable conditions. Although the latter is discussed in many textbooks, one may also be interested in the articles [25, 115, 116, 169], as well as [47, 184, 185], in connection with Lebesgue measure and integration. Of course, Lebesgue measure and integration are discussed in many textbooks too, and one may be interested in the articles [40, 58, 117] as well.

Alternatively, one can consider (6.12.4) in terms of n-dimensional Hausdorff measure on R^{n} with respect to d_{N}(·, ·). It is well known that this is equal to a constant multiple of Lebesgue outer measure on R^{n}.

If α is any positive real number, then α-dimensional Hausdorff measure may be defined on any metric space. There is an analogue of (6.12.4) for Lipschitz mappings between arbitrary metric spaces, using Hausdorff measures of the same dimension on the domain and range.

If k is a positive integer less than or equal to n, then k-dimensional Hausdorff measure on R^{n} with respect to the standard Euclidean metric is related to the usual k-dimensional volume of reasonably nice k-dimensional submanifolds of R^{n}.

6.13 Bounded vector-valued functions

Let X be a nonempty set, and let m be a positive integer. Consider the spaces

c(X, R^{m}), c(X, C^{m}) (6.13.1)

of functions on X with values in R^{m}, C^{m}, respectively. These spaces with m = 1 were discussed in Section 3.10. As before, if f and g are functions on X with values in R^{m} or C^{m}, then f + g defines a function on X with values in R^{m} or C^{m}, as appropriate. Similarly, if t is a real or complex number, as appropriate, then t f is a function on X with values in R^{m} or C^{m}, as appropriate. As usual, c(X, R^{m}) and c(X, C^{m}) are basic classes of examples of vector spaces over the real and complex numbers, respectively. We shall also be interested in linear subspaces of these spaces, which are subsets of the spaces that contain 0 and are closed under addition and scalar multiplcation, as before.

If f is a function on X with values in R^{m} or C^{m}, then let f_{j}(x) be the jth coordinate of f(x) for each j = 1, …, m and x ∈ X. Thus f_{j} is a real or complex-valued function on X, as appropriate, for each j. Of course, any m-tuple of real or complex-valued functions on X determines a function on X with values in R^{m} or C^{m}, as appropriate, in this way.

Let N be a norm on R^{m} or C^{m}, so that d_{N}(v, w) = N(v − w) defines a metric on R^{m} or C^{m}, as appropriate. Using this, we get the corresponding space B(X, R^{m}) = B_{N}(X, R^{m}) or B(X, C^{m}) = B_{N}(X, C^{m}) (6.13.2)

of mappings from X into R^{m} or C^{m}, as appropriate, that are bounded with respect to d_{N}(·, ·), as in Section 1.11. This space does not depend on the particular norm N, because any norm on R^{m} or C^{m} can be compared with the standard Euclidean norm up to constant factors, as in Sections 6.6 and 6.7. The corresponding supremum metric does depend on N, but it can be compared with the supremum metric associated to the standard Euclidean norm on R^{m} or C^{m}, as appropriate, with the same constant factors.

We may also use the notation

ℓ^{∞}(X, R^{m}) = ℓ_{N}^{∞}(X, R^{m}) or ℓ^{∞}(X, C^{m}) = ℓ_{N}^{∞}(X, C^{m}) (6.13.3)

for (6.13.2), as appropriate. Note that a function f on X with values in R^{m} or C^{m}, as appropriate, is bounded with respect to d_{N}(·, ·) if and only if

N(f(x)) (6.13.4)

is bounded as a real-valued function on X. This happens if and only if f_{j} is bounded as a real or complex-valued function on X, as appropriate, for each j, because of the usual comparisons of N with the standard Euclidean norm. It is easy to see that (6.13.3) is a linear subspace of c(X, R^{m}) or c(X, C^{m}), as appropriate.

If f is an element of (6.13.3), as appropriate, then put

∥f∥_{∞,N} = sup{N(f(x)) : x ∈ X}. (6.13.5)

This is a nonnegative real number, which is equal to 0 exactly when f = 0 on X. One can check that ∥t f∥_{∞,N} = |t| ∥f∥_{∞,N} (6.13.6)

for every t ∈ R or C, as appropriate. If g is another element of (6.13.3), as appropriate, then one can verify that

∥f + g∥_{∞,N} ≤ ∥f∥_{∞,N} + ∥g∥_{∞,N}. (6.13.7)

This means that (6.13.5) defines a norm on (6.13.3), as appropriate. This is the supremum norm associated to N. This can be compared with the supremum norm associated to the standard Euclidean norm on R^{m} or C^{m}, as appropriate, with the same constant factors as before. Note that

∥f − g∥_{∞,N} (6.13.8)

is the same as the supremum metric on (6.13.3), as appropriate, corresponding to d_{N}(·, ·).

Suppose now that (X, d_{X}) is a nonempty metric space, and consider the space C(X, R^{m}) = C_{N}(X, R^{m}) or C(X, C^{m}) = C_{N}(X, C^{m}) (6.13.9)

of continuous mappings from X into R^{m} or C^{m}, as appropriate, with respect to d_{N}(·, ·) on the range. This is the same as the space of continuous mappings from X into R^{m} or C^{m}, as appropriate, with respect to the standard Euclidean metric on the range, because of the usual comparisons between N and the standard Euclidean norm. Equivalently, one can check that a mapping f from X into R^{m} or C^{m} is continuous if and only if f_{j} is continuous as a real or complex-valued function on X, as appropriate, for each j = 1, …, m. In particular, the space of continuous mappings from X into R^{m} or C^{m} is a linear subspace of c(X, R^{m}) or c(X, C^{m}), as appropriate.

Similarly, we get the space

C_{b}(X, R^{m}) = C_{b,N}(X, R^{m}) or C_{b}(X, C^{m}) = C_{b,N}(X, C^{m}) (6.13.10)

of bounded continuous mappings from X into R^{m} or C^{m}, as appropriate, with respect to d_{N}(·, ·) on the range. This is the same as the analogous space defined using the standard Euclidean metric on the range, as before. This is a linear subspace of (6.13.3) and (6.13.9), as appropriate.

Matrix norms and Lipschitz conditions

7.1 Real and complex matrices

Let m and n be positive integers, and let us consider m×n matrices with entries in the real or complex numbers. Such a matrix may be denoted as

[a_{j,l}] = [a_{j,l}]_{j,l=1}^{m,n}, (7.1.1)

where a_{j,l} is a real or complex number, as appropriate, for each j = 1, …, m and l = 1, …, n. Let M_{m,n}(R) and M_{m,n}(C) be the spaces of m × n matrices with entries in R and C, respectively.

If [a_{j,l}] and [b_{j,l}] are m × n matrices with real or complex entries, then their sum is defined as an m × n matrix by adding the corresponding entries, so that

[a_{j,l}] + [b_{j,l}] = [a_{j,l} + b_{j,l}]. (7.1.2)

Similarly, if t is a real or complex number, as appropriate, then t times [a_{j,l}] is defined as an m × n matrix by multiplying the entries of [a_{j,l}] by t,

t [a_{j,l}] = [t a_{j,l}]. (7.1.3)

This makes M_{m,n}(R) and M_{m,n}(C) into vector spaces over the real and complex numbers, respectively. Of course, one can also identify M_{m,n}(R) and M_{m,n}(C) with R^{m n} and C^{m n}, respectively.

As usual, a nonnegative real-valued function N defined on M_{m,n}(R) or M_{m,n}(C) is said to be a norm if it satisfies the following three conditions. First, N([a_{j,l}]) = 0 (7.1.4)

if and only if [a_{j,l}] = 0 as a matrix, which means that a_{j,l} = 0 for every j = 1, …, m and l = 1, …, n. Second,

N(t [a_{j,l}]) = |t| N([a_{j,l}]) (7.1.5) for every [a_{j,l}] ∈ M_{m,n}(R) or M_{m,n}(C) and t ∈ R or C, as appropriate. Third,

N([a_{j,l}] + [b_{j,l}]) ≤ N([a_{j,l}]) + N([b_{j,l}]) (7.1.6)

for every [a_{j,l}], [b_{j,l}] ∈ M_{m,n}(R) or M_{m,n}(C), as appropriate. In this case,

d_{N}([a_{j,l}], [b_{j,l}]) = N([a_{j,l}] − [b_{j,l}]) = N([a_{j,l} − b_{j,l}]) (7.1.7)

defines a metric on M_{m,n}(R) or M_{m,n}(C), as appropriate. If M_{m,n}(R) and M_{m,n}(C) are identified with R^{m n} and C^{m n}, respectively, as before, then the definition of a norm on M_{m,n}(R) or M_{m,n}(C) corresponds exactly to the earlier definitions for R^{m n} and C^{m n} in Sections 1.3 and 1.4. Similarly, the metric associated to a norm on M_{m,n}(R) or M_{m,n}(C) corresponds to the analogous notions for R^{m n} and C^{m n}.

Put

∥[a_{j,l}]∥_{HS} = ( ∑_{j=1}^{m} ∑_{l=1}^{n} |a_{j,l}|^{2})^{1/2} (7.1.8)

for every [a_{j,l}] ∈ M_{m,n}(R) or M_{m,n}(C). This is known as the Hilbert–Schmidt norm on M_{m,n}(R) and M_{m,n}(C). More precisely, (7.1.8) corresponds to the standard Euclidean norm on R^{m n} and C^{m n}, using the identifications mentioned earlier. In particular, the fact that (7.1.8) defines a norm on M_{m,n}(R) and M_{m,n}(C) follows from the analogous statements for the standard Euclidean norms on R^{m n} and C^{m n}. More precisely, the triangle inequality for (7.1.8) follows from the triangle inequality for the Euclidean norm, and the first two requirements of a norm can be verified directly.

7.2 Matrices and linear mappings

Let m and n be positive integers again. As usual, a mapping A from R^{n} into R^{m} is said to be linear if

A(v + w) = A(v) + A(w) (7.2.1)

for every v, w ∈ R^{n}, and A(t v) = t A(v) (7.2.2)

for every v ∈ R^{n} and t ∈ R. Similarly, a mapping A from C^{n} into C^{m} is said to be (complex) linear if (7.2.1) holds for every v, w ∈ C^{n}, and (7.2.2) holds for every v ∈ C^{n} and t ∈ C.

Let [a_{j,l}] be an m × n matrix whose entries are real or complex numbers. If v = (v_{1}, …, v_{n}) ∈ R^{n} or C^{n}, as appropriate, then put

(A(v))_{j} = ∑_{l=1}^{n} a_{j,l} v_{l} (7.2.3)

for each j = 1, …, m. This defines A(v) as an element of R^{m} or C^{m}, as appropriate, where the jth coordinate of A(v) is given by (7.2.3) for each j = 1, …, m. It is easy to see that A is linear as a mapping from R^{n} into R^{m} or from C^{n} into C^{m}, as appropriate. Conversely, it is well known and not too difficult to show that every linear mapping A from R^{n} into R^{m} or from C^{n} into C^{m} corresponds to a unique m × n matrix with real or complex entries, as appropriate, in this way.

Let e_{k} = (e_{k,1}, …, e_{k,n}) be the kth standard basis vector in R^{n} or C^{n} for each k = 1, …, n. Thus e_{k,l} = 1 when k = l, and e_{k,l} = 0 when k ≠ l. If A corresponds to [a_{j,l}] ∈ M_{m,n}(R) or M_{m,n}(C) as in (7.2.3), then

(A(e_{k}))_{j} = a_{j,k} (7.2.4)

for every j = 1, …, m and k = 1, …, n. If A is any linear mapping from R^{n} into R^{m} or from C^{n} into C^{m}, then one can use (7.2.4) to define an m×n matrix [a_{j,l}] of real or complex numbers, as appropriate. This implies that (7.2.3) holds for every v ∈ R^{n} or C^{n}, as appropriate, because of linearity, and by expressing v as a linear combination of e_{1}, …, e_{n}.

Let [a_{j,l}] ∈ M_{m,n}(R) or M_{m,n}(C) be given again, and let A be the corresponding linear mapping from R^{n} into R^{m} or from C^{n} into C^{m}, as in (7.2.3). Using (7.2.4), we get that

∥A(e_{k})∥_{2} = ( ∑_{j=1}^{m} |a_{j,k}|^{2})^{1/2} (7.2.5)

for every k = 1, …, n. More precisely, the left side of (7.2.5) refers to the standard Euclidean norm of A(e_{k}) in R^{m} or C^{m}, as appropriate. It follows that the Hilbert–Schmidt norm of [a_{j,l}] defined in the previous section can be given by

∥[a_{j,l}]∥_{HS} = ( ∑_{k=1}^{n} ∥A(e_{k})∥_{2}^{2})^{1/2}. (7.2.6)

Let v = (v_{1}, …, v_{n}) ∈ R^{n} or C^{n} be given, as appropriate. Remember that

v = ∑_{k=1}^{n} v_{k} e_{k}, (7.2.7)

where the right side is a linear combination of e_{1}, …, e_{n} in R^{n} or C^{n}, as appropriate. Using the linearity of A, we get that

A(v) = ∑_{k=1}^{n} v_{k} A(e_{k}), (7.2.8)

where the right side is a linear combination of A(e_{1}), …, A(e_{k}) in R^{m} or C^{m}, as appropriate. It follows that

∥A(v)∥_{2} ≤ ∑_{k=1}^{n} |v_{k}| ∥A(e_{k})∥_{2}, (7.2.9)

where ∥ · ∥_{2} refers to the standard Euclidean norm on R^{m} or C^{m} again, as appropriate. This implies that

∥A(v)∥_{2} ≤ ( ∑_{k=1}^{n} |v_{k}|^{2})^{1/2} ∥[a_{j,l}]∥_{HS}, (7.2.10)

using the Cauchy–Schwarz inequality on the right side of (7.2.9).

7.3 Some related estimates

Let m and n be positive integers, and let A be a linear mapping from R^{n} into R^{m}, or from C^{n} into C^{m}. Also let N be a norm on R^{m} or C^{m}, as appropriate. If v ∈ R^{n} or C^{n}, as appropriate, then v can be expressed as a linear combination of the standard basis vectors e_{1}, …, e_{n} in R^{n} or C^{n} as in (7.2.7). Hence A(v) can be expressed as in (7.2.8), so that

N(A(v)) ≤ ∑_{k=1}^{n} |v_{k}| N(A(e_{k})). (7.3.1)

This corresponds to (7.2.9) when N is the standard Euclidean norm on R^{m} or C^{m}, as appropriate.

Let us now look at this in terms of various norms on R^{n} or C^{n}, as appropriate. Let us start with the standard Euclidean norm

∥v∥_{2} = ( ∑_{k=1}^{n} |v_{k}|^{2})^{1/2} (7.3.2)

of v. As before, we can apply the Cauchy–Schwarz inequality to the right side of (7.3.1) to get that

N(A(v)) ≤ ( ∑_{k=1}^{n} N(A(e_{k}))^{2})^{1/2} ∥v∥_{2}. (7.3.3)

This is the same as (7.2.10) when N is the standard Euclidean norm on R^{m} or C^{m}, as appropriate.

Consider the norm

∥v∥_{1} = ∑_{k=1}^{n} |v_{k}| (7.3.4)

of v discussed in Sections 1.3 and 1.4. It is easy to see that

N(A(v)) ≤ ( max_{1≤k≤n} N(A(e_{k}))) ∥v∥_{1}, (7.3.5)

using (7.3.1). Similarly, we can consider the norm

∥v∥_{∞} = max_{1≤k≤n} |v_{k}| (7.3.6)

of v discussed in Sections 1.3 and 1.4 as well. Observe that

N(A(v)) ≤ ( ∑_{k=1}^{n} N(A(e_{k}))) ∥v∥_{∞}, (7.3.7)

by (7.3.1) again.

Let N_{0} be any norm on R^{n} or C^{n}, as appropriate. One can check that there is a nonnegative real number C such that

N(A(v)) ≤ C N_{0}(v) (7.3.8)

for every v ∈ R^{n} or C^{n}, as appropriate. This follows from the previous remarks when N_{0}(v) is given by (7.3.2), (7.3.4), or (7.3.6). Otherwise, one can use the fact that (7.3.2) is bounded by a constant multiple of N_{0}(v), as in Section 6.7.

If v, w ∈ R^{n} or C^{n}, as appropriate, then

N(A(v) − A(w)) = N(A(v − w)) ≤ C N_{0}(v − w). (7.3.9)

This uses the linarity of A in the first step, and (7.3.8) in the second step. Thus

d_{N}(A(v), A(w)) ≤ C d_{N_{0}}(v, w), (7.3.10)

where d_{N_{0}} is the metric associated to N_{0} on R^{n} or C^{n}, and d_{N} is the metric associated to N on R^{m} or C^{m}, as appropriate.

7.4 Spaces of linear mappings

Let m and n be positive integers again. The space of linear mappings from R^{n} into R^{m} may be denoted L(R^{n}, R^{m}). Similarly, the space of (complex) linear mappings from C^{n} into C^{m} may be denoted L(C^{n}, C^{m}).

Let A, B be linear mappings from R^{n} into R^{m} or from C^{n} into C^{m}, and let t be a real or complex number, as appropriate. Thus t A can be defined as a mapping from R^{n} into R^{m}, or from C^{n} into C^{m}, as appropriate, by putting

(t A)(v) = t A(v) (7.4.1)

for every v ∈ R^{n} or C^{n}, as appropriate. Similarly, A + B can be defined as a mapping from R^{n} into R^{m}, or from C^{n} into C^{m}, as appropriate, by putting

(A + B)(v) = A(v) + B(v) (7.4.2)

for every v ∈ R^{n} or C^{n}, as appropriate. It is easy to see that these are also linear as mappings from R^{n} into R^{m}, or from C^{n} into C^{m}, as appropriate.

As in Section 7.2, there are standard one-to-one correspondences between M_{m,n}(R) and L(R^{n}, R^{m}), and between M_{m,n}(C) and L(C^{n}, C^{m}). These correspondences are linear, in the sense that sums of matrices are associated to sums of linear mappings, and similarly for multiplication of matrices and linear mappings by real or complex numbers, as appropriate.

A nonnegative real-valued function N on L(R^{n}, R^{m}) or L(C^{n}, C^{m}) is said to be a norm if it satisfies the usual three conditions, as follows. First, if A is a linear mapping from R^{n} into R^{m} or from C^{n} into C^{m}, as appropriate, then

N(A) = 0 (7.4.3)

if and only if A = 0, which means that A(v) = 0 for every v ∈ R^{n} or C^{n}, as appropriate. Second, if A is a linear mapping from R^{n} into R^{m} or from C^{n} into C^{m} and t ∈ R or C, as appropriate, then

N(t A) = |t| N(A). (7.4.4)

Third, if A and B are linear mappings from R^{n} into R^{m} or from C^{n} into C^{m}, as appropriate, then N(A + B) ≤ N(A) + N(B). (7.4.5)

Under these conditions,

d_{N}(A, B) = N(A − B) (7.4.6)

defines a metric on L(R^{n}, R^{m}) or L(C^{n}, C^{m}), as appropriate. Observe that norms on L(R^{n}, R^{m}) and L(C^{n}, C^{m}) correspond exactly to norms on M_{m,n}(R) and M_{m,n}(C), respectively, using the correspondence between matrices and linear mappings described in Section 7.2. The metrics associated to these norms correspond to each other in the same way.

Let A be a linear mapping from R^{n} into R^{m} or from C^{n} into C^{m} again. Also let e_{1}, …, e_{n} be the standard basis vectors in R^{n} or C^{n}, as in Section 7.2, and let ∥ · ∥_{2} be the standard Euclidean norm on R^{m} or C^{m}, as appropriate. Put

∥A∥_{HS} = ( ∑_{k=1}^{n} ∥A(e_{k})∥_{2}^{2})^{1/2}, (7.4.7)

which corresponds exactly to the Hilbert–Schmidt norm of the matrix associated to A, as in Section 7.2. This defines a norm on each of L(R^{n}, R^{m}) and L(C^{n}, C^{m}), which may be called the Hilbert–Schmidt norm as well.

7.5 Operator norms

Let m and n be positive integers, and let A be a linear mapping from R^{n} into R^{m}, or from C^{n} into C^{m}. Also let N_{0} be a norm on R^{n} or C^{n}, and let N be a norm on R^{m} or C^{m}, as appropriate. The corresponding operator norm

∥A∥_{op} (7.5.1)

of A is defined to be the supremum of

\frac{N(A(v))}{N_{0}(v)} (7.5.2)

over all v ∈ R^{n} or C^{n}, as appropriate, with v ≠ 0. Note that this ratio is bounded, by (7.3.8), so that the supremum is finite. More precisely, (7.3.8) is equivalent to saying that the nonnegative real number C is an upper bound for (7.5.2) when v ≠ 0, because (7.3.8) holds automatically when v = 0. Thus (7.3.8) holds exactly when ∥A∥_{op} ≤ C. (7.5.3)

In particular, (7.3.8) holds with C = ∥A∥_{op}. Alternatively, ∥A∥_{op} is the infimum of the nonnegative real numbers C for which (7.3.8) holds.

Note that

\frac{N(A(v))}{N_{0}(v)} = N((1/N_{0}(v)) A(v)) = N(A((1/N_{0}(v)) v)) (7.5.4)

for every v ∈ R^{n} or C^{n}, as appropriate, with v ≠ 0. Similarly,

N_{0}((1/N_{0}(v)) v) = N_{0}(v)/N_{0}(v) = 1 (7.5.5)

for every v ∈ R^{n} or C^{n}, as appropriate, with v ≠ 0. Using this, one can verify that ∥A∥_{op} is the same as the supremum of

N(A(v)) (7.5.6)

over all v ∈ R^{n} or C^{n}, as appropriate, such that N_{0}(v) = 1. This is also the same as the supremum of (7.5.6) over all v ∈ R^{n} or C^{n}, as appropriate, with N_{0}(v) ≤ 1. Of course, (7.5.6) is equal to 0 when v = 0, and otherwise

N(A(v)) = N(A((1/N_{0}(v)) v)) N_{0}(v), (7.5.7)

as in (7.5.4).

By construction, ∥A∥_{op} is a nonnegative real number, and ∥A∥_{op} = 0 if and only if A(v) = 0 for every v ∈ R^{n} or C^{n}, as appropriate. If t ∈ R or C, as appropriate, then t A also defines a linear mapping from R^{n} into R^{m} or from C^{n} into C^{m}, as appropriate. It is easy to see that

∥t A∥_{op} = |t| ∥A∥_{op}, (7.5.8)

because N(t A(v)) = |t| N(A(v)) (7.5.9)

for every v ∈ R^{n} or C^{n}, as appropriate. Let B be another linear mapping from R^{n} into R^{m} or from C^{n} into C^{m}, as appropriate, so that A + B defines another such linear mapping. Observe that

N((A + B)(v)) = N(A(v) + B(v)) ≤ N(A(v)) + N(B(v))
(7.5.10)  ≤ ∥A∥_{op} N_{0}(v) + ∥B∥_{op} N_{0}(v)
= (∥A∥_{op} + ∥B∥_{op}) N_{0}(v)

for every v ∈ R^{n} or C^{n}, as appropriate. This implies that

∥A + B∥_{op} ≤ ∥A∥_{op} + ∥B∥_{op}. (7.5.11)

Thus the operator norm satisfies the requirements of a norm on the space of linear mappings from R^{n} into R^{m} or from C^{n} into C^{m}, as appropriate.

Suppose that N_{0} is the standard Euclidean norm on R^{n} or C^{n}, and that N is the standard Euclidean norm on R^{m} or C^{m}, as appropriate. In this case, we get that ∥A∥_{op} ≤ ∥A∥_{HS}, (7.5.12)

as in (7.2.10), where the right side is the Hilbert–Schmidt norm of A, as in the previous section. One can check that

∥A∥_{HS} ≤ n^{1/2} ∥A∥_{op}, (7.5.13)

directly from the definitions of these two norms.

7.6 Determinants and volume

Let n be a positive integer, and let [a_{j,l}] be an n × n matrix of real or complex numbers. The determinant det[a_{j,l}] (7.6.1)

can be defined as a real or complex number, as appropriate, in a standard way. If A is the linear mapping from R^{n} or C^{n} into itself, as appropriate, associated to [a_{j,l}], then the determinant of A is defined by

det A = det[a_{j,l}]. (7.6.2)

The determinant of the identity mapping I on R^{n} or C^{n} is equal to 1. Similarly, if t is a real or complex number, as appropriate, then

det(t I) = t^{n}. (7.6.3)

Let A be a linear mapping from R^{n} into itself, and let E be a subset of R^{n}. It is well known that

(7.6.4) Vol_{n}(A(E)) = | det A| Vol_{n}(E),

where Vol_{n}(·) is the usual n-dimensional volume of a subset of R^{n}. This is discussed in many textbooks, and one may also be interested in the articles [31, 84].

More precisely, E should be sufficiently nice for the volume to be defined, depending on the definition of the volume being used. The right side of (7.6.4) should be interpreted as being equal to 0 when det A = 0, even if Vol_{n}(E) may be +∞. Note that | det A| is uniquely determined by (7.6.4) when Vol_{n}(E) is positive and finite.

Let N be a norm on R^{n}, and let ∥A∥_{op} be the corresponding operator norm of A, as in the previous section. More precisely, this uses N as the norm on R^{n} as both the domain and the range of A. It is well known that

| det A| ≤ ∥A∥_{op}^{n} (7.6.5) under these conditions.

Suppose for the moment that N is the standard Euclidean norm on R^{n}. If A can be diagonalized using an orthonormal basis for R^{n} with respect to the standard inner product, then (7.6.5) can be verified directly. Otherwise, one can reduce to that case, by considering the composition of A with its adjoint with respect to the standard inner product on R^{n}.

If N is any norm on R^{n}, then let \overline{B}_{N} be the closed unit ball in R^{n} with respect to N, as in Section A.6. It is easy to see that

A(\overline{B}_{N}) ⊆ ∥A∥_{op} \overline{B}_{N}, (7.6.6)

by the definition of the operator norm. Note that ∥A∥_{op} \overline{B}_{N} is the same as the closed ball in R^{n} centered at 0 with radius ∥A∥_{op} with respect to the metric d_{N}(v, w) = N(v − w) associated to N. Using (7.6.6), we get that

(7.6.7) Vol_{n}(A(\overline{B}_{N})) ≤ Vol_{n}(∥A∥_{op} \overline{B}_{N}) = ∥A∥_{op}^{n} Vol_{n}(\overline{B}_{N}).

One can use this to get (7.6.5) from (7.6.4), with E = \overline{B}_{N}.

Let A be a linear mapping from R^{n} or C^{n} into itself again. If v ∈ R^{n} or C^{n} satisfies A(v) = λ v (7.6.8)

for some λ ∈ R or C, as appropriate, then v is said to be an eigenvector of A with eigenvalue λ. Let N be a norm on R^{n} or C^{n}, as appropriate, and let ∥A∥_{op} be the corresponding operator norm of A, using N on both the domain and the range. Observe that

|λ| N(v) = N(λ v) = N(A(v)) ≤ ∥A∥_{op} N(v), (7.6.9)

by (7.6.8) and the definition of the operator norm. This implies that

|λ| ≤ ∥A∥_{op} (7.6.10)

when v ≠ 0.

In the complex case, it is well known that the determinant of A is equal to the product of its n eigenvalues, associated to nonzero eigenvectors, counted with their appropriate multiplicities. One can use this to get (7.6.5) from (7.6.10).

7.7 Lipschitz constants

Let (X, d_{X}) and (Y, d_{Y}) be (nonempty) metric spaces. Remember that a mapping f from X into Y is said to be Lipschitz if

d_{Y} (f(x), f(w)) ≤ C d_{X}(x, w) (7.7.1)

for some nonnegative real number C and every x, w ∈ X. In this case, we may also say that f is Lipschitz with constant C, to make the role of C more explicit. Let Lip(X, Y) be the space of all Lipschitz mappings from X into Y.

Let f be a mapping from X into Y again, and note that (7.7.1) holds automatically when x = w. If x, w are distinct elements of X, then (7.7.1) is the same as saying that

\frac{d_{Y} (f(x), f(w))}{d_{X}(x, w)} ≤ C. (7.7.2)

If f is Lipschitz, and X has at least two elements, then put

Lip(f) = sup \{\frac{d_{Y} (f(x), f(w))}{d_{X}(x, w)} : x, w ∈ X, x ≠ w\}. (7.7.3)

Otherwise, if X has only one element, then we take Lip(f) = 0. Thus (7.7.1) holds for some C ≥ 0 and every x, w ∈ X if and only if f is Lipschitz and

Lip(f) ≤ C. (7.7.4)

In particular, if f is Lipschitz, then (7.7.1) holds with C = Lip(f). Equivalently, Lip(f) is the infimum of the nonnegative real numbers C such that (7.7.1) holds for every x, w ∈ X.

Let m and n be positive integers, and suppose for the moment that X = R^{n} and Y = R^{m}, or that X = C^{n} and Y = C^{m}. Also let N_{0} be a norm on R^{n} or C^{n}, and let N be a norm on R^{m} or C^{m}, as appropriate. Thus N_{0} determines a metric d_{N_{0}} on R^{n} or C^{n}, and N determines a metric d_{N} on R^{m} or C^{m}, as appropriate. Let A be a linear mapping from R^{n} into R^{m}, or from C^{n} into C^{m}, as appropriate. Under these conditions, A is Lipschitz with respect to d_{N_{0}} and d_{N}, as in (7.3.10). More precisely,

Lip(A) = ∥A∥_{op}, (7.7.5)

where Lip(A) is defined using the metrics d_{N_{0}} and d_{N} as in the preceding paragraph, and the operator norm ∥A∥_{op} of A is defined using N_{0} and N as in Section 7.5. This is easy to verify, directly from the definitions.

Let (X, d_{X}) be any nonempty metric space again, and let m be a positive integer. Let us take Y = R^{m} or C^{m}, equipped with a norm N, which determines a metric d_{N} in the usual way. In this situation, if f is a mapping from X into Y, then (7.7.1) is the same as saying that

N(f(x) − f(w)) = d_{N}(f(x), f(w)) ≤ C d_{X}(x, w). (7.7.6)

Let t be a real or complex number, as appropriate, so that t f also defines a mapping from X into Y, and

(7.7.7) d_{N}(t f(x), t f(w)) = N(t f(x) − t f(w)) = N(t (f(x) − f(y)))
= |t| N(f(x) − f(w)) = |t| d_{N}(f(x), f(w))

for every x, w ∈ X. If f is Lipschitz, then it is easy to see that t f is Lipschitz too, with Lip(t f) = |t| Lip(f). (7.7.8)

Similarly, let g be another mapping from X into Y, so that f + g defines a mapping from X into Y as well. Observe that

d_{N}((f + g)(x), (f + g)(w)) = N((f(x) + g(x)) − (f(w) + g(w)))
(7.7.9)  = N((f(x) − f(w)) + (g(x) − g(w)))
≤ N(f(x) − f(w)) + N(g(x) − g(w))

for every x, w ∈ X. If f and g are both Lipschitz, then it follows that

d_{N}((f + g)(x), (f + g)(w)) ≤ Lip(f) d_{X}(x, w) + Lip(g) d_{X}(x, w)
(7.7.10)  = (Lip(f) + Lip(g)) d_{X}(x, w)

for every x, w ∈ X. This implies that f + g is Lipschitz, with

(7.7.11) Lip(f + g) ≤ Lip(f) + Lip(g).

In particular, the space of Lipschitz mappings from X into R^{m} or C^{m} may be considered as a vector space over the real or complex numbers, as appropriate, with respect to pointwise addition and scalar multiplication. In the terminology of Section A.6, Lip(f) defines a seminorm on this vector space. More precisely, Lip(f) = 0 if and only if f is a constant mapping on X.

7.8 Compositions and isometries

Let (X, d_{X}), (Y, d_{Y}), and (Z, d_{Z}) be (nonempty) metric spaces. If f is a mapping from X into Y, and g is a mapping from Y into Z, then the composition g ∘ f can be defined as a mapping from X into Z, as usual. Suppose that f and g are both Lipschitz, so that

(7.8.1)  d_{Z}((g ∘ f)(x), (g ∘ f)(w)) = d_{Z}(g(f(x)), g(f(w)))
≤ Lip(g) d_{Y} (f(x), f(w))
≤ Lip(f) Lip(g) d_{X}(x, w)

for every x, w ∈ X. This implies that g ∘ f is Lipschitz as well, with

(7.8.2) Lip(g ∘ f) ≤ Lip(f) Lip(g).

More precisely, Lip(f), Lip(g), and Lip(g ∘ f) are as defined in the previous section, using the appropriate metric spaces in the domains and ranges of these meppings.

A mapping f from X into Y is said to be an isometry if

d_{Y} (f(x), f(w)) = d_{X}(x, w) (7.8.3)

for every x, w ∈ X. In particular, this implies that f is Lipschitz, with constant C = 1. If a mapping g from Y into Z is an isometry too, then it is easy to see that the composition g ∘ f of f and g is an isometric mapping from X into Z.

Note that an isometric mapping f from X into Y is automatically injective. If f maps X onto Y, then the corresponding inverse mapping f^{−1} is an isometry from Y onto X.

Let n, m, and k be positive integers. Suppose that either A is a linear mapping from R^{n} into R^{m} and B is a linear mapping from R^{m} into R^{k}, or that A is a linear mapping from C^{n} into C^{m} and B is a linear mapping from C^{m} into C^{k}. Thus the composition B ∘ A is either defined as a mapping from R^{n} into R^{k}, or as a mapping from C^{n} into C^{k}. In both cases, B ∘ A is a linear mapping. Of course, the matrix associated to B ∘ A can be given in terms of the matrices associated to A and B using matrix multiplication.

Let N_{1}, N_{2}, and N_{3} be norms on R^{n}, R^{m}, and R^{k}, respectively, or on C^{n}, C^{m}, and C^{k}, respectively, as appropriate. Using these norms, the operator norms ∥A∥_{op}, ∥B∥_{op}, and ∥B ∘ A∥_{op} can be defined as in Section 7.5. If v ∈ R^{n} or C^{n}, as appropriate, then

(7.8.4)  N_{3}((B ∘ A)(v)) = N_{3}(B(A(v))) ≤ ∥B∥_{op} N_{2}(A(v))
≤ ∥A∥_{op} ∥B∥_{op} N_{1}(v).

It follows that ∥B ∘ A∥_{op} ≤ ∥A∥_{op} ∥B∥_{op}. (7.8.5)

This could also be obtained from (7.8.2), using (7.7.5).

In this situation, A is said to be an isometric linear mapping with respect to N_{1} and N_{2} if N_{2}(A(v)) = N_{1}(v) (7.8.6)

for every v ∈ R^{n} or C^{n}, as appropriate. This implies that

d_{N_{2}}(A(v), A(w)) = N_{2}(A(v) − A(w)) = N_{2}(A(v − w))
(7.8.7)  = N_{1}(v − w) = d_{N_{1}}(v, w)

for every v, w ∈ R^{n} or C^{n}, as appropriate, where d_{N_{1}} and d_{N_{2}} are the metrics associated to N_{1} and N_{2}, respectively. Conversely, (7.8.6) follows from (7.8.7) by taking w = 0. Thus A is isometric as a linear mapping with respect to N_{1} and N_{2} if and only if A is isometric with respect to the corresponding metrics d_{N_{1}} and d_{N_{2}}. If B is also an isometric linear mapping with respect to N_{2} and N_{3}, then their composition B ∘ A is an isometric linear mapping with respect to N_{1} and N_{3}.

7.9 Bilipschitz embeddings

Let (X, d_{X}) and (Y, d_{Y}) be (nonempty) metric spaces, and let f be a mapping from X into Y. Also let c be a positive real number, and consider the following condition: for every x, w ∈ X, we have that

c d_{X}(x, w) ≤ d_{Y} (f(x), f(w)). (7.9.1)

In particular, this condition implies that f is injective. If f is a one-to-one mapping from X onto Y, then (7.9.1) is the same as saying that the corresponding inverse mapping f^{−1} is Lipschitz with constant 1/c as a mapping from Y into X. Otherwise, if f is injective but not necessarily surjective, then one can consider the inverse mapping f^{−1} as a mapping from the image f(X) of X under f into X, and use the restriction of d_{Y} to f(X).

A mapping f from X into Y is said to be bilipschitz if f is Lipschitz and (7.9.1) holds for some c > 0. To be more precise, one may say that f is bilipschitz with constant C ≥ 1 if f is Lipschitz with constant C and (7.9.1) holds with c = 1/C. Using this terminology, an isometric mapping from X into Y is the same as a bilipschitz mapping with constant C = 1. A one-to-one mapping f from X onto Y is bilipschitz if and only if f is Lipschitz and the inverse mapping f^{−1} is Lipschitz as a mapping from Y into X. If f is not surjective, then one can consider the inverse mapping f^{−1} as a mapping from f(X) into X, using the restriction of d_{Y} to f(X), as before.

Let (Z, d_{Z}) be another metric space, and let g be a mapping from Y into Z. Suppose that there is a positive real number c′ such that

c′ d_{Y} (y, u) ≤ d_{Z}(g(y), g(u)) (7.9.2)

for every u, y ∈ Y. If f : X → Y satisfies (7.9.1) for some c > 0, then the composition g ∘ f satisfies an analogous condition as a mapping from X into Z. More precisely, for each x, w ∈ X, we have that

(7.9.3) c c′ d_{X}(x, w) ≤ c′ d_{Y} (f(x), f(w)) ≤ d_{Z}(g(f(x)), g(f(w)))
= d_{Z}((g ∘ f)(x), (g ∘ f)(w)).

In particular, if f and g are both bilipschitz, then g∘f is bilipschitz as a mapping from X into Z.

Let m and n be positive integers, and let A be a linear mapping from R^{n} into R^{m} or from C^{n} into C^{m}. Also let N_{0} and N be norms on R^{n} and R^{m}, respectively, or on C^{n} and C^{m}, respectively, as appropriate. Suppose that

c N_{0}(v) ≤ N(A(v)) (7.9.4)

for some positive real number c and every v ∈ R^{n} or C^{n}, as appropriate. If v, w ∈ R^{n} or C^{n}, as appropriate, then we get that

c d_{N_{0}}(v, w) = c N_{0}(v − w) ≤ N(A(v − w))
(7.9.5)  = N(A(v) − A(w)) = d_{N}(A(v), A(w)),

where d_{N_{0}} and d_{N} are the metrics associated to N_{0} and N, respectively. Of course, (7.9.5) corresponds to (7.9.1) in this situation. Note that (7.9.5) implies (7.9.4), by taking w = 0. Observe too that (7.9.4) implies directly that v = 0 when A(v) = 0, which is to say that the kernel of A is trivial.

Let k be another positive integer, and let B be a linear mapping from R^{m} into R^{k} or from C^{m} into C^{k}, as appropriate. Also let N_{3} be a norm on R^{k} or C^{k}, as appropriate. Suppose that

c′ N(y) ≤ N_{3}(B(y)) (7.9.6)

for some positive real number c′ and every y ∈ R^{m} or C^{m}, as appropriate. If v ∈ R^{n} or C^{n}, as appropriate, then we get that

c c′N_{0}(v) ≤ c′ N(A(v)) ≤ N_{3}(B(A(v))) = N_{3}((B ∘ A)(v)). (7.9.7)

This basically corresponds to (7.9.3) in this situation, as before.

7.10 Linear mappings and seminorms

Let m and n be positive integers, and let A be a linear mapping from R^{n} into R^{m} or from C^{n} into C^{m}. Also let N be a seminorm on R^{m} or C^{m}, as appropriate, as in Section A.6. More precisely, the definition of a seminorm was mentioned previously in the real case, and the complex case is analogous. Under these conditions, one can check that N(A(v)) defines a seminorm on R^{n} or C^{n}, as appropriate. Indeed, it is easy to see that N(A(v)) satisfies the homogeneity requirement for a seminorm on R^{n} or C^{n}, as appropriate, because of the analogous property for N and the linearity of A. Similarly, N(A(v)) satisfies the triangle inequality on R^{n} or C^{n}, as appropriate, because of the triangle inequality for N and the linearity of A. Suppose for the rest of the section that N is a norm on R^{m} or C^{m}, as appropriate, so that

N(A(v)) = 0 (7.10.1)

only when A(v) = 0.

If the kernel of A is trivial, then (7.10.1) holds only when v = 0. This means that N(A(v)) defines a norm on R^{n} or C^{n}, as appropriate. Let N_{0} be another norm on R^{n} or C^{n}, as appropriate. Under these conditions, one can get that (7.9.4) holds for some c > 0 using the remarks in Section 6.7.

Suppose now that m = n, so that A is a linear mapping from R^{n} into itself, or from C^{n} into itself. If the kernel of A is trivial, then it is well known that A maps R^{n} or C^{n} onto itself, as appropriate. This means that A has an inverse mapping A^{−1} on R^{n} or C^{n}, as appropriate. Of course, the inverse mapping is also linear.

In this case, (7.9.4) can be reformulated as saying that

N_{0}(A^{−1}(u)) ≤ (1/c) N(u) (7.10.2)

for every u ∈ R^{n} or C^{n}, as appropriate. We have seen previously that this type of condition holds, because A^{−1} is a linear mapping on R^{n} or C^{n}, as appropriate. More precisely, this condition holds with

1/c = ∥A^{−1}∥_{op}, (7.10.3) where the operator norm of A^{−1} is defined using N on the domain of A^{−1}, and N_{0} on the range of A^{−1}. Note that ∥A^{−1}∥_{op} > 0, because A^{−1} ≠ 0. Of course, if (7.10.2) holds for some c > 0, then we have that

∥A^{−1}∥_{op} ≤ 1/c. (7.10.4)

7.11 Small perturbations

Let (X, d_{X}) be a nonempty metric space, let m be a positive integer, and let f be a mapping from X into R^{m} or C^{m}. Also let N be a norm on R^{m} or C^{m}, as appropriate, which leads to a metric d_{N} in the usual way. Suppose that

c d_{X}(x, w) ≤ d_{N}(f(x), f(w)) = N(f(x) − f(w)) (7.11.1)

for some positive real number c and all x, w ∈ X. Let g be another mapping from X into R^{m} or C^{m}, as appropriate. We would like to show that g satisfies an analogous condition when g is sufficiently close to f, in a suitable sense. More precisely, we ask first that f − g be Lipschitz with respect to d_{N} on the range. This means that

N((f(x) − g(x)) − (f(w) − g(w))) = d_{N}(f(x) − g(x), f(w) − g(w))
(7.11.2)  ≤ Lip(f − g) d_{X}(x, w)

for every x, w ∈ X, where Lip(f − g) is as defined in Section 7.7. Of course,

(7.11.3) N(f(x) − f(w)) ≤ N((f(x) − g(x)) − (f(w) − g(w)))
+N(g(x) − g(w))

for every x, w ∈ X, by the triangle inequality. Combining this with (7.11.1) and (7.11.2), we get that

c d_{X}(x, w) ≤ N(g(x) − g(w)) + Lip(f − g) d_{X}(x, w) (7.11.4)

for every x, w ∈ X. It follows that

(c − Lip(f − g)) d_{X}(x, w) ≤ N(g(x) − g(w)) = d_{N}(g(x), g(w)) (7.11.5)

for every x, w ∈ X. If Lip(f − g) < c, then this is the same type of condition as before.

Let n be a positive integer, and let us now take X = R^{n} or C^{n}, as appropriate. Let N_{0} be a norm on R^{n} or C^{n}, as appropriate, which leads to a metric d_{N_{0}}, as usual. Also let A be a linear mapping from R^{n} into R^{m} or from C^{n} into C^{m}, as appropriate. Suppose that

c N_{0}(v) ≤ N(A(v)) (7.11.6)

for some c > 0 and all v ∈ R^{n} or C^{n}, as appropriate. Let B be another linear mapping from R^{n} into R^{m} or from C^{n} into C^{m}, as appropriate. We would like to verify that B satisfies the same type of condition when B is sufficiently close to A. This could be obtained from the remarks in the preceding paragraph, but the analogous argument is a bit simpler in this case, as follows.

Remember that

N(A(v) − B(v)) ≤ ∥A − B∥_{op} N_{0}(v) (7.11.7)

for every v ∈ R^{n} or C^{n}, as appropriate, where the operator norm ∥A − B∥_{op} of A − B is as defined in Section 7.5. Thus

(7.11.8)  N(A(v)) ≤ N(A(v) − B(v)) + N(B(v))
≤ N(B(v)) + ∥A − B∥_{op} N_{0}(v)

for every v ∈ R^{n} or C^{n}, as appropriate, using the triangle inequality in the first step. This implies that

c N_{0}(v) ≤ N(B(v)) + ∥A − B∥_{op} N_{0}(v) (7.11.9)

for every v ∈ R^{n} or C^{n}, as appropriate, by (7.11.6). Hence

(c − ∥A − B∥_{op}) N_{0}(v) ≤ N(B(v)) (7.11.10)

for every v ∈ R^{n} or C^{n}, as appropriate. This is the same type of condition as before when ∥A − B∥_{op} < c.

In particular, if ∥A − B∥_{op} < c, then (7.11.10) implies that the kernel of B is trivial. If m = n, then it follows that B is invertible.

7.12 The contraction mapping theorem

Let (X, d(x, y)) be a nonempty metric space, and let f be a mapping from X into itself. Suppose that f is Lipschitz with constant c ≥ 0, so that

d(f(x), f(y)) ≤ c d(x, y) (7.12.1)

for every x, y ∈ X. If c < 1, and if X is complete with respect to d(·, ·), then a famous theorem states that there is a unique point x ∈ X such that f(x) = x. More precisely, uniqueness can be verified directly, without using completeness.

To get the existence of the fixed point, let x_{0} be any element of X, and let x_{1}, x_{2}, x_{3}, … be the sequence of elements of X defined recursively by

x_{j} = f(x_{j−1}) (7.12.2)

when j ≥ 1. Observe that

d(x_{j}, x_{j+1}) = d(f(x_{j−1}), f(x_{j})) ≤ c d(x_{j−1}, x_{j}) (7.12.3)

for every j ≥ 1. This implies that

d(x_{j}, x_{j+1}) ≤ c^{j} d(x_{0}, x_{1}) (7.12.4) for every j ≥ 1. If l < n are positive integers, then it follows that

d(x_{l}, x_{n}) ≤ ∑_{j=l}^{n−1} d(x_{j}, x_{j+1}) ≤ ∑_{j=l}^{n−1} c^{j} d(x_{0}, x_{1})

(7.12.5)  = c^{l} ∑_{j=0}^{n−l−1} c^{j} d(x_{0}, x_{1}) ≤ \frac{c^{l}}{1 − c} d(x_{0}, x_{1}).

This implies that {x_{j}}_{j=0}^{∞} is a Cauchy sequence in X, because c^{l} → 0 as l → ∞ when c < 1.

If X is complete, then it follows that {x_{j}}_{j=0}^{∞} converges to an element x of X. We also have that {f(x_{j})}_{j=0}^{∞} converges to f(x) in X, because f is continuous. By construction, {f(x_{j})}_{j=0}^{∞} is the same as {x_{j+1}}_{j=0}^{∞}, which converges to x. Hence f(x) = x, as desired.

Let n be a positive integer, and let \overline{B}(0, 1) be the closed unit ball in R^{n} with respect to the standard Euclidean metric. Also let f be a continuous mapping from \overline{B}(0, 1) into itself, with respect to the restriction of the standard Euclidean metric on R^{n} to \overline{B}(0, 1). Under these conditions, Brouwer’s fixed-point theorem states that f has a fixed point, which is to say that there is an x ∈ \overline{B}(0, 1) such that f(x) = x. If n = 1, then this can be obtained from the intermediate value theorem. These and related matters are discussed in many textbooks, as well as the articles [22, 23, 26, 51, 110, 115, 130, 132, 135, 138, 150, 158, 164], for instance.

Let n be a positive integer again, and let N be a norm on R^{n}. Note that R^{n} is complete with respect to the metric d_{N} associated to N. This can be obtained from the completeness of R^{n} with respect to the standard Euclidean metric, and the comparability of N with the standard Euclidean norm on R^{n}, as in Section 6.7.

Let g be a Lipschitz mapping from R^{n} into itself, with respect to d_{N}, and with constant c ≥ 0. This means that

N(g(x) − g(y)) ≤ c N(x − y) (7.12.6)

for every x, y ∈ R^{n} in this situation. Let a ∈ R^{n} be given, and put

g_{a}(x) = g(x) + a (7.12.7)

for every x ∈ R^{n}. Observe that

N(g_{a}(x) − g_{a}(y)) = N(g(x) − g(y)) ≤ c N(x − y) (7.12.8)

for every x, y ∈ R^{n}. Thus g_{a} is also Lipschitz with constant c as a mapping from R^{n} into itself, with respect to d_{N}.

Suppose that c < 1, so that the contraction mapping theorem can be applied to g_{a} on R^{n}. It follows that there is a unique point x(a) ∈ R^{n} such that

g(x(a)) + a = g_{a}(x(a)) = x(a). (7.12.9)

Put h(x) = x − g(x) (7.12.10)

for every x ∈ R^{n}, which defines a mapping from R^{n} into itself. The previous statement can be reformulated as saying that for every a ∈ R^{n} there is a unique x(a) ∈ R^{n} such that h(x(a)) = a. (7.12.11)

Of course, this is the same as saying that h is a one-to-one mapping from R^{n} onto itself. Note that h is Lipschitz with constant 1 + c on R^{n} with respect to d_{N}, as in Section 7.7. We also have that

(1 − c) N(x − w) ≤ N(h(x) − h(w)) (7.12.12)

for every x, y ∈ R^{n}, as in the previous section.

7.13 A localized condition

Let n be a positive integer again, and let N be a norm on R^{n}. Also let

\overline{B}_{N}(r) = \{x ∈ R^{n} : N(x) ≤ r\} (7.13.1)

be the closed ball in R^{n} centered at 0 with radius r ≥ 0 with respect to the metric d_{N} on R^{n} associated to N. Remember that this is a closed set in R^{n} with respect to d_{N} for each r ≥ 0. It is easy to see that \overline{B}_{N}(r) is complete as a metric space with respect to the restriction of d_{N} to \overline{B}_{N}(r) for every r ≥ 0, as in Section 1.7. More precisely, any Cauchy sequence of elements of \overline{B}_{N}(r) with respect to d_{N} may be considered as a Cauchy sequence in R^{n} with respect to d_{N} as well. Such a sequence converges to an element of R^{n} with respect to d_{N}, because R^{n} is complete with respect to d_{N}, as in the previous section. The limit of this sequence is contained in \overline{B}_{N}(r), because \overline{B}_{N}(r) is a closed set in R^{n} with respect to d_{N}.

Let r be a positive real number, and let g be a Lipschitz mapping from \overline{B}_{N}(r) into R^{n} with constant c ≥ 0, with respect to d_{N} and its restriction to \overline{B}_{N}(r). Thus N(g(x) − g(y)) ≤ c N(x − y) (7.13.2)

for every x, y ∈ \overline{B}_{N}(r). Suppose also that

g(0) = 0. (7.13.3)

This implies that N(g(x)) ≤ c N(x) ≤ c r (7.13.4)

for every x ∈ \overline{B}_{N}(r), using (7.13.2) in the first step.

Suppose too that c < 1, (7.13.5)

124 CHAPTER 7. MATRIX NORMS AND LIPSCHITZ CONDITIONS and let a ∈ R^{n} be given, with N(a) ≤ (1 − c) r. (7.13.6) Put g_{a}(x) = g(x) + a (7.13.7) for each x ∈ \overline{B}_{N}(r), as before. Observe that N(g_{a}(x)) ≤ N(g(x)) + N(a) ≤ c r + (1 − c) r = r (7.13.8) for every x ∈ \overline{B}_{N}(r), using (7.13.4) and (7.13.6) in the second step.

This shows that g_{a} maps \overline{B}_{N}(r) into itself under these conditions. Note that g_{a} is Lipschitz with constant c with respect to d_{N} as well, as in (7.12.8). Hence we can apply the contraction mapping theorem to g_{a} on \overline{B}_{N}(r), because of (7.13.5). This implies that there is a unique point x(a) ∈ \overline{B}_{N}(r) such that g(x(a)) + a = g_{a}(x(a)) = x(a). (7.13.9)

Let h be the mapping from \overline{B}_{N}(r) into R^{n} defined by h(x) = x − g(x) (7.13.10) for every x ∈ \overline{B}_{N}(r). Observe that h(x(a)) = x(a) − g(x(a)) = a, (7.13.11) by (7.13.9). It follows that h(\overline{B}_{N}(r)) ⊇ \overline{B}_{N}((1 − c) r) (7.13.12) in this situation. Note that h is Lipschitz with constant 1 + c on \overline{B}_{N}(r) with respect to d_{N}, and (7.12.12) holds for all x, w ∈ \overline{B}_{N}(r), as in Sections 7.7 and 7.11.

7.14 Open mappings

Let (X, d_{X}) and (Y, d_{Y}) be metric spaces. A mapping f from X into Y is said to be an open mapping if for every open set U ⊆ X, (7.14.1) f(U) is an open set in Y. One can check that this happens if and only if for every x ∈ X and r > 0 there is a t > 0 such that B_{Y} (f(x), t) ⊆ f(B_{X}(x, r)). (7.14.2)

Let n be a positive integer, and let N be a norm on R^{n}. If v ∈ R^{n} and r > 0, then let B_{N}(v, r) = {w ∈ R^{n} : N(v − w) < r} (7.14.3) and \overline{B}_{N}(v, r) = \{w ∈ R^{n} : N(v − w) ≤ r\} (7.14.4)

be the open and closed balls in R^{n} centered at v with radius r with respect to the metric d_{N} associated to N.

Let v_{0} ∈ R^{n} and r_{0} > 0 be given, and let f be a mapping from \overline{B}_{N}(v_{0}, r_{0}) into R^{n}. Suppose that f(x) − x is Lipschitz with constant c ≥ 0 on \overline{B}_{N}(v_{0}, r_{0}), with respect to d_{N} and its restriction to \overline{B}_{N}(v_{0}, r_{0}). If c < 1, then

\overline{B}_{N}(f(v_{0}), (1 − c) r_{0}) ⊆ f(\overline{B}_{N}(v_{0}, r_{0})). (7.14.5)

This follows from the remarks in the previous section when v_{0} = f(v_{0}) = 0, and otherwise one can reduce to this case.

Similarly, let v ∈ R^{n} and r > 0 be given, and let f be a mapping from B_{N}(v, r) into R^{n}. Suppose that f(x)−x is Lipschitz with constant c, 0 ≤ c < 1, on B_{N}(v, r), with respect to d_{N} and its restriction to B_{N}(v, r). Let w ∈ B_{N}(v, r) be given, and let r_{1} be a positive real number such that

r_{1} ≤ r − N(v − w). (7.14.6)

Note that r − N(v − w) > 0, and that (7.14.6) implies that

B_{N}(w, r_{1}) ⊆ B_{N}(v, r). (7.14.7)

If 0 < r_{0} < r_{1}, then \overline{B}_{N}(w, r_{0}) ⊆ B_{N}(w, r_{1}), and

\overline{B}_{N}(f(w), (1 − c) r_{0}) ⊆ f(\overline{B}_{N}(w, r_{0})), (7.14.8)

as in (7.14.5). This implies that

B_{N}(f(w), (1 − c) r_{1}) ⊆ f(B_{N}(w, r_{1})). (7.14.9)

In particular, this means that f is an open mapping from B_{N}(v, r) into R^{n}, with respect to d_{N} and its restriction to B_{N}(v, r).

Some topics related to differentiability

8.1 An integral triangle inequality

Let m be a positive integer, and let N be a norm on R^{m} or C^{m}. If v_{1}, …, v_{l} are finitely many elements of R^{m} or C^{m} and t_{1}, …, t_{l} are real or complex numbers, as appropriate, then it is easy to see that

N( ∑_{k=1}^{l} t_{k} v_{k}) ≤ ∑_{k=1}^{l} |t_{k}| N(v_{k}). (8.1.1)

We would like to consider analogous statements for integrals instead of finite sums. Although this works for Riemann–Stieltjes integrals, it is sufficient to consider Riemann integrals for the result in the next section.

Let a and b be real numbers with a < b, and let f be a continuous function defined on the closed interval [a, b] with values in R^{m} or C^{m}, as appropriate. This implicitly uses the restriction of the standard Euclidean metric on the real line to [a, b], and one can also use the standard Euclidean metric on R^{m} or C^{m}, as appropriate. Equivalently, f can be given as

f(t) = (f_{1}(t), …, f_{m}(t)), (8.1.2)

where f_{1}, …, f_{m} are continuous real or complex-valued functions on [a, b], as appropriate. Of course, the continuity of a complex-valued function on [a, b] is equivalent to the continuity of its real and imaginary parts, as real-valued functions on [a, b].

Let α be a monotonically increasing real-valued function on [a, b]. As before, it suffices to consider the case where α(t) = t for every t ∈ [a, b] for the result discussed in the next section, which corresponds to using ordinary Riemann integrals on [a, b]. To define the Riemann–Stieltjes integral

∫_{a}^{b} f(t) dα(t) (8.1.3)

as an element of R^{m} or C^{m}, as appropriate, one can take the jth coordinate of (8.1.3) to be the Riemann–Stieltjes integral of f_{j} with respect to α on [a, b] for each j = 1, …, m. Similarly, the Riemann–Stieltjes integral of a continuous complex-valued function on [a, b] can be reduced to the real case, by considering the real and imaginary parts of the function.

Remember that N is (uniformly) continuous as a real-valued function on R^{m} or C^{m}, as appropriate, as in Section 6.7. Thus N(f(t)) defines a nonnegative real-valued continuous function on [a, b]. In particular, its Riemann–Stieltjes integral over [a, b] with respect to α can be defined as a nonnegative real number in the usual way.

Under these conditions, one can check that

N( ∫_{a}^{b} f(t) dα(t)) ≤ ∫_{a}^{b} N(f(t)) dα(t). (8.1.4)

More precisely, these integrals can be approximated by Riemann–Stieltjes sums associated to sufficiently fine partitions of [a, b]. To get (8.1.4), one can use the analogous inequalities for Riemann–Stieltjes sums, which follow from (8.1.1). One can also consider Riemann–Stieltjes integrability conditions on [a, b], instead of continuity.

8.2 A basic Lipschitz estimate

Let a and b be real numbers with a < b, and let m be a positive integer. Also let f be a function defined on [a, b] with values in R^{m} or C^{m}. The derivative f′(t) of f at a point t ∈ (a, b) can be defined in the usual way, as the limit of difference quotients, when the limit exists. Similarly, if t = a or b, then one can consider the corresponding one-sided limit. The differentiability of f at any t ∈ [a, b] is equivalent to the differentiability of the jth component f_{j} of f at t for each j = 1, …, m, as a real or complex-valued function on [a, b]. Similarly, the differentiability of a complex-valued function on [a, b] is equivalent to the differentiability of its real and imaginary parts. If f is differentiable at t ∈ [a, b], then the jth coordinate of f′(t) is equal to the derivative of f_{j} at t for each j = 1, …, m, and their common value is denoted f′_{j}(t).

128 CHAPTER 8. SOME TOPICS RELATED TO DIFFERENTIABILITY

Let us suppose from now on in this section that f is continuously differentiable on [a, b]. This means that the derivative f′(t) exists for every t ∈ [a, b], and that f′(t) is continuous as a function on [a, b] with values in R^{m} or C^{m}, as appropriate. Here we use the restriction of the standard Euclidean metric on R to [a, b], and the standard Euclidean metric on R^{m} or C^{m}, as appropriate. Equivalently, f_{j} should be continuously differentiable as a real or complex-valued function on [a, b] for each j = 1, …, m. Of course, f′(t) is really a one-sided derivative when t = a or b. If x ∈ [a, b], then the fundamental theorem of calculus implies that

∫_{a}^{x} f′(t) dt = f(x) − f(a), (8.2.1)

where the Riemann integral on the left side can be defined as in the previous section. Of course, this can be reduced to the case of real-valued functions in the usual way. Similarly, if a ≤ w ≤ x ≤ b, then

∫_{w}^{x} f′(t) dt = f(x) − f(w). (8.2.2)

Let N be a norm on R^{m} or C^{m}, as appropriate. If a ≤ w ≤ x ≤ b, then

N(f(x) − f(w)) = N( ∫_{w}^{x} f′(t) dt) ≤ ∫_{w}^{x} N(f′(t)) dt. (8.2.3)

This uses (8.2.2) in the first step, and (8.1.4) in the second step, applied to f′ and α(t) = t. Note that N(f′(t)) is continuous as a real-valued function on [a, b], as in the previous section. In particular, N(f′(t)) is bounded on [a, b], because [a, b] is compact, and in fact the maximum of N(f′(t)) is attained on [a, b]. It follows from (8.2.3) that

N(f(x) − f(w)) ≤ |x − w| sup_{a≤t≤b} N(f′(t)) (8.2.4)

for every x, w ∈ [a, b]. Thus f is Lipschitz with respect to the restriction of the standard Euclidean metric on R to [a, b] and the metric d_{N} associated to N on R^{m} or C^{m}, as appropriate.

More precisely,

Lip(f) = sup_{a≤t≤b} N(f′(t)), (8.2.5)

where Lip(f) is as defined in Section 7.7. The fact that Lip(f) is less than or equal to the right side of (8.2.5) follows from (8.2.4). To get the opposite inequality, it suffices to verify that N(f′(t)) is less than or equal to Lip(f) for every t ∈ [a, b]. The difference quotients used to define f′(t) have norm less than or equal to Lip(f), by definition of Lip(f). This implies that N(f′(t)) ≤ Lip(f), by taking the limit of the difference quotient.

8.3 Some partial Lipschitz conditions

Let n be a positive integer, and let E be a subset of R^{n}. Also let (Y, d_{Y}) be a metric space, and let f be a mapping from E into Y. Suppose that l is a positive integer less than or equal to n, and that C_{l} is a nonnegative real number. Let us say that f is partially Lipschitz in the lth variable with constant C_{l} on E if

d_{Y} (f(x), f(x′)) ≤ C_{l} |x_{l} − x′_{l}| (8.3.1) for every x, x′ ∈ E such that x_{j} = x′_{j} when j ≠ l. This means that f(x) is Lipschitz as a function of x_{l} with constant C_{l}, when the other coordinates of x are fixed. More precisely, this uses the restriction of the standard Euclidean metric on R to the set of real numbers that occur as lth coordinates of elements of E, when the other coordinates are fixed. This is the same as saying that if L is any line in R^{n} that is parallel to the lth coordinate axis, then the restriction of f to E ∩ L is Lipschitz with constant C_{l}.

Let a_{1}, …, a_{n} and b_{1}, …, b_{n} be real numbers such that a_{k} < b_{k} for each k = 1, …, n. Remember that the corresponding set

(8.3.2) C = {x ∈ R^{n} : a_{k} ≤ x_{k} ≤ b_{k} for each k = 1, …, n}

is called a cell in R^{n}, as in [155]. This is the same as the Cartesian product of the closed intervals [a_{k}, b_{k}], k = 1, …, n, as mentioned in Section 6.11. Suppose now that f is a mapping from C into Y. Thus f(x) may be considered as a function of x_{l} on [a_{l}, b_{l}], when the other variables are fixed elements of [a_{k}, b_{k}], k ≠ l.

Let C_{l} be a nonnegative real number for each l = 1, …, n, and suppose that f is partially Lipschitz in the lth variable with constant C_{l} for each l = 1, …, n. Let x, w ∈ C be given, and observe that

d_{Y} (f(x), f(w)) ≤ ∑_{l=1}^{n} C_{l} |x_{l} − w_{l}|. (8.3.3)

To see this, one can go from x to w in n steps, only changing one coordinate in each step. It follows that

d_{Y} (f(x), f(w)) ≤ ( max_{1≤l≤n} C_{l}) ∥x − w∥_{1}, (8.3.4)

where ∥ · ∥_{1} is defined on R^{n} as in Section 1.3. Similarly,

d_{Y} (f(x), f(w)) ≤ ( ∑_{l=1}^{n} C_{l}^{2})^{1/2} ∥x − w∥_{2}, (8.3.5)

where ∥ · ∥_{2} is the standard Euclidean norm on R^{n}. This uses the Cauchy– Schwarz inequality, applied to the sum on the right side of (8.3.3). We also get that

d_{Y} (f(x), f(w)) ≤ ( ∑_{l=1}^{n} C_{l}) ∥x − w∥_{∞}, (8.3.6)

where ∥·∥_{∞} is as defined in Section 1.3. In particular, f is Lipschitz with respect to the restrictions of the metrics on R^{n} associated to these norms to C.

Let f be a mapping from a set E ⊆ R^{n} into Y again, and suppose that f is partially Lipschitz in the lth variable on E with constant C_{l} ≥ 0 for every l = 1, …, n. If C is a cell in R^{n} with

C ⊆ E, (8.3.7) then the restriction of f to C has the same properties mentioned in the preceding paragraph. In particular, if E = R^{n}, then these properties hold for every x, w ∈ R^{n}.

8.4 Partial derivatives

Let m and n be positive integers, and let U be a nonempty open subset of R^{n}, with respect to the standard Euclidean metric. Also let f be a mapping from U into R^{m}, and let x ∈ U be given. If l is a positive integer less than or equal to n, then the lth partial derivative

∂_{l}f(x) = D_{l}f(x) = \frac{∂f}{∂x_{l}} (x) (8.4.1)

of f at x can be defined, as usual, as the derivative of f in the lth variable at x_{l}, when it exists, and with the other variables being fixed. More precisely, if we consider f as a function of the lth variable, with the kth variable equal to x_{k} when k ≠ l, then f is defined on an open subset of the real line that contains x_{l}. Note that ∂_{l}f(x) exists if and only if the lth partial derivative ∂_{l}f_{j}(x) of the jth component f_{j} of f at x exists for each j = 1, …, m, in which case ∂_{l}f_{j}(x) is the same as the jth component of ∂_{l}f(x) for each j = 1, …, m, as an element of R^{m}.

Let a_{1}, …, a_{n} and b_{1}, …, b_{n} be real numbers with a_{k} < b_{k} for each k = 1, …, n, and let C be the corresponding cell in R^{n}, as in (8.3.2). Suppose now that f is a mapping from C into R^{m}. Let x ∈ C and a positive integer l ≤ n be given. If a_{l} < x_{l} < b_{l}, then the lth partial derivative of f at x can be defined as the derivative of f as a function of the lth variable on [a_{l}, b_{l}] at x_{l}, when it exists, and with the other variables being fixed. If x_{l} = a_{l} or b_{l}, then one can use the corresponding one-sided derivative, as before.

Suppose that the lth partial derivative of f exists everywhere on C, and that it is continuous as a mapping from C into R^{m}, with respect to the restriction of the standard Euclidean metric on R^{n} to C, and the standard Euclidean metric on R^{m}. Let N be a norm on R^{m}. Remember that N is continuous on R^{m}, as in Section 6.7, so that N(∂_{l}f(x)) is continuous as a real-valued function on C. This implies that N(∂_{l}f(x)) is bounded on C, because C is compact, and that its maximum on C is attained.

Let d_{N} be the metric on R^{m} associated to N. Using the remarks in Section 8.2, we get that f is partially Lipschitz in the lth variable on C, with respect to d_{N} on R^{m}, with constant

C_{l} = sup_{x∈C} N(∂_{l}f(x)). (8.4.2)

This is the smallest value of C_{l} with this property, for the same reasons as before.

Let f be a mapping from an open subset U of R^{n} into R^{m} again. Let us say that f is continuously differentiable on U if for each l = 1, …, n, the lth partial derivative ∂_{l}f(x) of f exists at every x ∈ U, and defines a continuous mapping from U into R^{m}. This uses the restriction of the standard Euclidean metric on R^{n} to U, and the standard Euclidean metric on R^{m}, as usual. If C is a cell in R^{n} and C ⊆ U, then the restriction of f to C satisfies the analogous continuous differentiability property on C.

8.5 Using the mean value theorem

Let n be a positive integer, let a_{1}, …, a_{n} and b_{1}, …, b_{n} be real numbers with a_{k} < b_{k} for each k = 1, …, n, and let C be the corresponding cell in R^{n} again, as in (8.3.2). Also let f be a real-valued function on C, and let 1 ≤ l ≤ n be given. Suppose that f(x) is continuous as a real-valued function of x_{l} on [a_{l}, b_{l}], when x_{k} ∈ [a_{k}, b_{k}] is kept fixed for k ≠ l.

Suppose in addition that f(x) is differentiable as a function of x_{l} on (a_{l}, b_{l}) when x_{k} ∈ [a_{k}, b_{k}] is kept fixed for k ≠ l, so that ∂_{l}f(x) is defined under these conditions. Let us suppose as well that ∂_{l}f(x) is bounded, and put

C_{l} = sup{|∂_{l}f(x)| : x ∈ C, a_{l} < x_{l} < b_{l}}. (8.5.1)

Under these conditions, one can use the mean value theorem to get that f is partially Lipschitz in the lth variable on C, with respect to the standard Euclidean metric on R, with constant C_{l}. This is the smallest value of C_{l} with this property, as usual.

Now let U be a nonempty open subset of R^{n}, with respect to the standard Euclidean metric, and let f be a real-valued function on U. Suppose that ∂_{l}f(x) exists for every x ∈ U, which implies that f is continuous as a function of x_{l}, with the other variables kept fixed, everywhere on U. If

C ⊆ U (8.5.2)

and ∂_{l}f is bounded on C, then we get that f is partially Lipschitz in the lth variable on C, with constant C_{l} as in (8.5.1), as before.

If ∂_{l}f is bounded on U, then this holds for every cell C as in (8.5.2). If ∂_{l}f exists and is bounded on U for each l = 1, …, n, then one can use this to get that f is continuous on U, as in Exercise 7 on p239 of [155].

8.6 Directional derivatives

Let m and n be positive integers, and let U be an open subset of R^{n}, with respect to the standard Euclidean metric. Also let f be a mapping from U into R^{m}, and let x ∈ U and v ∈ R^{n} be given. It is easy to see that

U(x, v) = {t ∈ R : x + t v ∈ U} (8.6.1)

is an open set in the real line, with respect to the standard Euclidean metric on R. Of course, 0 ∈ U(x, v), because x ∈ U. Let us consider

f_{x,v}(t) = f(x + t v) (8.6.2) as an R^{m}-valued function of t ∈ U(x, v). If this function is differentiable at 0, then put D_{v}f(x) = f′_{x,v}(0). (8.6.3)

This is the directional derivative of f at x in the direction v.

Let r ∈ R be given, so that r v ∈ R^{n} too. Observe that

U(x, r v) = {t ∈ R : r t ∈ U(x, v)}, (8.6.4)

and f_{x,r v}(t) = f_{x,v}(r t) (8.6.5)

on U(x, r v). If D_{v}f(x) exists, then it is easy to see that D_{r v}f(x) exists, with

D_{r v}f(x) = r D_{v}f(x). (8.6.6)

Let e_{1}, …, e_{n} be the usual standard basis vectors in R^{n}, so that the lth coordinate of e_{k} is equal to 1 when k = l, and to 0 when k ≠ l. The directional derivative D_{e_{k}}f(x) (8.6.7)

is the same as the kth partial derivative ∂_{k}f(x), when it exists, for each k = 1, …, n.

If j ∈ {1, …, m}, then let f_{j}(x) ∈ R be the jth coordinate of f(x), as an element of R^{m}. Thus f_{j}(x) defines a real-valued function on U for each j = 1, …, m. Of course, the directional derivative D_{v}f(x) exists if and only if the directional derivative D_{v}f_{j}(x) exists for every j = 1, …, m, in which case D_{v}f_{j}(x) is the jth coordinate of D_{v}f(x) for each j. Similarly, note that f is continuously differentiable on U if and only if for each j = 1, …, m, f_{j} is continuously-differentiable as a real-valued function on U.

Suppose for the moment that D_{v}f(x) exists for every v ∈ R^{n}. In some situations, we may also have that

D_{v+w}f(x) = D_{v}f(x) + D_{w}f(x) (8.6.8)

for every v, w ∈ R^{n}. This means that

v ↦ D_{v}f(x) (8.6.9)

defines a linear mapping from R^{n} into R^{m}, because of (8.6.6). In this case, we get that

D_{v}f(x) = ∑_{k=1}^{n} v_{k} D_{e_{k}}f(x) = ∑_{k=1}^{n} v_{k} ∂_{k}f(x) (8.6.10)

for every v ∈ R^{n}, because v = ∑_{k=1}^{n} v_{k} e_{k}.

8.7 Differentiable mappings

Let m and n be positive integers again, and let U be a nonempty open subset of R^{n}, with respect to the standard Euclidean metric on R^{n}. Also let f be a mapping from U into R^{m}, and let x ∈ U be given. We say that f is differentiable at x if there is a linear mapping A from R^{n} into R^{m} such that

lim_{h→0} \frac{∥f(x + h) − f(x) − A(h)∥_{2,R^{m}}}{∥h∥_{2,R^{n}}} = 0. (8.7.1)

Here ∥ · ∥_{2,R^{m}} and ∥ · ∥_{2,R^{n}} are the standard Euclidean norms on R^{m} and R^{n}, respectively. It is easy to see that this implies that f is continuous at x.

Put U_{x} = {h ∈ R^{n} : x + h ∈ U}, (8.7.2)

which is an open set in R^{n} that contains 0. Of course, f(x + h) is defined for every h ∈ U_{x}, by construction. Put

a(h) = f(x + h) − f(x) − A(h) (8.7.3)

for each h ∈ U_{x}, so that

f(x + h) = f(x) + A(h) + a(h) (8.7.4)

for every h ∈ U_{x}. Using this, (8.7.1) says that

lim_{h→0} \frac{∥a(h)∥_{2,R^{m}}}{∥h∥_{2,R^{n}}} = 0. (8.7.5)

Similarly, if h ∈ U_{x} and h ≠ 0, then put

α(h) = a(h) ∥h∥_{2,R^{n}}^{−1}. (8.7.6)

Let us put α(0) = 0, so that

f(x + h) = f(x) + A(h) + α(h) ∥h∥_{2,R^{n}} (8.7.7)

for every h ∈ U_{x}, as in (8.7.4). Clearly (8.7.5) is equivalent to

lim_{h→0} ∥α(h)∥_{2,R^{m}} = 0. (8.7.8)

One can check directly that A is unique, when it exists. In this case, we put f′(x) = A, which may be called the differential of f at x.

If n = 1, then this reduces to the usual definition of the derivative of a function of one variable. More precisely, a linear mapping from R into R^{m} corresponds to multiplying a real number by a fixed element of R^{m}. The differential of f at x is the linear mapping that corresponds to multiplying a real number by the usual derivative of f at x, as an element of R^{m}.

Suppose that f is differentiable at x. If v ∈ R^{n}, then one can verify that the directional derivative of f at x in the direction v exists, with

D_{v}f(x) = f′(x)(v). (8.7.9)

In particular, if k ∈ {1, …, n}, then the kth partial derivative of f at x exists, with

\frac{∂f}{∂x_{k}} (x) = f′(x)(e_{k}). (8.7.10)

Here e_{k} is the kth standard basis vector in R^{n}, as before.

Remember that linear mappings from R^{n} into R^{m} are associated to m × n matrices of real numbers in a standard way, using the standard basis vectors in R^{n}. The entries of the matrix associated to f′(x) are given by the partial derivatives of the m components of f at x. If m = 1, then this is related to the gradient of f at x, as in (8.6.10).

If t ∈ R, then t f is a function defined on U with values in R^{m} too. It is easy to see t f is differentiable at x as well, with

(t f)′(x) = t f′(x). (8.7.11)

Similarly, let g be another mapping from U into R^{m} that is differentiable at x. One can check that f + g is differentiable at x, with

(f + g)′(x) = f′(x) + g′(x). (8.7.12)

One can verify that a mapping from U into R^{m} is differentiable at x if and only if its m components are differentiable at x as real-valued functions on U. Suppose now that f, g are real-valued functions on U that are differentiable at x. It is not too difficult to show that f g is differentiable at x as well, with

(f g)′(x) = g(x) f′(x) + f(x) g′(x). (8.7.13)

8.8 Pointwise Lipschitz conditions

Let (X, d_{X}) and (Y, d_{Y}) be metric spaces, and let f be a mapping from X into Y. Also let x be an element of X, let C be a nonnegative real number, and let r be a positive real number. Let us say that

(8.8.1) f is pointwise Lipschitz at x with constant C up to the scale r

if for every w ∈ X with d_{X}(x, w) < r, (8.8.2)

we have that d_{Y} (f(x), f(w)) ≤ C d_{X}(x, w). (8.8.3)

Of course, this implies that

(8.8.4) f is continuous at x.

We may allow r = +∞ here too, so that (8.8.2) holds for every w ∈ X.

Let m and n be positive integers, let U be a nonempty open subset of R^{n}, and let f be a mapping from U into R^{m}. Also let x ∈ U be given, and suppose that (8.8.5) f is differentiable at x.

If N and N_{0} are norms on R^{m} and R^{n}, respectively, then (8.7.1) is equivalent to

lim_{h→0} \frac{N(f(x + h) − f(x) − f′(x)(h))}{N_{0}(h)} = 0. (8.8.6)

This follows from the comparison between N, N_{0} and the standard Euclidean norms on R^{m}, R^{n}, respectively, as in Section 6.7.

Let ϵ > 0 be given, so that there is a δ > 0 such that

\frac{N(f(x + h) − f(x) − f′(x)(h))}{N_{0}(h)} < ϵ (8.8.7)

for every h ∈ R^{n} such that h ≠ 0,

N_{0}(h) < δ, (8.8.8)

and x+h ∈ U. This uses the comparison between N_{0} and the standard Euclidean norm on R^{n} again, to express (8.8.8) in terms of N_{0}(h). We may as well take δ small enough so that (8.8.8) implies that x + h ∈ U.

If h ∈ R^{n} satisfies (8.8.8), then it follows that

N(f(x + h) − f(x) − f′(x)(h)) ≤ ϵ N_{0}(h). (8.8.9)

This implies that

N(f(x + h) − f(x)) ≤ N(f(x + h) − f(x) − f′(x)(h)) + N(f′(x)(h))
(8.8.10)  ≤ N(f′(x)(h)) + ϵ N_{0}(h)

when (8.8.8) holds. Using this, we get that

(8.8.11)  N(f(x + h) − f(x)) ≤ ∥f′(x)∥_{op} N_{0}(h) + ϵ N_{0}(h)
= (∥f′(x)∥_{op} + ϵ) N_{0}(h)

when (8.8.8) holds, where ∥f′(x)∥_{op} is the operator norm of f′(x) corresponding to N_{0} and N, as in Section 7.5. This is a pointwise Lipschitz condition for f at x, with respect to the metrics associated to N_{0} and N.

Using (8.8.9) and the triangle inequality for N again, we get that

N(f′(x)(h)) ≤ N(f(x + h) − f(x)) + ϵ N_{0}(h) (8.8.12)

when h ∈ R^{n} satisfies (8.8.8). Let r be a positive real number, let C(r) be a nonnegative real number, and suppose that

(8.8.13) f is pointwise Lipschitz at x with constant C(r) up to scale r, with respect to the metric on R^{m} associated to N, and the restriction to U of the metric on R^{n} associated to N_{0}. This means that

N(f(x + h) − f(x)) ≤ C(r) N_{0}(h) (8.8.14)

for every h ∈ R^{n} such that N_{0}(h) < r (8.8.15)

and x + h ∈ U. As before, we may as well take r small enough so that (8.8.15) implies that x + h ∈ U, using the comparison between N_{0} and the standard Euclidean norm on R^{n}. Combining (8.8.12) and (8.8.14), we get that

N(f′(x)(h)) ≤ C(r) N_{0}(h) + ϵ N_{0}(h) = (C(r) + ϵ) N_{0}(h) (8.8.16)

for every h ∈ R^{n} with N_{0}(h) < min(δ, r). (8.8.17)

It is easy to see that this implies that (8.8.16) holds for every h ∈ R^{n}, because f′(x)(h) is linear in h. More precisely, one can reduce to the case where (8.8.17) holds, by multiplying h by a sufficiently small positive real number. It follows that ∥f′(x)∥_{op} ≤ C(r) + ϵ. (8.8.18)

This implies that ∥f′(x)∥_{op} ≤ C(r), (8.8.19)

because ϵ > 0 is arbitrary.

8.9 The chain rule

Let m, n, and p be positive integers, let U be a nonempty open subset of R^{n}, and let f be a mapping from U into R^{m}. Also let V be an open set in R^{m}, and suppose that f(U) ⊆ V. (8.9.1)

If g is a mapping from V into R^{p}, then the composition g ∘ f of f and g is defined as a mapping from U into R^{p}. Suppose that

(8.9.2) f is differentiable at x ∈ U,

and that (8.9.3) g is differentiable at f(x) ∈ V.

Thus the differential f′(x) of f at x is defined as a linear mapping from R^{n} into R^{m}, and the differential g′(f(x)) of g at f(x) is defined as a linear mapping from R^{m} into R^{p}.

Under these conditions, it is not too difficult to show that

(8.9.4) g ∘ f is differentiable at x, with (g ∘ f)′(x) = g′(f(x)) ∘ f′(x). (8.9.5)

This is the analogue of the chain rule in this situation. One can check that this reduces to the usual version of the chain rule for real-valued functions of one real varaible when m = n = p = 1.

If g is a linear mapping from R^{m} into R^{p}, then one can check the chain rule directly. Otherwise, one can use this to reduce to the case where

g′(f(x)) = 0, (8.9.6)

and one can verify that (g ∘ f)′(x) = 0 (8.9.7)

when f satisfies a pointwise Lipschitz condition at x.

In particular, (8.9.5) can be used to obtain the partial derivatives of g ∘ f at x in terms of the partial derivatives of f at x and the partial derivatives of g at f(x). Similarly, let v ∈ R^{n} be given, so that

w = f′(x)(v) = D_{v}f(x) (8.9.8)

is an element of R^{m}. Using (8.9.5), we get that

(8.9.9) D_{v}(g ∘ f)(x) = (g ∘ f)′(x)(v) = g′(f(x))(f′(x)(v))
= g′(f(x))(w) = (D_{w}g)(f(x)).

If m = 1, then this can be obtained from the usual chain rule for differentiable real-valued functions of one real variable.

Let l_{1}, l_{2} be positive integers. Of course, constant mappings from R^{l_{1}} into R^{l_{2}} are differentiable at every point in R^{l_{1}}, with differential equal to 0. If f = A is a linear mapping from R^{l_{1}} into R^{l_{2}}, then f is differentiable at every point x ∈ R^{l_{1}}, with f′(x) = A.

It is well known that t ↦ t^{2} is differentiable as a real-valued function on the real line, with derivative equal to 2 t. Let f be a real-valued function on U, and suppose that f is differentiable at x ∈ U. Using the chain rule, we get that

(8.9.10) f^{2} is differentiable at x

too, with (f^{2})′(x) = 2 f(x) f′(x). (8.9.11)

Let g be another real-valued function on U that is differentiable at x, so that g^{2} is differentiable at x as well, with

(g^{2})′(x) = 2 g(x) g′(x). (8.9.12)

Similarly, f + g is differentiable at x, so that (f + g)^{2} is differentiable at x, with

((f + g)^{2})′(x) = 2 (f(x) + g(x)) (f′(x) + g′(x)). (8.9.13)

Note that f g = (1/2) ((f + g)^{2} − f^{2} − g^{2}) (8.9.14)

on U. It follows that f g is differentiable at x, with (f g)′(x) as in (8.7.13).

Similarly, it is well known that t ↦ 1/t is differentiable as a real-valued function on R \ {0}, with derivative equal to −1/t^{2}. Let f be a real-valued function on U such that f(w) ≠ 0 for every w ∈ U, and suppose that f is differentiable at x ∈ U. The chain rule implies that

(8.9.15) 1/f is differentiable at x,

with (1/f)′(x) = (−1/f(x)^{2}) f′(x). (8.9.16)

In particular, one can use these remarks to get that polynomial functions on R^{n} are differentiable at every point. Similarly, rational functions are differentiable on open sets where the denominator is not zero.

8.10 Continuous differentiability

Let m and n be positive integers, let U be an open set in R^{n}, and let f be a mapping from U into R^{m}. If

(8.10.1) f is differentiable at every point in U,

then f is said to be differentiable on U. Remember that f is said to be continuously differentiable on U if for each k = 1, …, n, the kth partial derivative ∂_{k}f(x) exists at every x ∈ U, and is continuous as a mapping from U into R^{m}, as in Section 8.4. It is well known that

(8.10.2) continuously-differentiable mappings on U are differentiable

in the sense just mentioned. More precisely, if x ∈ U, then f′(x) is the linear mapping from R^{n} into R^{m} defined by

f′(x)(h) = ∑_{k=1}^{n} h_{k} ∂_{k}f(x) (8.10.3)

for every h ∈ R^{n}.

To see this, suppose that f is continuously differentiable on U, and let x ∈ U be given. Suppose for the moment that

∂_{k}f(x) = 0 for each k = 1, …, n, (8.10.4)

so that we would like to show that f is differentiable at x, with f′(x) = 0. This is equivalent to saying that

lim_{h→0} \frac{∥f(x + h) − f(x)∥_{2,R^{m}}}{∥h∥_{2,R^{n}}} = 0. (8.10.5)

Because ∂_{k}f is continuous on U, (8.10.4) implies that ∂_{k}f is as small as we want near x. One can use this to get (8.10.5), as in Sections 8.3 and 8.4.

Alternatively, it is not difficult to reduce to the case where m = 1. This permits one to use the mean value theorem, as in Section 8.5. In order to estimate

f(x + h) − f(x), (8.10.6)

one can go from x to x + h in n steps, only changing one coordinate in each step, as in Section 8.3.

We can reduce to the case where (8.10.4) holds, as follows. If w ∈ U, then put

f̃(w) = f(w) − ∑_{k=1}^{n}(w_{k} − x_{k}) ∂_{k}f(x). (8.10.7)

This defines a mapping from U into R^{m}, as a function of w. The partial derivatives of f̃ are given by

∂_{l}f̃(w) = ∂_{l}f(w) − ∂_{l}f(x) (8.10.8)

for every w ∈ U and l = 1, …, n. In particular, f̃ is continuously differentiable on U, with

∂_{l}f̃(x) = ∂_{l}f(x) − ∂_{l}f(x) = 0 (8.10.9)

for every l = 1, …, n. This implies that

lim_{h→0} \frac{∥f̃(x + h) − f̃(x)∥_{2,R^{n}}}{∥h∥_{2,R^{m}}} = 0, (8.10.10)

as before. This is the same as saying that f is differentiable at x, with f′(x) as in (8.10.3).

We can think of f′(x) as a function of x ∈ U with values in the space L(R^{n}, R^{m}) of linear mappings from R^{n} into R^{m}. One can get nice metrics on L(R^{n}, R^{m}) from norms in the usual way, such as the Hilbert–Schmidt norm or the operator norm associated to the standard Euclidean norms on R^{n} and R^{m}. If f is continuously differentiable on U, then

(8.10.11) f′ is continuous as a mapping from U into L(R^{n}, R^{m}),

with respect to such a metric. Conversely, if f is differentiable on U, and if f′ is continuous on U in this sense, then

(8.10.12) the partial derivatives of f are continuous on U.

Thus continuous differentiability on U can be defined equivalently in this way.

8.11 Another basic Lipschitz estimate

Let m and n be positive integers, let U be a nonempty open subset of R^{n}, and let f be a continuously-differentiable mapping from U into R^{m}. Thus f is differentiable at every x ∈ U, as in the previous section, so that f′(x) is defined as a linear mapping from R^{n} into R^{m} for every x ∈ U. We also have that f′ is continuous on U, as before.

Let N and N_{0} be norms on R^{m} and R^{n}, respectively, and let ∥ · ∥_{op} be the corresponding operator norm for linear mappings from R^{n} into R^{m}, as in Section 7.5. Remember that N and N_{0} can be compared with the standard Euclidean norms on R^{m} and R^{n}, respectively, as in Section 6.7. This leads to an analogous comparison between ∥ · ∥_{op} and the operator norm for linear mappings from R^{n} into R^{m} with respect to the standard Euclidean norms on R^{n} and R^{m}. In particular, one can use this to check that ∥·∥_{op} is continuous on the space of linear mappings from R^{n} into R^{m}, with respect to the metric that corresponds to the Hilbert–Schmidt norm on m × n matrices of real numbers. This implies that

(8.11.1) ∥f′(x)∥_{op} is continuous as a real-valued function on U,

because f′(x) is continuous on U. If x ∈ U and v ∈ R^{n}, then the directional derivative D_{v}f(x) of f at x in the direction of v is the same as f′(x) applied to v, as in (8.7.9). Hence

N(D_{v}f(x)) = N(f′(x)(v)) ≤ ∥f′(x)∥_{op} N_{0}(v), (8.11.2)

by the definition of ∥ · ∥_{op}.

Let us suppose from now on in this section that U is also convex in R^{n}. This means that for each x, w ∈ U and t ∈ R with 0 ≤ t ≤ 1 we have that

t x + (1 − t) w ∈ U. (8.11.3)

Let x, w ∈ U be given, and consider

f(t x + (1 − t) w) = f(w + t (x − w)) (8.11.4)

as a function of t ∈ [0, 1] with values in R^{m}. Because f is differentiable on U, we can differentiate (8.11.4) in t, to get that

\frac{d}{dt}(f(t x + (1 − t) w)) = (D_{(x−w)}f)(t x + (1 − t) w) (8.11.5)

for every t ∈ [0, 1]. It follows that

∫_{0}^{1}(D_{(x−w)}f)(t x + (1 − t) w) dt = f(x) − f(w), (8.11.6)

by the fundamental theorem of calculus.

This implies that

N(f(x) − f(w)) ≤ ∫_{0}^{1} N((D_{(x−w)}f)(t x + (1 − t) w)) dt, (8.11.7)

as in Section 8.1. Combining this with (8.11.2), we get that

N(f(x) − f(w)) ≤ N_{0}(x − w) ∫_{0}^{1} ∥f′(t x + (1 − t) w)∥_{op} dt. (8.11.8)

Suppose that f′ is bounded on U, and observe that

N(f(x) − f(w)) ≤ ( sup_{u∈U} ∥f′(u)∥_{op}) N_{0}(x − w), (8.11.9)

by (8.11.8). This means that f is Lipschitz as a mapping from U into R^{m}, using the restriction to U of the metric on R^{n} associated to N_{0}, and the metric on R^{m} associated to N. More precisely, the corresponding Lipschitz constant is given by

Lip(f) = sup_{u∈U} ∥f′(u)∥_{op}. (8.11.10)

Indeed, Lip(f) is less than or equal to the right side of (8.11.10), because of (8.11.9). In order to get the opposite inequality, one can check directly that ∥f′(u)∥_{op} is less than or equal to Lip(f) for every u ∈ U, as in Section 8.8.

If m = 1, then we can take N to be the usual absolute value function on R. In this case, one can use the mean value theorem to get that

|f(x) − f(w)| ≤ ( sup_{u∈U} ∥f′(u)∥_{op}) N_{0}(x − w) (8.11.11)

for every x, w ∈ U when f is differentiable on U, and f′ is bounded on U. The corresponding Lipschitz constant of f on U is given by (8.11.10), as before.

8.12 Some remarks about connectedness

Let (X, d_{X}) be a metric space. Remember that subsets A, B of X are said to be separated in X if \overline{A} ∩ B = A ∩ \overline{B} = ∅, (8.12.1)

where \overline{A}, \overline{B} are the closures of A, B in X, respectively. A subset E of X is said to be connected if E cannot be expressed as the union of two nonempty separated subsets of X.

Let X_{0} be a subset of X, and remember that X_{0} may be considered as a metric space with respect to the restriction of d_{X}(x, w) to x, w ∈ X_{0}. If A ⊆ X_{0}, then let \overline{A}_{X} be the closure of A in X, and let \overline{A}_{X_{0}} be the closure of A in X_{0}. One can check that \overline{A}_{X_{0}} = \overline{A}_{X} ∩ X_{0} (8.12.2)

142 CHAPTER 8. SOME TOPICS RELATED TO DIFFERENTIABILITY

for every A ⊆ X_{0}. More precisely, if x is any element of X_{0}, then one can verify that x is a limit point of A as a subset of X if and only if x is a limit point of A as a subset of X_{0}.

If A, B are subsets of X_{0}, then it follows that (8.12.3) A, B are separated as subsets of X_{0} if and only if (8.12.4) A, B are separated as subsets of X. If E is a subset of X_{0}, then the previous statement implies that (8.12.5) E is connected as a subset of X_{0} if and only if

E is connected as a subset of X.

Note that disjoint closed subsets of X are separated in X. One can check that (8.12.6) disjoint open subsets of X are separated in X too. If X is not connected, then X can be expressed as the union of two nonempty separated sets A, B. In this case, one can verify that (8.12.7) A and B are each both open and closed in X.

Suppose that E ⊆ X is not connected, so that E = A∪B for some nonempty separated subsets A, B of X. If E is a closed set in X, then one can check that (8.12.8) A and B are closed sets in X. If E is an open set in X, then one can verify that (8.12.9) A and B are open sets in X.

8.13 Locally constant mappings

Let (X, d_{X}) be a metric space, let Y be a set, and let f be a mapping from X into Y. We say that f is locally constant on X if for every x ∈ X there is a positive real number r(x) such that f(w) = f(x) (8.13.1) for every w ∈ X with d_{X}(x, w) < r(x). If Y is equipped with a metric d_{Y}, then (8.13.2) any locally constant mapping from X into Y is continuous. If Y is equipped with the discrete metric, as mentioned in Section 3.7, then it is easy to see that (8.13.3) every continuous mapping from X into Y is locally constant.

Let f be a locally constant mapping from X into Y, and let E be a subset of X. If E is connected as a subset of X, then

(8.13.4) f is constant on E.

Equivalently, if f is not constant on E, then E is not connected. One way to see this is to use the fact that f is continuous with respect to the discrete metric on Y. It is well known that any continuous mapping from X into another metric space maps connected subsets of X to connected subsets of the range. If E is connected in X, then it follows that

(8.13.5) f(E) is connected with respect to the discrete metric on Y.

However, one can check that a subset of Y is connected with respect to the discrete metric if and only if it has at most one element.

If X is not connected, then X can be expressed as the union of two nonempty disjoint open sets, as in the previous section. One can use this to get a mapping f from X into any set Y with at least two elements such that f is locally constant on X, and not constant on X.

Let m and n be positive integers, let U be a nonempty open subset of R^{n}, and let f be a mapping from U into R^{m}. Suppose that f is differentiable on U, and that f′(x) = 0 (8.13.6)

for every x ∈ U. If U is convex, then it follows that f is constant on U, as in Section 8.11. Similarly,

(8.13.7) the restriction of f to any convex open subset of U is constant.

One can check that open balls in R^{n} with respect to the metric associated to any norm on R^{n} are convex. This implies that

(8.13.8) f is locally constant on U,

with respect to the restriction to U of the standard Euclidean metric on R^{n}. Of course, if f is any mapping from U into R^{m} that is locally constant with respect to the restriction to U of the standard Euclidean metric on R^{n}, then f is differentiable on U, and satisfies (8.13.6) for every x ∈ U.

If U is also connected as a subset of R^{n}, then (8.13.8) implies that

(8.13.9) f is constant on U,

as before. More precisely, this uses the fact that U is connected as a subset of itself, with respect to the restriction of the standard Euclidean metric on R^{n} to U, as in the previous section. This is related to Exercise 9 on p239 of [155]. If U is not connected as a subset of R^{n}, then U can be expressed as the union of two nonempty disjoint open subsets of R^{n}, as in the previous section. One can use this to get a locally constant mapping from U into R^{m} that is not constant on U, as before.

144 CHAPTER 8. SOME TOPICS RELATED TO DIFFERENTIABILITY

8.14 Some local Lipschitz conditions

Let (X, d_{X}) and (Y, d_{Y}) be metric spaces, and let f be a mapping from X into Y. If x ∈ X and r > 0, then we let B_{X}(x, r) (8.14.1) be the open ball in X centered at x with radius r with respect to d_{X}, as usual. Let us say that (8.14.2) f is locally Lipschitz with constant C ≥ 0 at x if there is a positive real number r such that (8.14.3) the restriction of f to B_{X}(x, r) is Lipschitz with constant C, with respect to the restriction of d_{X}(·, ·) to B_{X}(x, r). Note that this implies that f is pointwise Lipschitz at x with constant C up to the scale r, as in Section 8.8.

We may also simply say that (8.14.4) f is locally Lipschitz at x if there is a nonnegative real number C such that f is locally Lipschitz at x with constant C. Similarly, we may say that (8.14.5) f is locally Lipschitz on X if f is locally Lipschitz at every x ∈ X. Of course, this implies that (8.14.6) f is continuous on X. Note that f is locally constant on X exactly when (8.14.7) f is locally Lipschitz with constant C = 0 at every x ∈ X.

Let W be an open subset of X, and suppose that the restriction of f to W is Lipschitz with constant C ≥ 0, with respect to the restriction of d_{X}(·, ·) to W. This implies that for each w ∈ W, f is locally Lipschitz at w with constant C.

Let K be a compact subset of X, and suppose that (8.14.8) for each x ∈ K, f is locally Lipschitz at x. This means that for every x ∈ K, there is a positive real number r(x) and a nonnegative real number C(x) such that the restriction of f to B_{X}(x, r(x)) is Lipschitz with constant C(x). Thus K can be covered by open balls of this type, and so there are finitely many elements x_{1}, …, x_{l} of K such that

K ⊆ ⋃_{j=1}^{l} B_{X}(x_{j}, r(x_{j})), (8.14.9)

because K is compact. If we put

C = max_{1≤j≤l} C(x_{j}), (8.14.10)

then it is easy to see that

(8.14.11) f is locally Lipschitz with constant C

at every x ∈ K. More precisely, this holds at every x in the right side of (8.14.9).

Let m and n be positive integers, let U be a nonempty open subset of R^{n}, and let f be a continuously-differentiable mapping from U into R^{m}. Also let N and N_{0} be norms on R^{m} and R^{n}, respectively, and let ∥ · ∥_{op} be the corresponding operator norm for linear mappings from R^{n} into R^{m}. Suppose that E is a nonempty convex subset of R^{n} such that E ⊆ U, and that

(8.14.12) ∥f′(u)∥_{op} is bounded on E.

If x, w ∈ E, then

N(f(x) − f(w)) ≤ ( sup_{u∈E} ∥f′(u)∥_{op}) N_{0}(x − w), (8.14.13)

as in Section 8.11. This means that

(8.14.14) the restriction of f to E is Lipschitz

with respect to the metric on R^{m} associated to N, and the restriction to E of the metric on R^{n} associated to N_{0}.

If E is also closed and bounded in R^{n}, then E is compact in R^{n}. It is well known that this implies that E is compact as a subset of U, with respect to the restriction to U of the standard Euclidean metric on R^{n}. Remember that ∥f′(u)∥_{op} is continuous as a real-valued function on U, as in Section 8.11. It follows that (8.14.12) holds under these conditions.

One can check that open and closed balls in R^{n} with respect to the metric associated to a norm are convex subsets of R^{n}, as before. If v ∈ U and ϵ is a positive real number, then

∥f′(u)∥_{op} < ∥f′(v)∥_{op} + ϵ (8.14.15)

when u ∈ U is sufficiently close to v, because ∥f′(u)∥_{op} is continuous at v. One can use this to get that f is locally Lipschitz at v with constant

C = ∥f′(v)∥_{op} + ϵ, (8.14.16)

with respect to the metric on R^{m} associated to N, and the restriction to U of the metric on R^{n} associated to N_{0}. We also have that ∥f′(u)∥_{op} is bounded on any closed ball contained in U, as in the preceding paragraph. This implies that the restriction of f to such a ball is Lipschitz, as before.

More on differentiable mappings

9.1 Continuously-differentiable mappings

Let m and n be positive integers, let U be an open subset of R^{n}, and let f be a continuously-differentiable mapping from U into R^{m}. Also let x be an element of U, and put A = f′(x), (9.1.1)

for convenience. Observe that f − A is continuously differentiable as a mapping from U into R^{m} too. If w ∈ U, then

(f − A)′(w) = f′(w) − A = f′(w) − f′(x). (9.1.2)

This tends to 0 as w approaches x, because f is continuously differentiable on U.

Let N_{0} and N be norms on R^{n} and R^{m}, respectively. Thus d_{N_{0}}(v, w) = N_{0}(v−w) and d_{N}(y, z) = N(y−z) are metrics on R^{n} and R^{m}, respectively. The corresponding operator norm for linear mappings from R^{n} into R^{m} is denoted ∥ · ∥_{op}, as usual. In particular,

∥f′(w) − A∥_{op} = ∥f′(w) − f′(x)∥_{op} → 0 as w → x, (9.1.3)

because f is continuously-differentiable on U.

Let U_{1} be a convex open subset of R^{n} such that x ∈ U_{1} and U_{1} ⊆ U. Suppose that f′ is bounded on U_{1}, so that f′ − A is bounded on U_{1} as well. Put

b_{1} = sup_{w∈U_{1}} ∥f′(w) − A∥_{op}. (9.1.4)

The restriction of f − A to U_{1} is Lipschitz with constant b_{1}, as in Sections 8.11 and 8.14. This uses the restriction of d_{N_{0}} to U_{1}, and d_{N} on R^{m}.

Suppose that N(A(v)) = N(f′(x)(v)) ≥ c N_{0}(v) (9.1.5)

for some c > 0 and all v ∈ R^{n}. This implies that

d_{N}(A(u), A(v)) ≥ c d_{N_{0}}(u, v) (9.1.6)

for every u, v ∈ R^{n}, as in (7.9.5). Remember that there is a positive real number c such that (9.1.5) holds for all v ∈ R^{n} exactly when A is one-to-one on R^{n}, as in Sections 7.9 and 7.10.

Suppose that b_{1} < c. (9.1.7)

Under these conditions, we have that

d_{N}(f(u), f(v)) ≥ (c − b_{1}) d_{N_{0}}(u, v) (9.1.8)

for every u, v ∈ U_{1}. More precisely, this corresponds to (7.11.5), with some changes in notation.

Similarly, if w ∈ U_{1}, then

N(f′(w)(v)) ≥ (c − b_{1}) N_{0}(v) (9.1.9)

for every v ∈ R^{n}. This corresponds to (7.11.10).

Suppose for instance that U_{1} is the open ball in R^{n} centered at x with radius r > 0 with respect to d_{N_{0}}. This is a convex open subset of R^{n}, as before. If r is small enough, then U_{1} ⊆ U, because U is an open set in R^{n} that contains x. We can make b_{1} as small as we want by taking r small enough, because of (9.1.3). In particular, (9.1.7) holds when r is sufficiently small.

Some more properties of mappings like these will be discussed in Section 9.7.

9.2 Invertible linear mappings

Let n be a positive integer, and let

L(R^{n}) = L(R^{n}, R^{n}) (9.2.1)

be the space of linear mappings from R^{n} into itself. Also let N be a norm on R^{n}, and let ∥A∥_{op} be the corresponding operator norm for A ∈ L(R^{n}). More precisely, this uses N on R^{n} as both the domain and range of A. Remember that d_{op}(A, B) = ∥A − B∥_{op} (9.2.2)

defines a metric on L(R^{n}).

Let I be the identity mapping on R^{n}, which sends every element of R^{n} to itself. It is easy to see that ∥I∥_{op} = 1. (9.2.3)

148 CHAPTER 9. MORE ON DIFFERENTIABLE MAPPINGS

If A, B ∈ L(R^{n}), then their composition B ∘ A defines a linear mapping from R^{n} into itself as well, so that B ∘ A ∈ L(R^{n}). Remember that ∥B ∘ A∥_{op} ≤ ∥A∥_{op} ∥B∥_{op}, (9.2.4) as in Section 7.8.

A linear mapping A from R^{n} into itself is said to be invertible if A is a one-to-one mapping from R^{n} onto itself. In this case, the inverse mapping A^{−1} is linear as well. The collection of invertible linear mappings on R^{n} may be denoted GL(R^{n}). It is well known that A ∈ L(R^{n}) is one-to-one if and only if A(R^{n}) = R^{n}.

If A ∈ GL(R^{n}), then N(A^{−1}(u)) ≤ ∥A^{−1}∥_{op} N(u) (9.2.5) for every u ∈ R^{n}. More precisely, ∥A^{−1}∥_{op} is the smallest nonnegative real number with this property, by definition of the operator norm. Equivalently, (9.2.5) says that N(v) ≤ ∥A^{−1}∥_{op} N(A(v)) (9.2.6) for every v ∈ R^{n}. Note that ∥A^{−1}∥_{op} > 0, because A^{−1} ≠ 0. Thus (9.2.6) is the same as saying that ∥A^{−1}∥_{op}^{−1} N(v) ≤ N(A(v)) (9.2.7) for every v ∈ R^{n}.

Suppose that B ∈ L(R^{n}) satisfies ∥A − B∥_{op} < 1/∥A^{−1}∥_{op}. (9.2.8) Using (9.2.7), we get that (∥A^{−1}∥_{op}^{−1} − ∥A − B∥_{op}) N(v) ≤ N(B(v)) (9.2.9) for every v ∈ R^{n}, as in Section 7.11. In particular, this implies that B(v) = 0 only when v = 0, so that B is injective. It follows that B(R^{n}) = R^{n}, and hence B ∈ GL(R^{n}), (9.2.10) as before. This shows that (9.2.11) GL(R^{n}) is an open set in L(R^{n}), with respect to the metric (9.2.2) associated to the operator norm.

Note that N(B^{−1}(u)) ≤ (∥A^{−1}∥_{op}^{−1} − ∥A − B∥_{op})^{−1} N(u) (9.2.12) for every u ∈ R^{n}, by (9.2.9). This means that

(9.2.13)  ∥B^{−1}∥_{op} ≤ (∥A^{−1}∥_{op}^{−1} − ∥A − B∥_{op})^{−1}
= \frac{∥A^{−1}∥_{op}}{1 − ∥A^{−1}∥_{op} ∥A − B∥_{op}}

9.3. THE INVERSE FUNCTION THEOREM 149 in this situation. If ∥A − B∥_{op} ≤ 1/(2∥A^{−1}∥_{op}), (9.2.14) then we get that ∥B^{−1}∥_{op} ≤ 2 ∥A^{−1}∥_{op}. (9.2.15) Of course, (9.2.14) implies (9.2.8).

We would like to look at continuity properties of A ↦ A^{−1} (9.2.16) on GL(R^{n}). If A, B ∈ GL(R^{n}), then it is easy to see that A^{−1} − B^{−1} = A^{−1} ∘ (B − A) ∘ B^{−1}. (9.2.17) This implies that ∥A^{−1} − B^{−1}∥_{op} ≤ ∥A^{−1}∥_{op} ∥B^{−1}∥_{op} ∥A − B∥_{op}. (9.2.18) If (9.2.14) holds, then we get that ∥A^{−1} − B^{−1}∥_{op} ≤ 2 ∥A^{−1}∥_{op}^{2} ∥A − B∥_{op}, (9.2.19) by (9.2.15). It follows that B ↦ B^{−1} is continuous at A, with respect to the metric (9.2.2) associated to the operator norm, and its restriction to GL(R^{n}).

9.3 The inverse function theorem

Let n be a positive integer, and let N be a norm on R^{n}, so that d_{N}(v, w) = N(v − w) is a metric on R^{n}. Using N, we get the corresponding operator norm ∥ · ∥_{op} on the space L(R^{n}) of linear mappings from R^{n} into itself, and its associated metric, as in (9.2.2). Of course, one can simply take N to be the standard Euclidean norm on R^{n}.

Let W be an open subset of R^{n}, and let f be a continuously-differentiable mapping from W into R^{n}. Also let w ∈ W be given, and suppose that (9.3.1) f′(w) is invertible as a linear mapping from R^{n} into itself. Under these conditions, the inverse function theorem states that there are open subsets U and V of R^{n} with the following properties, which we describe in two parts.

In the first part, w ∈ U, U ⊆ W, and f(w) ∈ V. (9.3.2) We are able to choose U and V so that (9.3.3) f is a one-to-one mapping from U onto V.

We can also choose U so that

(9.3.4) f′(x) is invertible as a linear mapping on R^{n} for each x ∈ U.

In particular, we can define a mapping g from V onto U to be the inverse of the restriction of f to U, so that

g(f(x)) = x (9.3.5)

for every x ∈ U. The second part of the inverse function theorem says that

(9.3.6) g is continuously-differentiable as a mapping from V into R^{n},

with g′(y) = (f′(g(y)))^{−1} (9.3.7)

for every y ∈ V. More precisely, the right side of (9.3.7) is the inverse of f′(g(y)), as a linear mapping on R^{n}.

If x ∈ U and g is differentiable at f(x) ∈ V, then the chain rule implies that

g′(f(x)) ∘ f′(x) = I, (9.3.8)

because of (9.3.5). This is the same as (9.3.7), with y = f(x). The proof of the inverse function theorem will be discussed in the next section. Note that the proof could be simplified when n = 1, using the mean value theorem and the intermediate value theorem, which is related to Exercise 2 on p114 of [155].

9.4 Proving the inverse function theorem

Let us continue with the same notation and hypotheses as in the previous section. Remember that f′(x) is continuous as a function of x ∈ W with values in L(R^{n}), as in Section 8.10. We have also seen that the collection GL(R^{n}) of invertible linear mapping from R^{n} into itself is an open set in L(R^{n}), as in Section 9.2. It follows that

{x ∈ W : f′(x) ∈ GL(R^{n})} (9.4.1)

is an open set in R^{n}, because W is an open set, by hypothesis. Of course, (9.4.1) is the same as (f′)^{−1}(GL(R^{n})), (9.4.2)

where f′ is considered as a mapping from W into L(R^{n}).

To prove the inverse function theorem, we can reduce to the case where

(9.4.3) f′(w) is the identity mapping I on R^{n}.

More precisely, this can be obtained by replacing f with

(f′(w))^{−1} ∘ f, (9.4.4) as a continuously-differentiable mapping from W into Rn. It is easy to see that the differential of (9.4.4) at w is equal to I, by construction. If we can prove the inverse function theorem in this case, then the analogous conclusions for f can be obtained by composing (9.4.4) with f′(w), as a linear mapping from R^{n} into itself.

Let B_{N}(w, r) be the open ball in R^{n} centered at w with radius r > 0 with respect to d_{N}(·, ·), as before. Note that

B_{N}(w, r) ⊆ W (9.4.5)

when r is sufficiently small, because W is an open set that contains w. We also have that f′(x) − I is as small as we want when x ∈ R^{n} is close enough to w, because f′(x) is continuous at w. In particular, we can choose r > 0 small enough so that (9.4.5) holds and

∥f′(x) − I∥_{op} ≤ 1/2 (9.4.6)

for every x ∈ R^{n} with N(x − w) < r. This implies that f′(x) is invertible when N(x − w) < r, as in Section 9.2.

Using (9.4.6), we get that

(9.4.7) the restriction of f(x) − x to B_{N}(w, r)

is Lipschitz with constant c = 1/2,

with respect to d_{N}(·, ·) and its restriction to B_{N}(w, r), as in Section 8.11. This uses the fact that the differential of f(x)−x is equal to f′(x)−I for each x ∈ W. It follows that (9.4.8) f is bilipschitz on B_{N}(w, r),

as in Section 7.11, and in particular that f is one-to-one on B_{N}(w, r). The restriction of f to B_{N}(w, r) is an open mapping too, as in Section 7.14. The first part of the inverse function theorem now follows by taking

U = B_{N}(w, r) (9.4.9)

and V = f(B_{N}(w, r)). (9.4.10)

Let g be the inverse of the restriction of f to B_{N}(w, r), as before. This is a Lipschitz mapping from V into R^{n}, with respect to d_{N}(·, ·) and its restriction to V, because f is bilipschitz on B_{N}(w, r), as in the preceding paragraph. One can verify that (9.4.11) g is differentiable on V,

with differential as in (9.3.7), using the differentiability of f. This corresponds to part of part (b) of Theorem 9.24 on p221 of [155], for instance. The continuity of g′ on V follows from (9.3.7), because f′ and g are continuous, and using the continuity of (9.2.16) on GL(R^{n}).

9.5 Some remarks about R^{n+m}

Let n and m be positive integers, so that R^{n}, R^{m}, and R^{n+m} are the usual spaces of n-tuples, m-tuples, and (n + m)-tuples of real numbers, respectively. It will sometimes be convenient for us to identify R^{n+m} with the Cartesian product R^{n} × R^{m} of R^{n} and R^{m}. Thus, if x = (x_{1}, …, x_{n}) ∈ R^{n} and y = (y_{1}, …, y_{m}) ∈ R^{m}, then we may identify

(x, y) ∈ R^{n} × R^{m} (9.5.1)

with (x_{1}, …, x_{n}, y_{1}, …, y_{m}) ∈ R^{n+m}. (9.5.2)

In particular, (x, 0) and (0, y) can be identified with elements of R^{n+m}. If f is a function defined on a subset of R^{n+m}, then we may use

f(x, y) (9.5.3)

to denote the value of f at the point in R^{n+m} identified with (x, y), when that point is in the domain of f.

Let A be a linear mapping from R^{n+m} into R^{n}. If x ∈ R^{n} and y ∈ R^{m}, then put A_{1}(x) = A(x, 0) (9.5.4)

and A_{2}(y) = A(0, y). (9.5.5)

This defines A_{1} and A_{2} as linear mappings from R^{n} and R^{m} into R^{n}, respectively. Note that A(x, y) = A_{1}(x) + A_{2}(y) (9.5.6)

for every x ∈ R^{n} and y ∈ R^{m}. Conversely, if A_{1} and A_{2} are linear mappings from R^{n} and R^{m} into R^{n}, respectively, then (9.5.6) defines a linear mapping from R^{n+m} into R^{n}.

Let A be a linear mapping from R^{n+m} into R^{n}, and let A_{1} and A_{2} be as in (9.5.4) and (9.5.5), respectively. Also let x ∈ R^{n} and y ∈ R^{m} be given, and observe that A(x, y) = 0 (9.5.7)

if and only if A_{1}(x) = −A_{2}(y), (9.5.8)

by (9.5.6). If A_{1} is a one-to-one mapping from R^{n} onto itself, then (9.5.8) is the same as saying that x = −A_{1}^{−1}(A_{2}(y)). (9.5.9)

In particular, for each y ∈ R^{m} there is a unique x ∈ R^{n} such that (9.5.7) holds in this situation.

Clearly A_{1}(R^{n}) ⊆ A(R^{n+m}), (9.5.10) by the definition (9.5.4) of A_{1}. In particular, if A_{1}(R^{n}) = R^{n}, then

A(R^{n+m}) = R^{n}. (9.5.11)

Consider the mapping Â from R^{n+m} into itself defined by

Â(x, y) = (A(x, y), y) (9.5.12)

for every x ∈ R^{n} and y ∈ R^{m}. This is a linear mapping from R^{n+m} into itself, which can also be expressed as

Â(x, y) = (A_{1}(x) + A_{2}(y), y) (9.5.13)

for every x ∈ R^{n} and y ∈ R^{m}, by (9.5.6). If A_{1} is a one-to-one mapping from R^{n} onto itself, then one can check that

(9.5.14)  Â is a one-to-one mapping from R^{n+m} onto itself.

It is easy to see that the converse holds as well.

Let u_{1}, …, u_{n+m} be the standard basis vectors in R^{n+m}, so that for each l = 1, …, m+n, the lth coordinate of u_{l} is equal to 1, and the other coordinates of u_{l} are equal to 0. If A is a linear mapping from R^{n+m} onto R^{n}, then

(9.5.15) R^{n} is spanned by A(u_{1}), …, A(u_{n+m}).

Under these conditions, it is well known that there is a subset K of {1, …, n+m} such that K has exactly n elements, and

(9.5.16) A(u_{k}), k ∈ K, forms a basis for R^{n}.

If K = {1, …, n}, then the linear mapping A_{1} on R^{n} associated to A as in (9.5.4) is invertible. Otherwise, one could rearrange the coordinates on R^{n+m} to reduce to this case.

9.6 The implicit function theorem

Let n and m be positive integers again, and let O be an open subset of R^{n+m}, with respect to the standard Euclidean metric. Also let f be a continuouslydifferentiable mapping from O into R^{n}. Suppose that a ∈ R^{n} and b ∈ R^{m} have the properties that (a, b) ∈ O and

f(a, b) = 0, (9.6.1)

where (a, b) is identified with an element of R^{n+m}, as in the previous section.

Put A = f′(a, b), (9.6.2)

which is a linear mapping from R^{n+m} into R^{n}. This leads to a linear mappings A_{1} and A_{2} from R^{n} and R^{m} into R^{n}, respectively, as in (9.5.4) and (9.5.5).

Suppose that

(9.6.3) A_{1} is a one-to-one mapping from R^{n} onto itself.

Under these conditions, the implicit function theorem states that there are open sets U ⊆ R^{n+m} and W ⊆ R^{m} with the following properties, as in Theorem 9.28 on p224 of [155].

First, (a, b) ∈ U, U ⊆ O, and b ∈ W. (9.6.4)

Second, for each y ∈ W there is a unique x ∈ R^{n} such that

(x, y) ∈ U and f(x, y) = 0. (9.6.5)

If y ∈ W, then let g(y) be the element of R^{n} just mentioned, so that g is a mapping from W into R^{n}, g(b) = a, and for every y ∈ W we have that

(g(y), y) ∈ U (9.6.6)

and f(g(y), y) = 0. (9.6.7)

We also have that g is continuously differentiable on W, with

g′(b) = −A_{1}^{−1} ∘ A_{2}. (9.6.8)

To prove the implicit function theorem, we put

F (x, y) = (f(x, y), y) (9.6.9)

for every (x, y) ∈ O, where the right side is identified with an element of R^{n+m}, as before. This defines F as a continuously-differentiable mapping from O into R^{n+m}, with F′(a, b) = Â, (9.6.10)

where Â is as in (9.5.12). In this situation, Â is a one-to-one linear mapping from R^{n+m} onto itself, so that the inverse function theorem can be applied to F at (a, b).

Thus there are open sets U, V in R^{n+m} such that (a, b) ∈ U, U ⊆ O,

F (a, b) = (f(a, b), b) = (0, b) ∈ V, (9.6.11)

and F is a one-to-one mapping from U onto V. The inverse G of the restriction of F to U is a continuously-differentiable mapping from V onto U. We also have that G preserves the last m coordinates of points in its domain, because of the analogous property of F.

Observe that W = {y ∈ R^{m} : (0, y) ∈ V } (9.6.12)

is an open subset of R^{m} that contains b. If y ∈ W, then we can take g(y) ∈ R^{n} so that (g(y), y) = G(0, y). (9.6.13)

See [155] or other texts for more details.

9.7 Local embeddings

Let n and m be positive integers, and let us continue to identify R^{n+m} with R^{n} × R^{m}, as in Section 9.5. Suppose that A is a linear mapping from R^{n} into R^{n+m}. If v ∈ R^{n}, then A(v) ∈ R^{n+m} can be expressed as

A(v) = (A_{1}(v), A_{2}(v)), (9.7.1)

where A_{1}(v) ∈ R^{n} and A_{2}(v) ∈ R^{m}, using the identification just mentioned. More precisely, this defines A_{1} and A_{2} as linear mappings from R^{n} into R^{n} and R^{m}, respectively. Conversely if A_{1} and A_{2} are linear mappings from R^{n} into R^{n} and R^{m}, respectively, then (9.7.1) defines a linear mapping from R^{n} into R^{n+m}.

Suppose now that (9.7.2) A_{1} is injective on R^{n},

which implies that (9.7.3) A_{1} is invertible on R^{n},

as before. Of course, (9.7.2) also implies that

(9.7.4) A is injective on R^{n}.

It can be shown that (9.7.4) implies a condition like (9.7.2), after rearranging the coordinates on R^{n+m} is a suitable way. We shall discuss this further at the end of the section.

Note that A_{2} ∘ A_{1}^{−1} (9.7.5)

defines a linear mapping from R^{n} into R^{m}. Put

B(v) = (v, A_{2}(A_{1}^{−1}(v))) (9.7.6)

for every v ∈ R^{n}, which defines a linear mapping from R^{n} into R^{n+m}. Thus

A = B ∘ A_{1}, (9.7.7)

as a mapping from R^{n} into R^{n+m}.

Let W be an open subset of R^{n}, and let f be a mapping from W into R^{n+m}. If w ∈ W, then f(w) ∈ R^{n+m} can be expressed as

f(w) = (f_{1}(w), f_{2}(w)), (9.7.8)

where f_{1}(w) ∈ R^{n} and f_{2}(w) ∈ R^{m}, as before. This defines f_{1} and f_{2} as mappings from W into R^{n} and R^{m}, respectively. Conversely, if f_{1} and f_{2} are mappings from W into R^{n} and R^{m}, respectively, then (9.7.8) defines f as a mapping from W into R^{n+m}.

It is easy to see that f is differentiable at a point w ∈ W if and only if

(9.7.9) f_{1} and f_{2} are differentiable at w.

In this case, f′(w)(v) = (f′_{1}(w)(v), f′_{2}(w)(v)) (9.7.10)

for every v ∈ R^{n}. Equivalently, this means that A = f′(w) corresponds to A_{1} = f′_{1}(w) and A_{2} = f′_{2}(w) as in (9.7.1).

Similarly, one can check that f is continuously-differentiable as a mapping from W into R^{n+m} if and only if

(9.7.11) f_{1} and f_{2} are continuously-differentiable

as mappings from W into R^{n} and R^{m}, respectively. In this situation, suppose that w ∈ W has the property that

(9.7.12) f′_{1}(w) is invertible

as a linear mapping from R^{n} into itself. Thus the inverse function theorem implies that the restriction of f_{1} to a suitable neighborhood U of w is a oneto-one mapping onto a neighborhood V of f_{1}(w), and that the corresponding inverse mapping has some additional nice properties.

This can be used to obtain more information about the behavior of f near w. Of course, this is all much simpler when f_{1} is the identity mapping on R^{n}. Otherwise, one can try to reduce to this case, at least locally near w. If f_{1} has a local inverse near w, as in the preceding paragraph, then f can be expressed near w as the composition of f_{1} with a simpler mapping into R^{n+m}.

Let e_{1}, …, e_{n} be the standard basis vectors in R^{n}, and let u_{1}, …, u_{n+m} be the standard basis vectors for R^{n+m}. If (9.7.4) holds, then

(9.7.13) A(e_{1}), …, A(e_{n}) are linearly independent in R^{n+m}.

In this case, it is well known that there is a set L ⊆ {1, …, n + m} with exactly m elements such that

(9.7.14) A(e_{1}), …, A(e_{n}) together with u_{k}, k ∈ L, is a basis for R^{n+m}.

If L = {n + 1, …, n + m}, then it is easy to see that

A_{1}(R^{n}) = R^{n}, (9.7.15)

so that (9.7.3) holds. Otherwise, one can get an analogous condition with the coordinates on R^{n+m} rearranged.

9.8 Ranks of linear mappings

In this section, we suppose that the reader has some familiarity with linear algebra on Euclidean spaces. Let m and n be arbitrary positive integers, and let A be a linear mapping from R^{n} into R^{m}. It is easy to see that the image A(R^{n}) of R^{n} under A is a linear subspace of R^{m}. The rank of A is defined to be (9.8.1) the dimension of A(R^{n}).

Thus the rank of A is a nonnegative integer less than or equal to m.

Of course, the rank of A is equal to 0 exactly when A = 0 on R^{n}. The rank of A is equal to m exactly when A(R^{n}) = R^{m}.

Suppose that A has rank r ≥ 1, and let w_{1}, …, w_{r} be a basis for A(R^{n}). If j ∈ {1, …, r}, then choose v_{j} ∈ R^{n} such that

A(v_{j}) = w_{j}. (9.8.2)

Observe that (9.8.3) v_{1}, …, v_{r} are linearly independent in R^{n},

because w_{1}, …, w_{r} are linearly independent in R^{m}. Let V be the linear span of v_{1}, …, v_{r} in R^{n}, so that

(9.8.4) V is a linear subspace of R^{n} of dimension r.

In particular, r ≤ n (9.8.5)

automatically. By construction, A(V) = A(R^{n}). One can check that

(9.8.6) the restriction of A to V is one-to-one

in this situation.

Alternatively, let u_{1}, …, u_{n} be any basis for R^{n}, such as the standard basis. Thus (9.8.7) A(R^{n}) is spanned by A(u_{1}), …, A(u_{n}).

It is well known that a subset of the vectors A(u_{1}), …, A(u_{n}) forms a basis for A(R^{n}). Such a subset has exactly r elements, by hypothesis. The corresponding subset of the vectors u_{1}, …, u_{n} can be used as a basis for a linear subspace V of R^{n} as in the preceding paragraph.

Now let V_{0} be a linear subspace of R^{n}, and suppose that

(9.8.8) the restriction of A to V_{0} is one-to-one.

The dimension of V_{0} is equal to the dimension of A(V_{0}), which is less than or equal to the dimension of A(R^{n}). Equivalently, the dimension of V_{0} is less than or equal to r. The remarks in the previous paragraphs show that it is always possible to choose V_{0} so that its dimension is equal to r. This shows that

(9.8.9) the rank of A is equal to the maximum of the dimensions of the linear subspaces V_{0} of R^{n} on which A is one-to-one.

If A is one-to-one on R^{n}, then the rank of A is equal to n. Conversely, if the rank of A is equal to n, then A is one-to-one on R^{n}.

Let V_{0} be a linear subspace of R^{n} on which A is one-to-one again, and let B be another linear mapping from R^{n} into R^{m}. If B is sufficiently close to A, as linear mappings from R^{n} into R^{m}, then

(9.8.10) the restriction of B to V_{0} is one-to-one as well. This can be obtained from the remarks in Section 7.11. It follows that

(9.8.11) the rank of B is greater than or equal to the rank of A

when B is sufficiently close to A. Note that the rank of B may be greater than the rank of A in this case.

Let f be a continuously-differentiable mapping from an open subset W of R^{n} into R^{m}, and let w be an element of W. Suppose that if x ∈ W is sufficiently close to w, then

(9.8.12) the rank of f′(x) is equal to the rank of f′(w),

as linear mappings from R^{n} into R^{m}. The rank theorem describes the behavior of f near w in this situation, and we shall return to this in Section 9.12.

9.9 Some remarks about determinants

Let n be a positive integer, and let [a_{j,l}] be an n × n matrix of real or complex numbers. The determinant det[a_{j,l}] of [a_{j,l}] can be defined as a real or complex number, as appropriate, in a standard way, as mentioned in Section 7.6. More precisely, det[a_{j,l}] is a polynomial of degree n in the entries a_{j,l}. In particular,

(9.9.1) det[a_{j,l}] is continuous

as a real or complex-valued function on the spaces M_{n,n}(R), M_{n,n}(C) of n × n matrices with entries in R, C, respectively, with respect to the metric associated to the Hilbert–Schmidt norm.

Now let A be a linear mapping from R^{n} or C^{n} into itself. Remember that A corresponds to an n×n matrix of real or complex numbers, as appropriate, as in Section 7.2. The determinant det A of A is defined to be the determinant of the corresponding matrix, as in Section 7.6 again. This defines continuous real and complex-valued functions on the spaces L(R^{n}), L(C^{n}) of linear mappings from R^{n}, C^{n} into themselves, respectively, with respect to the metrics associated to the corresponding Hilbert–Schmidt norms. One could also use the metrics associated to the operator norms corresponding to the standard Euclidean norms on R^{n}, C^{n}, or to any other norms on R^{n}, C^{n}.

It is well known that a linear mapping A from R^{n} or C^{n} into itself is invertible if and only if det A ≠ 0. (9.9.2)

The fact that the sets of invertible elements of L(R^{n}), L(C^{n}) are open sets with respect to the metrics associated to the Hilbert–Schmidt or operator norms can be obtained from this, and the continuity of the determinant.

If A is an invertible linear mapping on R^{n} or C^{n}, then it is well known that the matrix corresponding to the inverse of A can be obtained from the determinant of A and the determinants of (n − 1) × (n − 1) submatrices of the matrix corresponding to A, as in Cramer’s rule. This can be used to get the continuity of A ↦ A^{−1} (9.9.3)

on the spaces of invertible elements of L(R^{n}) or L(C^{n}), with respect to the restrictions of the metrics associated to the Hilbert–Schmidt or operator norms to these spaces.

9.10 Some remarks about linear subspaces

In this section, we suppose again that the reader has some familiarity with linear algebra on Euclidean spaces, although some basic notions will also be reviewed here. In particular, if n is a positive integer, then a linear subspace of R^{n} is a subset V of R^{n} such that u + v ∈ V (9.10.1)

for every u, v ∈ V, and t v ∈ V (9.10.2)

for every v ∈ R^{n} and t ∈ R. If V ≠ {0}, then it is well known that V can be spanned by finitely many of its elements. The smallest number of elements of V that span V is known as the dimension of V, and is denoted dim V. This is a nonnegative integer less than or equal to n, which is interpreted as being equal to 0 when V = {0}.

If V, W be linear subspaces of R^{n}, then their sum may be defined as the subset of R^{n} given by

V + W = {v + w : v ∈ V, w ∈ W }. (9.10.3)

It is easy to see that this is a linear subspace of R^{n} too. Note that

(9.10.4) dim(V + W) ≤ dim V + dim W.

It is easy to see that the intersection of V and W is a linear subspace of R^{n} as well. If V ∩ W = {0}, (9.10.5)

then V and W may be said to be transverse as linear subspaces of R^{n}. This means that the elements of V + W can be expressed in a unique way as v + w, with v ∈ V and w ∈ W. In this case,

(9.10.6) dim(V + W) = dim V + dim W,

because one can combine bases for V and W to get a basis for V +W. Conversely, it is not too difficult to show that (9.10.6) implies (9.10.5).

Let m be another positive integer, and let A be a linear mapping from R^{n} into R^{m}. The kernel or null space is defined by

ker A = {v ∈ R^{n} : A(v) = 0}. (9.10.7)

160 CHAPTER 9. MORE ON DIFFERENTIABLE MAPPINGS

It is easy to see that this is also a linear subspace of R^{n}. One can check that ker A = {0} (9.10.8) if and only if A is one-to-one on R^{n}. Of course, A(R^{m}) is a linear subspace of R^{m}, as mentioned in Section 9.8.

Suppose that A ≠ 0, so that the dimension r of A(R^{n}) is positive. One can choose v_{1}, …, v_{r} ∈ R^{n} such that (9.10.9) A(v_{1}), …, A(v_{r}) is a basis for A(R^{n}), as in Section 9.8. In this case, v_{1}, …, v_{r} are linearly independent in R^{n}, as before. If V is the linear span of v_{1}, …, v_{r} in R^{n}, then one can check that V + ker A = R^{n} (9.10.10) and V ∩ (ker A) = {0}. (9.10.11) This implies the well-known fact that (9.10.12) dim ker A + dim A(R^{n}) = n, because dim A(R^{n}) = r = dim V.

9.11 Complementary linear subspaces

Let n be a positive integer, and let V, W be linear subspaces of R^{n}. We say that V and W are complementary linear subspaces of R^{n} if V + W = R^{n} (9.11.1) and (9.10.5) holds. This means that every element of R^{n} can be expressed in a unique way as a sum of elements of V and W, as before. Note that dim V + dim W = n (9.11.2) in this case. If V and W are any two linear subspaces of R^{n} that satisfy (9.11.2), then one can check that (9.10.5) is equivalent to (9.11.1).

If V is any linear subspace of R^{n}, then one can find a linear subspace W of R^{n} such that (9.11.3) V and W are complementary linear subspaces of R^{n}. One way to do this is to start with a basis for V, and extend it to a basis for R^{n}. In this case, one can check that the linear span of the additional basis elements is complementary to V in R^{n}.

A linear mapping P from R^{n} into itself is said to be a projection if P ∘ P = P (9.11.4)

9.12. THE RANK THEOREM 161

on R^{n}. In this case, one can check that (9.11.5) ker P and P (R^{n}) are complementary linear subspaces of R^{n}. One can also verify that (9.11.6) I − P is a projection on R^{n} as well. In fact, ker(I − P) = P (R^{n}) (9.11.7) and (I − P)(R^{n}) = ker P. (9.11.8)

If V and W are comlementary linear subspaces of R^{n}, then there is a unique projection P on R^{n} such that ker P = V (9.11.9) and P (R^{n}) = W. (9.11.10) More precisely, P (v + w) = w (9.11.11) for every v ∈ V and w ∈ W. This can be used to define P on R^{n}, because every element of R^{n} can be expressed in a unique way as a sum of elements of V and W, as before.

9.12 The rank theorem

Let m, n, and r be positive integers with r ≤ min(m, n). (9.12.1) Also let W be a nonempty open subset of R^{n}, and let F be a continuouslydifferentiable mapping from W into R^{m}. Suppose that (9.12.2) the rank of F′(w) is equal to r for every w ∈ W.

Let a ∈ W be given, and put A = F′(a). (9.12.3) Also put Y_{1} = A(R^{n}), (9.12.4) which is a linear subspace of R^{m} of dimension r. Let Y_{2} be a linear subspace of R^{m} such that (9.12.5) Y_{1} and Y_{2} are complementary linear subspaces of R^{m}, as in the previous section. This leads to a projection P on R^{m} such that

P (R^{m}) = Y_{1} (9.12.6)

and ker P = Y_{2}, (9.12.7) as before.

Under these conditions, there are open subsets U, V of R^{n} with the following properties. First, a ∈ U and U ⊆ W. (9.12.8) Second, there is a one-to-one mapping H from V onto U such that

(9.12.9) H is continuously differentiable on V, (9.12.10) H^{−1} is continuously differentiable on U,

and P (F (H(x))) = A(x) (9.12.11) for every x ∈ V. This is part of the rank theorem, as on p229 of [155]. More precisely, (9.12.11) corresponds to (71) on p230 of [155].

Of course, (9.12.2) implies that

(9.12.12) the rank of A = F′(a) is equal to r.

If w ∈ W is sufficiently close to a, then (9.12.12) implies that

(9.12.13) the rank of F′(w) is greater than or equal to r,

because F′(w) is close to F′(a), as in Section 9.8. Remember that the rank of any linear mapping from R^{n} into R^{m} is less than or equal to min(m, n). If r = min(m, n), then (9.12.12) implies that

(9.12.14) the rank of F′(w) is equal to r when w ∈ W is sufficiently close to a.

In this case, we can get (9.12.2) by replacing W with a sufficiently small open set in R^{n} that contains a.

If r = m, then (9.12.12) is the same as saying that

Y_{1} = A(R^{n}) = R^{m}. (9.12.15)

This implies that Y_{2} = {0}, so that P is the identity mapping on R^{m}. In this case, (9.12.11) means that F (H(x)) = A(x) (9.12.16) for every x ∈ V.

If r = m = n, then A = F′(a) is invertible on R^{n}, and the rank theorem is basically the same as the inverse function theorem. If r = m < n, then the rank theorem is closely related to the implicit function theorem.

If r = n, then (9.12.12) is the same as saying that

(9.12.17) A = F′(a) is injective on R^{n}.

If r = n < m, then the rank theorem is related to the remarks in Section 9.7.

9.13 Proving this part

Let us continue with the same notation and hypotheses as in the previous section. Note that dim Y_{1} = r, (9.13.1)

by (9.12.4) and (9.12.12). Let y_{1}, …, y_{r} be a basis for Y_{1}.

Choose z_{j} ∈ R^{n} for each j = 1, …, r so that

A(z_{j}) = y_{j}. (9.13.2)

Let T be the unique linear mapping from Y_{1} into R^{n} such that

T (y_{j}) = z_{j} (9.13.3)

for each j = 1, …, r. Thus A(T (y_{j})) = A(z_{j}) = y_{j} for each j = 1, …, r, so that

A(T (y)) = y (9.13.4)

for every y ∈ Y_{1}.

If w ∈ W, then put

G(w) = w + T (P (F (w) − A(w))), (9.13.5)

as in (69) on p229 of [155]. This defines a continuously-differentiable mapping from W into R^{n}. It is easy to see that

G′(a) = I_{R^{n}}, (9.13.6)

the identity mapping on R^{n}, because A = F′(a). Thus the inverse function theorem implies that there are open subsets U and V of R^{n} such that U satisfies (9.12.8), and the restriction of G to U is a one-to-one mapping from U onto V whose inverse is continuously differentiable too. Let H be the inverse of the restriction of G to U, which is a one-to-one mapping from V onto U that satisfies (9.12.9) and (9.12.10).

Note that (9.13.7) H′(x) is invertible for every x ∈ V,

as in Section 9.3. We may also suppose that

V is convex, (9.13.8)

by replacing it with a convex open subset that contains G(a), if necessary, and adjusting U appropriately.

Let us check that A ∘ T ∘ P ∘ A = A, (9.13.9)

as a linear mapping from R^{n} into R^{m}. More precisely,

P ∘ A = A, (9.13.10) as a linear mapping from R^{n} into Y_{1} = A(R^{n}), because P is a projection of R^{m} onto Y_{1}, as in the previous section. We also have that A ∘ T is the identity mapping on Y_{1}, as in (9.13.4). This implies (9.13.9).

If w ∈ W, then

A(G(w)) = A(w) + A(T (P (F (w) − A(w)))), (9.13.11)

by (9.13.5). This implies that

A(G(w)) = A(T (P (F (w)))), (9.13.12)

because of (9.13.9). It follows that

A(G(w)) = P (F (w)), (9.13.13)

as in (70) on p230 of [155]. This uses (9.13.4), and the fact that P maps R^{m} into Y_{1}.

It is easy to see that (9.12.11) follows from (9.13.13), by taking w = H(x), x ∈ V. Some more properties of F ∘ H will be discussed in the next section.

9.14 Some more properties of F ∘ H

We continue with the same notation and hypotheses as in the previous two sections. Put Φ(x) = F (H(x)) (9.14.1)

for each x ∈ V, so that Φ is a continuously-differentiable mapping from V into R^{m}. Of course, Φ′(x) = F′(H(x)) ∘ H′(x) (9.14.2)

for every x ∈ V, by the chain rule. This implies that

(9.14.3) the rank of Φ′(x) is equal to the rank of F′(H(x))

for every x ∈ V, because of (9.13.7). It follows that

(9.14.4) the rank of Φ′(x) is equal to r

for every x ∈ V, because of (9.12.2), as in (75) on p230 of [155].

Observe that P ∘ Φ = A (9.14.5)

on V, by (9.12.11). This implies that

P ∘ Φ′(x) = A (9.14.6)

for every x ∈ V, by the chain rule.

Let x ∈ V be given, and put

M = (Φ′(x))(R^{n}) (9.14.7)

9.14. SOME MORE PROPERTIES OF F ∘ H 165 which is a linear subspace of R^{m}. Note that dim M = r, (9.14.8) by (9.14.4). We also have that P (M) = A(R^{n}) = Y_{1}, (9.14.9) using (9.14.6) in the first step, and the definition (9.12.4) of Y_{1} in the second step. It follows that (9.14.10) P is one-to-one on M, because M and Y_{1} have the same dimension r.

Suppose that h ∈ R^{n} satisfies A(h) = 0. (9.14.11) This means that P ((Φ′(x))(h)) = 0, (9.14.12) because of (9.14.6). Using (9.14.10), we get that (Φ′(x))(h) = 0, (9.14.13) because h ∈ M.

Suppose that y ∈ V satisfies A(y) = A(x). (9.14.14) We would like to show that Φ(x) = Φ(y), (9.14.15) which corresponds to (74) on p230 of [155]. If t ∈ R and 0 ≤ t ≤ 1, then (1 − t) x + t y ∈ V, (9.14.16) because of (9.13.8). In this case,

\frac{d}{dt}Φ((1 − t) x + t y) = (Φ′((1 − t) x + t y))(y − x) = 0, (9.14.17)

using (9.14.13) in the second step, and the fact that A(y − x) = 0, by (9.14.14). This implies (9.14.15).

Put ψ(x) = Φ(x) − A(x) = F (H(x)) − A(x) (9.14.18) for every x ∈ V, as in (72) on p230 of [155]. This defines a continuouslydifferentiable mapping from V into R^{m} with P (ψ(x)) = 0 (9.14.19)

166 CHAPTER 9. MORE ON DIFFERENTIABLE MAPPINGS for every x ∈ V, because of (9.12.11), or equivalently (9.14.5). This means that ψ(V) ⊆ ker P = Y_{2}, (9.14.20) using (9.12.7) in the second step. Note that ψ(x) = ψ(y) (9.14.21) for all x, y ∈ V that satisfy (9.14.14), because of (9.14.15), as in (74) on p230 of [155].

One may prefer to consider ψ(x) as a function of A(x) ∈ A(R^{n}) = Y_{1}, as in

Product spaces and related matters

10.1 Products of metric spaces

Let (X_{1}, d_{X_{1}}), …, (X_{n}, d_{X_{n}}) be finitely many metric spaces. Also let

X = X_{1} × X_{2} × ⋯ × X_{n} = ∏_{j=1}^{n} X_{j} (10.1.1)

be the Cartesian product of X_{1}, …, X_{n}, as sets. This is the set of n-tuples x = (x_{1}, …, x_{n}) with x_{j} ∈ X_{j} for each j = 1, …, n.

We would like to define suitable metrics on X using d_{X_{1}}, …, d_{X_{n}}, and indeed there are various ways to do this, as usual. If x, y ∈ X, then put

d_{X,1}(x, y) = ∑_{j=1}^{n} d_{X_{j}}(x_{j}, y_{j}). (10.1.2)

It is easy to see that this defines a metric on X.

Similarly, put

d_{X,∞}(x, y) = max_{1≤j≤n} d_{X_{j}}(x_{j}, y_{j}) (10.1.3)

for every x, y ∈ X. One can check that this defines a metric on X as well. More precisely, the triangle inequality for d_{X,∞} follows from the triangle inequality for the norm ∥ · ∥_{∞} on R^{n}, as in Section 1.3.

Put

d_{X,2}(x, y) = ( ∑_{j=1}^{n} d_{X_{j}}(x_{j}, y_{j})^{2})^{1/2} (10.1.4)

for each x, y ∈ X, using of course the nonnegative square root on the right side. One can verify that this satisfies the triangle inequality, using the triangle inequality for the standard Euclidean norm on R^{n}. It follows that d_{X,2} defines another metric on X.

If n = 1, then X = X_{1}, and d_{X,1}, d_{X,2}, and d_{X,∞} are all the same as d_{X_{1}}. Suppose for the moment that X_{j} = R for each j = 1, …, n, equipped with

the standard Euclidean metric. In this case, X is the same as R^{n}. The metrics d_{X,1}, d_{X,2}, and d_{X,∞} on X are the same as the metrics associated to the norms ∥ · ∥_{1}, ∥ · ∥_{2}, and ∥ · ∥_{∞} on R^{n}, respectively, as in Section 1.3.

Let (X_{j}, d_{X_{j}}) be any metric space for j = 1, …, n again. Observe that

d_{X,∞}(x, y) ≤ d_{X,1}(x, y), d_{X,2}(x, y) (10.1.5)

for every x, y ∈ X. We also have that

d_{X,2}(x, y) ≤ d_{X,1}(x, y) (10.1.6)

for every x, y ∈ X, because the standard Euclidean norm on R^{n} is less than or equal to the norm ∥ · ∥_{1}, as in Section 1.5.

Similarly, d_{X,2}(x, y) ≤ n^{1/2} d_{X,∞}(x, y) (10.1.7)

and d_{X,1}(x, y) ≤ n d_{X,∞}(x, y) (10.1.8)

for every x, y ∈ X. As before, one can use the Cauchy–Schwarz inequality to get that d_{X,1}(x, y) ≤ n^{1/2} d_{X,2}(x, y) (10.1.9)

for every x, y ∈ X.

One can use these inequalities to get that d_{X,1}, d_{X,2}, and d_{X,∞} have many of the same properties on X, in essentially the same way as for the analogous metrics on R^{n}. In particular, these metrics determine the same collections of open sets, closed sets, compact sets, and so on, in X. They also determine the same limit points of subsets of X, and the same convergent sequences and Cauchy sequences. Some of these and related properties will be discussed further in the next sections.

These metrics have many of the same properties in terms of continuity conditions for mappings between X and other metric spaces too. More precisely, the identity mapping on X is Lipschitz as a mapping from X equipped with any of these three metrics into X equipped with any other of these three metrics. This implies that the identity mapping on X is uniformly continuous and thus continuous as a mapping from X equipped with any of these three metrics into X equipped with any other of these three metrics.

10.2 Open and closed sets

Let (X_{1}, d_{X_{1}}), …, (X_{n}, d_{X_{n}}) be finitely many metric spaces again, put X = ∏_{j=1}^{n} X_{j}, and let d_{X,1}, d_{X,2}, and d_{X,∞} be the metrics defined on X as in the previous section. Let x = (x_{1}, …, x_{n}) ∈ X and r > 0 be given, and for each j = 1, …, n, let B_{X_{j},d_{X_{j}}} (x_{j}, r), \overline{B}_{X_{j},d_{X_{j}}} (x_{j}, r) (10.2.1)

be the open and closed balls in X_{j} centered at x_{j} with radius r with respect to d_{X_{j}}, respectively. Similarly, let

B_{X,d_{X,1}}(x, r), B_{X,d_{X,2}}(x, r), B_{X,d_{X,∞}}(x, r) (10.2.2)

and \overline{B}_{X,d_{X,1}}(x, r), \overline{B}_{X,d_{X,2}}(x, r), \overline{B}_{X,d_{X,∞}}(x, r) (10.2.3)

be the open and closed balls in X centered at x with radius r with respect to d_{X,1}, d_{X,2}, and d_{X,∞}, respectively.

It is easy to see that

B_{X,d_{X,∞}}(x, r) = ∏_{j=1}^{n} B_{X_{j},d_{X_{j}}} (x_{j}, r) (10.2.4)

and

\overline{B}_{X,d_{X,∞}}(x, r) = ∏_{j=1}^{n} \overline{B}_{X_{j},d_{X_{j}}} (x_{j}, r), (10.2.5)

by the definition (10.1.3) of d_{X,∞}. Using (10.1.5) and (10.1.8), we get that

B_{X,d_{X,1}}(x, r) ⊆ B_{X,d_{X,∞}}(x, r) ⊆ B_{X,d_{X,1}}(x, n r) (10.2.6)

and \overline{B}_{X,d_{X,1}}(x, r) ⊆ \overline{B}_{X,d_{X,∞}}(x, r) ⊆ \overline{B}_{X,d_{X,1}}(x, n r). (10.2.7)

Similarly, B_{X,d_{X,2}}(x, r) ⊆ B_{X,d_{X,∞}}(x, r) ⊆ B_{X,d_{X,2}}(x, n^{1/2} r) (10.2.8)

and \overline{B}_{X,d_{X,2}}(x, r) ⊆ \overline{B}_{X,d_{X,∞}}(x, r) ⊆ \overline{B}_{X,d_{X,2}}(x, n^{1/2} r), (10.2.9)

by (10.1.5) and (10.1.7). We also have that

B_{X,d_{X,1}}(x, r) ⊆ B_{X,d_{X,2}}(x, r) ⊆ B_{X,d_{X,1}}(x, n^{1/2} r) (10.2.10)

and \overline{B}_{X,d_{X,1}}(x, r) ⊆ \overline{B}_{X,d_{X,2}}(x, r) ⊆ \overline{B}_{X,d_{X,1}}(x, n^{1/2} r), (10.2.11)

by (10.1.6) and (10.1.9).

Let U_{j} be an open subset of X_{j} with respect to d_{X,j} for each j = 1, …, n. One can check that

U = ∏_{j=1}^{n} U_{j} (10.2.12)

is an open set in X with respect to d_{X,∞}, using (10.2.4). It follows that U is an open set with respect to d_{X,1} and d_{X,2} as well, by (10.2.6) and (10.2.8).

170 CHAPTER 10. PRODUCT SPACES AND RELATED MATTERS

Let A_{j} ⊆ X_{j} be given for each j = 1, …, n, and put

A = ∏_{j=1}^{n} A_{j}. (10.2.13)

One can verify that the closure of A in X with respect to d_{X,∞} is equal to

∏_{j=1}^{n} \overline{A_{j}}, (10.2.14)

where \overline{A_{j}} is the closure of A_{j} in X_{j} with respect to d_{X_{j}} for every j = 1, …, n. This is the same as the closure of A in X with respect to d_{X,1} and d_{X,2}. In particular, if (10.2.15) A_{j} is a closed set in X_{j} with respect to d_{X_{j}} for each j = 1, …, n, then (10.2.16) A is a closed set in X with respect to d_{X,1}, d_{X,2}, and d_{X,∞}.

10.3 Sequences and bounded sets

Let (X_{1}, d_{X_{1}}), …, (X_{n}, d_{X_{n}}) be finitely many nonempty metric spaces, put X = ∏_{j=1}^{n} X_{j}, and let d_{X,1}, d_{X,2}, and d_{X,∞} be the metrics defined on X as in Section 10.1. Also let {x(l)}_{l=1}^{∞} be a sequence of elements of X, so that x(l) = (x_{1}(l), …, x_{n}(l)) (10.3.1) for each l ≥ 1. One can check that (10.3.2) {x(l)}_{l=1}^{∞} converges to x = (x_{1}, …, x_{n}) ∈ X with respect to d_{X,∞} if and only if (10.3.3) {x_{j}(l)}_{l=1}^{∞} converges to x_{j} in X_{j} with respect to d_{X_{j}} for each j = 1, …, n. This is equivalent to the convergence of {x(l)}_{l=1}^{∞} to x in X with respect to d_{X,1}, and with respect to d_{X,2}, as well.

Similarly, (10.3.4) {x(l)}_{l=1}^{∞} is a Cauchy sequence in X with respect to d_{X,∞} if and only if (10.3.5) {x_{j}(l)}_{l=1}^{∞} is a Cauchy sequence in X_{j} with respect to d_{X_{j}} for each j = 1, …, n. This is equivalent to {x(l)}_{l=1}^{∞} being a Cauchy sequence in X with respect to d_{X,1}, and with respect to d_{X,2}. Observe that the completeness of X with respect to any of the metrics d_{X,1}, d_{X,2}, and d_{X,∞} are equivalent. The completeness of X with respect to these metrics is equivalent to the completeness of X_{j} with respect to d_{X_{j}} for each j = 1, …, n.

Note that the boundedness of any subset of X with respect to any of the metrics d_{X,1}, d_{X,2}, or d_{X,∞} implies the boundedness of the set with respect to the other two metrics. Let E_{j} be a nonempty subset of X_{j} for each j = 1, …, n, and put

E = ∏_{j=1}^{n} E_{j}. (10.3.6)

If (10.3.7) E_{j} is bounded in X_{j}

with respect to d_{X_{j}} for each j = 1, …, n, then it is easy to see that

(10.3.8) E is bounded in X

with respect to d_{X,1}, d_{X,2}, and d_{X,∞}. Conversely, if E is bounded in X with respect to d_{X,1}, d_{X,2}, or d_{X,∞}, then E_{j} is bounded in X_{j} with respect to d_{X_{j}} for each j = 1, …, n.

Suppose that E_{j} is bounded in X_{j} with respect to d_{X_{j}} for each j = 1, …, n, and let diam_{X_{j},d_{X_{j}}} E_{j} (10.3.9)

be the diameter of E_{j} as a subset of X_{j} with respect to d_{X_{j}} for every j = 1, …, n, as in Section 4.1. One can verify that the diameter of E as a subset of X with respect to d_{X,∞} is given by

(10.3.10) diam_{X,d_{X,∞}}E = max_{1≤j≤n} (diam_{X_{j},d_{X_{j}}} E_{j}).

Similarly, the diameter of E with respect to d_{X,1} is given by

(10.3.11) diam_{X,d_{X,1}}E = ∑_{j=1}^{n} diam_{X_{j},d_{X_{j}}} E_{j}.

The diameter of E with respect to d_{X,2} is given by

(10.3.12) diam_{X,d_{X,2}}E = ( ∑_{j=1}^{n} (diam_{X_{j},d_{X_{j}}} E)^{2})^{1/2}.

The total boundedness of any subset of X with respect to any of d_{X,1}, d_{X,2}, and d_{X,∞} implies the total boundedness of that set with respect to the other two metrics. If (10.3.13) E_{j} is totally bounded in X_{j}

with respect to d_{X_{j}} for each j = 1, …, n, then one can check that

(10.3.14) E is totally bounded in X with respect to d_{X,∞}.

More precisely, let r > 0 be given, and suppose that

(10.3.15) E_{j} can be covered by L_{j}(r) balls of radius r

in X_{j} for each j = 1, …, n. One can verify that E can be covered by

L(r) = ∏_{j=1}^{n} L_{j}(r) (10.3.16)

balls of radius r in X with respect to d_{X,∞}. This uses the products of the balls in the covering of X_{j} for each j.

This implies that E is totally bounded with respect to d_{X,1} and d_{X,2}, as before. Conversely, if E is totally bounded in X with respect to d_{X,1}, d_{X,2}, or d_{X,∞}, then E_{j} is totally bounded in X_{j} with respect to d_{X_{j}} for each j = 1, …, n.

10.4 Products of compact sets

Let (X_{1}, d_{X_{1}}), …, (X_{n}, d_{X_{n}}) be finitely many metric spaces, put X = ∏_{j=1}^{n} X_{j}, and let d_{X,1}, d_{X,2}, and d_{X,∞} be the metrics defined on X as in Section 10.1. If a subset of X is compact with respect to d_{X,1}, d_{X,2}, or d_{X,∞}, then it is compact with respect to the other two metrics. This follows from the fact that an open subset of X with respect to d_{X,1}, d_{X,2}, or d_{X,∞} is an open set with respect to the other two metrics.

Similarly, if a subset of X is sequentially compact with respect to any of the metrics d_{X,1}, d_{X,2}, or d_{X,∞}, then one can check directly that it is sequentially compact with respect to the other two metrics. This uses the fact that convergence of sequences of elements of X with respect to d_{X,1}, d_{X,2}, and d_{X,∞} are the same, as in the previous section.

Let K_{j} be a compact subset of X_{j} with respect to d_{X_{j}} for each j = 1, …, n, and put

K = ∏_{j=1}^{n} K_{j}. (10.4.1)

It is well known that (10.4.2) K is compact in X,

with respect to any of the metrics d_{X,1}, d_{X,2}, and d_{X,∞}. This may be considered as a particular case of a famous theorem of Tychonoff for arbitrary topological spaces, instead of metric spaces. Let us mention a couple of other ways to look at this.

If K_{j} is sequentially compact in X_{j} with respect to d_{X_{j}} for each j = 1, …, n, then (10.4.3) K is sequentially compact in X

with respect to any of the metrics d_{X,1}, d_{X,2}, and d_{X,∞}. This can be seen using an argument like one in Section 5.1, when the set E considered there is {1, …, n}. Remember that compactness and sequential compactness are equivalent in metric spaces, as in Sections 4.5 and 4.6.

Alternatively, if (10.4.4) K_{j} is compact in X_{j}

with respect to d_{X_{j}} for each j = 1, …, n, then

(10.4.5) K_{j} is closed and totally bounded in X_{j}

with respect to d_{X_{j}} for every j = 1, …, n. This implies that

(10.4.6) K is closed and totally bounded in X

with respect to each of d_{X,1}, d_{X,2}, and d_{X,∞}, as in the previous two sections. If X is complete with respect to d_{X,1}, d_{X,2}, or d_{X,∞}, then it follows that K is compact, as in Section 4.7. If

(10.4.7) X_{j} is complete with respect to d_{X_{j}}

for each j = 1, …, n, then X is complete with respect to each of d_{X,1}, d_{X,2}, and d_{X,∞}, as in the previous section.

Note that one can reduce to the case where K_{j} = X_{j} for each j = 1, …, n, by standard results. It is well known that compact metric spaces are complete, as in Section 4.7. Thus the argument mentioned in the preceding paragraph can be used when K_{j} = X_{j} for each j = 1, …, n.

If K_{j} ≠ ∅ for each j = 1, …, n, and K is compact in X with respect to d_{X,1}, d_{X,2}, or d_{X,∞}, then it is not too difficult to show that K_{j} is compact in X_{j} with respect to d_{X_{j}} for each j = 1, …, n.

10.5 Some mappings on products

Let (X_{1}, d_{X_{1}}), …, (X_{n}, d_{X_{n}}) be finitely many nonempty metric spaces, put X = ∏_{j=1}^{n} X_{j}, and let d_{X,1}, d_{X,2}, and d_{X,∞} be the metrics defined on X as in Section 10.1. If x = (x_{1}, …, x_{n}) ∈ X and 1 ≤ l ≤ n, then put

p_{l}(x) = x_{l}. (10.5.1)

This defines a mapping p_{l} from X onto X_{l}.

Clearly

(10.5.2) d_{X_{l}}(p_{l}(x), p_{l}(w)) = d_{X_{l}}(x_{l}, w_{l}) ≤ d_{X,∞}(x, w)
≤ d_{X,1}(x, w), d_{X,2}(x, w)

for every x, w ∈ X. This means that p_{l} is Lipschitz with constant 1 with respect to d_{X,∞} on X, and thus with respect to d_{X,1} and d_{X,2} on X. In particular, p_{l} is uniformly continuous on X with respect to d_{X,1}, d_{X,2}, and d_{X,∞}.

If x ∈ X and r > 0, then

p_{l}(B_{X,d_{X,∞}}(x, r)) = p_{l}( ∏_{j=1}^{n} B_{X_{j},d_{X_{j}}} (x_{j}, r)) = B_{X_{l},d_{X_{l}}} (x_{l}, r), (10.5.3)

using (10.2.4) in the first step. It follows that

p_{l}(B_{X,d_{X,1}}(x, r)) ⊇ p_{l}(B_{X,d_{X,∞}}(x, r/n)) = B_{X_{l},d_{X_{l}}} (x_{l}, r/n), (10.5.4)

using (10.2.6) in the first step. Similarly,

p_{l}(B_{X,d_{X,2}}(x, r)) ⊇ p_{l}(B_{X,d_{X,∞}}(x, r/n^{1/2})) = B_{X_{l},d_{X_{l}}} (x_{l}, r/n^{1/2}), (10.5.5)

using (10.2.8) in the first step. This shows that p_{l} is an open mapping from X onto X_{l}, with respect to d_{X,1}, d_{X,2}, or d_{X,∞} on X, as in Section 7.14.

Let (Y, d_{Y}) be another metric space, let E be a subset of X, and let f be a mapping from E into Y. Let us say that f is partially Lipschitz in the lth variable with constant C_{l} ≥ 0 on E if

d_{Y} (f(x), f(x′)) ≤ C_{l} d_{X_{l}}(x_{l}, x′_{l}) (10.5.6)

for every x, x′ ∈ E such that x_{j} = x′_{j} when j ≠ l. This is analogous to the condition discussed in Section 8.3 for functions defined on subsets of R^{n}.

Suppose that E = ∏_{j=1}^{n} E_{j} for some E_{j} ⊆ X_{j}, 1 ≤ j ≤ n, and that f is partially Lipschitz in the lth variable with constant C_{l} ≥ 0 on E for each l = 1, …, n. If x, w ∈ E, then

d_{Y} (f(x), f(w)) ≤ ∑_{l=1}^{n} C_{l} d_{X_{l}}(x_{l}, w_{l}), (10.5.7)

as in Section 8.3. This implies that

d_{Y} (f(x), f(w)) ≤ ( max_{1≤l≤n} C_{l}) d_{X,1}(x, w), (10.5.8)

as before. Similarly,

d_{Y} (f(x), f(w)) ≤ ( ∑_{l=1}^{n} C_{l}^{2})^{1/2} d_{X,2}(x, w) (10.5.9)

and

d_{Y} (f(x), f(w)) ≤ ( ∑_{l=1}^{n} C_{l}) d_{X,∞}(x, w). (10.5.10)

This shows that f is Lipschitz on E with respect to the restrictions of d_{X,1}, d_{X,2}, and d_{X,∞} to E under these conditions.

Now let f be any mapping from X into Y. If f is continuous at a point x ∈ X with respect to any of the metrics d_{X,1}, d_{X,2}, or d_{X,∞} on X, then it is easy to see that f is continuous at x with respect to the other two metrics. This is often called joint continuity of f at x.

It is often convenient to consider f as a function of n variables in X_{1}, …, X_{n}. If 1 ≤ l ≤ n, then we may consider f as a function of the lth variable on X_{l} with values in Y, with the jth variable equal to x_{j} when j ≠ l. If f is continuous as a function of the lth variable on X_{l} at x_{l} in this way for each l = 1, …, n, then f is said to be separately continuous at x. Note that joint continuity at x implies separate continuity.

10.6 Uniform continuity

Let (X, d_{X}) and (Y, d_{Y}) be metric spaces, let f be a mapping from X into Y, and let A be a subset of X. Let us say that f is uniformly continuous along A if for every ϵ > 0 there is a δ > 0 such that for every x ∈ A and w ∈ X with d_{X}(x, w) < δ, we have that

d_{Y} (f(x), f(w)) < ϵ. (10.6.1)

This implies that the restriction of f to A is uniformly continuous, with respect to the restriction of d_{X} to A. This also implies that f is continuous at every element of A, as a mapping from X into Y.

Suppose for the moment that A is a compact subset of X. If f is continuous at every element of A, as a function on X, then one can show that f is uniformly continuous along A. This uses the same type of arguments as used to show that continuous mappings on compact metric spaces are uniformly continuous.

Let (X_{1}, d_{X_{1}}), (X_{2}, d_{X_{2}}) be metric spaces, and let us now take

X = X_{1} × X_{2}. (10.6.2)

We can define the metrics d_{X,1}, d_{X,2}, and d_{X,∞} on X using d_{X_{1}} and d_{X_{2}} as in Section 10.1, with n = 2. If a mapping f from X into Y is uniformly continuous along a set A ⊆ X with respect to any of these three metrics on X, then it is easy to see that f is uniformly continuous along A with respect to the other two metrics.

Let x_{1} ∈ X_{1} be given, and consider

A = {x_{1}} × X_{2}. (10.6.3)

Let f be a mapping from X into Y again, and for each x_{2} ∈ X_{2}, let us consider f(·, x_{2}) as a mapping from X_{1} into Y. Let

E = {f(·, x_{2}) : x_{2} ∈ X_{2}} (10.6.4)

be the collection of these mappings from X_{1} into Y.

If f is uniformly continuous along A with respect to any of the metrics d_{X,1}, d_{X,2}, or d_{X,3} on X, then one can check that E is equicontinuous at x_{1}, as a collection of mappings from X_{1} into Y. As before, uniform continuity along A implies that the restriction of f to A is uniformly continuous, which in this case means that f(x_{1}, ·) is uniformly continuous as a mapping from X_{2} into Y.

Conversely, suppose that f(x_{1}, ·) is uniformly continuous as a mapping from X_{2} into Y, and that E is equicontinuous at x_{1} as a collection of mappings from X_{1} into Y. Under these conditions, one can verify that f is uniformly continuous along A with respect to d_{X,1}, d_{X,2}, and d_{X,∞} on X.

Observe that x_{2} ↦ (x_{1}, x_{2}) (10.6.5)

defines an isometry from X_{2} into X, with respect to d_{X,1}, d_{X,2}, and d_{X,∞} on X. In particular, if X_{2} is compact, then it follows that A is compact in X, with respect to d_{X,1}, d_{X,2}, and d_{X,∞}. In this case, if f is continuous at every point in A as a mapping from X into Y, then f is uniformly continuous along A, as before.

10.7 Continuity and integration

Let (X_{1}, d_{X_{1}}) be a metric space, and let a, b be real numbers, with a < b. Let us take X_{2} = [a, b], (10.7.1)

and d_{X_{2}} to be the restriction of the standard Euclidean metric on R to X_{2}. As in the previous section, we take

X = X_{1} × X_{2} = X_{1} × [a, b], (10.7.2)

and d_{X,1}, d_{X,2}, and d_{X,∞} to be the metrics defined on X as in Section 10.1, with n = 2.

Let f be a continuous real-valued function on X, with respect to any of d_{X,1}, d_{X,2}, or d_{X,∞}, and thus with respect to each of these metrics. If x_{1} ∈ X_{1}, then f(x_{1}, ·) is a continuous real-valued function on X_{2}, and we put

F (x_{1}) = ∫_{a}^{b} f(x_{1}, x_{2}) dx_{2}. (10.7.3)

This defines a real-valued function on X_{1}. Note that

|F (x_{1})| ≤ ∫_{a}^{b} |f(x_{1}, x_{2})| dx_{2} ≤ (b − a) sup_{a≤x_{2}≤b} |f(x_{1}, x_{2})|. (10.7.4)

More precisely, the supremum on the right is finite, because [a, b] is compact as a subset of the real line, and thus as a subset of itself, with respect to the standard Euclidean metric on R and its restriction to [a, b].

If x_{1} ∈ X_{1}, then {x_{1}} × [a, b] = {x_{1}} × X_{2} (10.7.5)

is compact in X, with respect to d_{X,1}, d_{X,2}, and d_{X,∞}, as in the previous section. This implies that f is uniformly continuous along (10.7.5), as before. This means that E = {f(·, x_{2}) : a ≤ x_{2} ≤ b} (10.7.6) is equicontinuous at x_{1} as a collection of real-valued functions on X_{1}, as in the previous section.

One can use this to check that F is continuous at x_{1}, as a real-valued function on X_{1}. Indeed, if w_{1} ∈ X_{1}, then

(10.7.7) |F (x_{1}) − F (w_{1})| = |∫_{a}^{b}(f(x_{1}, x_{2}) − f(w_{1}, x_{2})) dx_{2}|
≤ ∫_{a}^{b} |f(x_{1}, x_{2}) − f(w_{1}, x_{2})| dx_{2}
≤ (b − a) sup_{a≤x_{2}≤b} |f(x_{1}, x_{2}) − f(w_{1}, x_{2})|,

as before. The right side can be made arbitrarily small by taking w_{1} sufficiently close to x_{1} in X_{1}, because of the equicontinuity of E at x_{1}.

Alternatively, let {x_{1,j}}_{j=1}^{∞} be a sequence of elements of X_{1} that converges to x_{1}. It is easy to see that f(x_{1,j}, ·) converges to f(x_{1}, ·) uniformly as a sequence of real-valued functions on [a, b], because of the equicontinuity of E at x_{1}. This implies that

lim_{j→∞} F (x_{1,j}) = F (x_{1}). (10.7.8)

It follows that F is continuous at x_{1}.

If E is uniformly equicontinuous on X_{1}, then it is easy to see that F is uniformly continuous on X_{1}, using (10.7.7). In particular, this holds when X_{1} is compact, because E is equicontinuous at every point in X_{1}, as in Section 5.2.

If X_{1} is compact, then X is compact with respect to any of the metrics d_{X,1}, d_{X,2}, and d_{X,∞}, as in Section 10.4, because X_{2} is compact. In this case, f is uniformly continuous with respect to any of these three metrics on X. Of course, uniform continuity of f on X with respect to any of these three metrics implies that E is uniformly equicontinuous on X_{1}.

10.8 Iterated integrals

Let n be a positive integer, and let a_{1}, …, a_{n} and b_{1}, …, b_{n} be real numbers, with a_{j} < b_{j} (10.8.1)

for each j = 1, …, n. Let us consider

X_{j} = [a_{j}, b_{j}] (10.8.2)

as a metric space for each j = 1, …, n, with d_{X_{j}} equal to the restriction of the standard Euclidean metric on R to X_{j}.

Put

X^{l} = ∏_{j=1}^{l} X_{j} (10.8.3)

for each l = 1, …, n. We can define metrics d_{X^{l},1}, d_{X^{l},2}, and d_{X^{l},∞} on X^{l}, as in Section 10.1. We shall refer to functions on X^{l} as being continuous if they are continuous with respect to any of these three metrics, and thus with respect to the other two metrics.

Of course, X^{l} is a subset of R^{l} for each l = 1, …, n. Note that d_{X^{l},1}, d_{X^{l},2}, and d_{X^{l},∞} are the same as the restrictions to X^{l} of the metrics on R^{l} associated to the norms ∥ · ∥_{1}, ∥ · ∥_{2}, and ∥ · ∥_{∞}, respectively, as in Section 1.3.

Let f be a continuous real-valued function on X^{n}, and put f_{n} = f. Suppose that f_{l} has been defined as a continuous real-valued function on X^{l} for some l = 1, …, n. If l ≥ 2 and x_{j} ∈ X_{j} for j = 1, …, l − 1, then put

f_{l−1}(x_{1}, …, x_{l−1}) = ∫_{a_{l}}^{b_{l}} f_{l}(x_{1}, …, x_{l−1}, x_{l}) dx_{l}. (10.8.4)

This defines f_{l−1} as a continuous real-valued function on X^{l−1}, as in the previous section. More precisely, this uses the obvious identification of X^{l} with X^{l−1}×X_{l}.

Continuing in this way, we define f_{l} on X^{l} for each l = 1, …, n. Similarly, put

f_{0} = ∫_{a_{1}}^{b_{1}} f_{1}(x_{1}) dx_{1}, (10.8.5)

which is the analogue of (10.8.4) with l = 1, and which is simply a real number. This can be used to define the n-fold iterated integral of f over X^{n}, as on p246 of [155].

Alternatively, one can define the integral of f over X^{n} as an n-dimensional Riemann integral. Any reasonable approach to this will give the same answer, because f is uniformly continuous on X^{n}, since X^{n} is compact.

Of course, one could also define n-fold iterated integrals of f over X^{n} by integrating the variables in a different order, and one would like to verify that this leads to the same result. One way to do this is to show that the iterated integrals are all the same as the corresponding n-dimensional Riemann integral. Another approach is given in Theorem 10.2 on p246 of [155].

10.9 Partitions of intervals

Let a, b be real numbers, with a < b. Suppose that P = {t_{j}}_{j=0}^{k} is a partition of [a, b], which is to say a finite sequence of real numbers such that

a = t_{0} < t_{1} < ⋯ < t_{k−1} < t_{k} = b. (10.9.1)

A real-valued function on [a, b] is said to be piecewise linear with breakpoints in P if the function is linear on [t_{j−1}, t_{j}] for each j = 1, …, k. It is easy to see that such a function is continuous, with respect to the standard Euclidean metric on R, and its restriction to [a, b]. Note that such a function is uniquely determined by its values at the points in P. The values of such a function at the points in P can be arbitrary real numbers. This is because linear functions 10.9. PARTITIONS OF INTERVALS 179 on R are uniquely determined by their values at any two distinct points, and those values can be arbitrary real numbers.

If f is any real-valued function on [a, b], then we let A_{P}(f) be the unique piecewise-linear function on [a, b] with breakpoints in P that is equal to f at the points in P. If x ∈ [a, b] and t_{j−1} ≤ x ≤ t_{j} for some j = 1, …, k, then

(10.9.2)  (A_{P}(f))(x) = f(t_{j−1}) (t_{j} − t_{j−1})^{−1} (t_{j} − x)
+f(t_{j}) (t_{j} − t_{j−1})^{−1} (x − t_{j−1}).

In this case,

f(x) − (A_{P}(f))(x) = (f(x) − f(t_{j−1})) (t_{j} − t_{j−1})^{−1} (t_{j} − x)

+(f(x) − f(t_{j})) (t_{j} − t_{j−1})^{−1} (x − t_{j−1}). (10.9.3) This implies that

|f(x) − (A_{P}(f))(x)| ≤ |f(x) − f(t_{j−1})| (t_{j} − t_{j−1})^{−1} (t_{j} − x)

+|f(x) − f(t_{j})| (t_{j} − t_{j−1})^{−1} (x − t_{j−1}). (10.9.4) It follows that |f(x) − (A_{P}(f))(x)| ≤ max(|f(x) − f(t_{j−1})|, |f(x) − f(t_{j})|), (10.9.5) because (t_{j} − t_{j−1})^{−1} (t_{j} − x) + (t_{j} − t_{j−1})^{−1} (x − t_{j−1}) (10.9.6)

= (t_{j} − t_{j−1})^{−1} (t_{j} − t_{j−1}) = 1,

where both terms on the first line are greater than or equal to 0.

Observe that

(10.9.7) ∫_{a}^{b}(A_{P}(f))(x) dx = ∑_{j=1}^{k} ∫_{t_{j−1}}^{t_{j}}(A_{P}(f))(x) dx
= ∑_{j=1}^{k}(1/2) (f(t_{j−1}) + f(t_{j})) (t_{j} − t_{j−1}).

Suppose that f is continuous on [a, b], with respect to the standard Euclidean metric on R and its restriction to [a, b]. Under these conditions, we have that

|∫_{a}^{b} f(x) dx − ∫_{a}^{b}(A_{P}(f))(x) dx| = |∫_{a}^{b}(f(x) − (A_{P}(f))(x)) dx|
(10.9.8)  ≤ ∫_{a}^{b} |f(x) − (A_{P}(f))(x)| dx
≤ (b − a) sup_{a≤x≤b} |f(x) − (A_{P}(f))(x)|.

Note that f is uniformly continuous on [a, b], because [a, b] is compact. Using this and (10.9.5), we get that f is uniformly approximated by A_{P}(f) on [a, b] when the partition P of [a, b] is sufficiently fine. More precisely, this means that

sup_{a≤x≤b} |f(x) − (A_{P}(f))(x)| (10.9.9)

is as small as we want when

max_{1≤j≤k}(t_{j} − t_{j−1}) (10.9.10)

is sufficiently small. It follows that the left side of (10.9.8) is as small as we want when (10.9.10) is sufficiently small. Similarly, one can check that the left side of (10.9.8) is arbitrarily small for suitable partitions P of [a, b] when f is Riemann integrable on [a, b].

10.10 Partitions and products

Let (X_{1}, d_{X_{1}}) be a metric space, and let a_{2}, b_{2} be real numbers, with a_{2} < b_{2}. Let us take X_{2} = [a_{2}, b_{2}], (10.10.1)

and d_{X_{2}} to be the restriction of the standard Euclidean metric on R to X_{2}. We also take X = X_{1} × X_{2} = X_{1} × [a_{2}, b_{2}], (10.10.2)

and d_{X,1}, d_{X,2}, and d_{X,∞} to be the metrics defined on X as in Section 10.1, with n = 2. We shall refer to functions on X as being continuous if they are continuous with respect to any of these three metrics, and thus with respect to the other two metrics, as before.

Let P_{2} = {t_{2,j}}_{j=0}^{k} be a partition of [a_{2}, b_{2}], so that

a_{2} = t_{2,0} < ⋯ < t_{2,k} = b_{2}. (10.10.3)

Let f be a continuous real-valued function on X, and let us define A_{2,P_{2}}(f) as a real-valued function on X in essentially the same way as in the previous section, as a function of the second variable. Thus, if x_{1} ∈ X_{1}, x_{2} ∈ [a_{2}, b_{2}], and t_{2,j−1} ≤ x_{2} ≤ t_{2,j} for some j = 1, …, k, then we put

(A_{2,P_{2}}(f))(x_{1}, x_{2}) = f(x_{1}, t_{2,j−1}) (t_{2,j} − t_{2,j−1})^{−1} (t_{2,j} − x_{2})

+f(x_{1}, t_{2,j}) (t_{2,j} − t_{2,j−1})^{−1} (x_{2} − t_{2,j−1}). (10.10.4)

Under these conditions, we get that

|f(x_{1}, x_{2}) − (A_{2,P_{2}}(f))(x_{1}, x_{2})|
(10.10.5) ≤ max(|f(x_{1}, x_{2}) − f(x_{1}, t_{2,j−1})|, |f(x_{1}, x_{2}) − f(x_{1}, t_{2,j})|),

as in (10.9.5).

As in Section 10.7,

∫_{a_{2}}^{b_{2}} f(x_{1}, x_{2}) dx_{2} (10.10.6)

defines a continuous real-valued function of x_{1} ∈ X_{1}. The analogous statement for A_{2,P_{2}}(f) can be seen more directly. Namely,

∫_{a_{2}}^{b_{2}}(A_{2,P_{2}}(f))(x_{1}, x_{2}) dx_{2} (10.10.7)

= ∑_{j=1}^{k}(1/2) (f(x_{1}, t_{2,j−1}) + f(x_{1}, t_{2,j})) (t_{2,j} − t_{2,j−1})

for every x_{1} ∈ X_{1}, as in (10.9.7). We also have that

|∫_{a_{2}}^{b_{2}} f(x_{1}, x_{2}) dx_{2} − ∫_{a_{2}}^{b_{2}}(A_{2,P_{2}}(f))(x_{1}, x_{2}) dx_{2}| (10.10.8)

≤ ∫_{a_{2}}^{b_{2}} |f(x_{1}, x_{2}) − (A_{2,P_{2}}(f))(x_{1}, x_{2})| dx_{2}
≤ (b_{2} − a_{2}) sup_{a_{2}≤x_{2}≤b_{2}} |f(x_{1}, x_{2}) − (A_{2,P_{2}}(f))(x_{1}, x_{2})|

for every x_{1} ∈ X_{1}, as in (10.9.8).

Suppose that E_{2} = {f(x_{1}, ·) : x_{1} ∈ X_{1}} (10.10.9)

is uniformly equicontinuous as a collection of real-valued functions on [a_{2}, b_{2}]. This implies that

sup_{a_{2}≤x_{2}≤b_{2}} |f(x_{1}, x_{2}) − (A_{2,P_{2}}(f))(x_{1}, x_{2})| (10.10.10)

is as small as we want, uniformly over x_{1} ∈ X_{1}, when

max_{1≤j≤k}(t_{2,j} − t_{2,j−1}) (10.10.11)

is sufficiently small, because of (10.10.5). It follows that (10.10.6) can be uniformly approximated by (10.10.7), as real-valued functions of x_{1} ∈ X_{1}, when (10.10.11) is sufficiently small, by (10.10.8).

If X_{1} is compact, then E_{2} is equicontinuous at every point in [a_{2}, b_{2}], as in Section 10.6. This implies that E_{2} is uniformly equicontinuous on [a_{2}, b_{2}], as in Section 5.2, because [a_{2}, b_{2}] is compact. Alternatively, if X_{1} is compact, then X is compact with respect to any of the metrics d_{X,1}, d_{X,2}, and d_{X,∞}, as in Section 10.4, because [a_{2}, b_{2}] is compact. This means that f is uniformly continuous on X, with respect to any of these three metrics. It is easy to see that this implies that E_{2} is uniformly equicontinuous on [a_{2}, b_{2}].

10.11 Partitions and integrals

Let a_{1}, b_{1} be real numbers, with a_{1} < b_{1}. We would like to continue with the same notation and hypotheses as in the previous section, with

X_{1} = [a_{1}, b_{1}], (10.11.1)

and with d_{X_{1}} equal to the restriction of the standard Euclidean metric on R to X_{1}. Thus X = X_{1} × X_{2} = [a_{1}, b_{1}] × [a_{2}, b_{2}], (10.11.2)

and d_{X,1}, d_{X,2}, and d_{X,∞} are the same as the restrictions to X of the metrics on R^{2} associated to the norms ∥·∥_{1}, ∥·∥_{2}, and ∥·∥_{∞}, respectively, as in Section 1.3. Note that f is uniformly continuous on X, because X is compact.

We can integrate f(x_{1}, x_{2}) in x_{1} to get a continuous real-valued function

∫_{a_{1}}^{b_{1}} f(x_{1}, x_{2}) dx_{1} (10.11.3)

of x_{2} on [a_{2}, b_{2}], as in Section 10.7. The analogous statement for A_{2,P_{2}}(f) can be seen more directly, as before. Indeed, if x_{2} ∈ [a_{2}, b_{2}] and t_{2,j−1} ≤ x_{2} ≤ t_{2,j} for some j = 1, …, k, then

∫_{a_{1}}^{b_{1}}(A_{2,P_{2}}(f))(x_{1}, x_{2}) dx_{1} (10.11.4)

= (t_{2,j} − t_{2,j−1})^{−1} (t_{2,j} − x_{2}) ∫_{a_{1}}^{b_{1}} f(x_{1}, t_{2,j−1}) dx_{1}

+(t_{2,j} − t_{2,j−1})^{−1} (x_{2} − t_{2,j−1}) ∫_{a_{1}}^{b_{1}} f(x_{1}, t_{2,j}) dx_{1},

by (10.10.4). Of course, this is equal to (10.11.3) when x_{2} = t_{2,j−1} or t_{2,j}. Observe that

|∫_{a_{1}}^{b_{1}} f(x_{1}, x_{2}) dx_{1} − ∫_{a_{1}}^{b_{1}}(A_{2,P_{2}}(f))(x_{1}, x_{2}) dx_{1}| (10.11.5)

= |∫_{a_{1}}^{b_{1}}(f(x_{1}, x_{2}) − (A_{2,P_{2}}(f))(x_{1}, x_{2})) dx_{1}|

≤ ∫_{a_{1}}^{b_{1}} |f(x_{1}, x_{2}) − (A_{2,P_{2}}(f))(x_{1}, x_{2})| dx_{1}
≤ (b_{1} − a_{1}) sup_{a_{1}≤x_{1}≤b_{1}} |f(x_{1}, x_{2}) − (A_{2,P_{2}}(f))(x_{1}, x_{2})|

for every x_{2} ∈ [a_{2}, b_{2}]. We also have that

sup_{a_{1}≤x_{1}≤b_{1}} |f(x_{1}, x_{2}) − (A_{2,P_{2}}(f))(x_{1}, x_{2})| (10.11.6)

is as small as we want, uniformly over x_{2} ∈ [a_{2}, b_{2}], when (10.10.11) is sufficiently small. Equivalently, this means that

sup_{a_{1}≤x_{1}≤b_{1}} sup_{a_{2}≤x_{2}≤b_{2}} |f(x_{1}, x_{2}) − (A_{2,P_{2}}(f))(x_{1}, x_{2})| (10.11.7)

is as small as we want when (10.10.11) is sufficiently small. This is the same as the analogous statement for (10.10.10), which follows from the uniform continuity of f on X, as before. Combining this with (10.11.5), we get that (10.11.3) can be uniformly approximated by (10.11.4), as real-valued functions of x_{2} ∈ [a_{2}, b_{2}], when (10.10.11) is sufficiently small.

One can check directly that

∫_{a_{1}}^{b_{1}} ( ∫_{a_{2}}^{b_{2}}(A_{2,P_{2}}(f))(x_{1}, x_{2}) dx_{2}) dx_{1} (10.11.8)

= ∫_{a_{2}}^{b_{2}} ( ∫_{a_{1}}^{b_{1}}(A_{2,P_{2}}(f))(x_{1}, x_{2}) dx_{1}) dx_{2}.

More precisely, one can verify that

∫_{a_{1}}^{b_{1}} ( ∫_{t_{2,j−1}}^{t_{2,j}}(A_{2,P_{2}}(f))(x_{1}, x_{2}) dx_{2}) dx_{1} (10.11.9)

= ∫_{t_{2,j−1}}^{t_{2,j}} ( ∫_{a_{1}}^{b_{1}}(A_{2,P_{2}}(f))(x_{1}, x_{2}) dx_{1})dx_{2}

for each j = 1, …, k, using the definition (10.10.4) of A_{2,P_{2}}(f). It is easy to see that (10.11.8) follows from (10.11.9), by summing over j.

We can use (10.11.8) to get that

∫_{a_{1}}^{b_{1}} ( ∫_{a_{2}}^{b_{2}} f(x_{1}, x_{2}) dx_{2}) dx_{1} = ∫_{a_{2}}^{b_{2}} ( ∫_{a_{1}}^{b_{1}} f(x_{1}, x_{2}) dx_{1}) dx_{2}, (10.11.10)

as follows. The left side of (10.11.10) can be approximated by the left side of (10.11.8) when (10.10.11) is sufficiently small, because (10.10.6) is uniformly approximated by (10.10.7), as before. Similarly, the right side of (10.11.10) can be approximated by the right side of (10.11.8) when (10.10.11) is sufficiently small, because (10.11.3) is uniformly approximated by (10.11.4). This implies (10.11.10), because we can take P_{2} to be a partition of [a_{2}, b_{2}] for which (10.10.11) is arbitarily small.

10.12 A simpler approximation

Let us continue with the same notation and hypotheses as in the previous two sections. Put

a_{2,P_{2}}(j) = sup_{a_{1}≤x_{1}≤b_{1}} sup_{t_{2,j−1}≤x_{2}≤t_{2,j}} |f(x_{1}, x_{2}) − f(x_{1}, t_{2,j})| (10.12.1)

for each j = 1, …, k, and

a_{2,P_{2}} = max_{1≤j≤k} a_{2,P_{2}}(j). (10.12.2)

It is easy to see that a_{2,P_{2}} is as small as we want when (10.10.11) is sufficiently small, because f is uniformly continuous on X.

Observe that

∫_{a_{1}}^{b_{1}} ( ∫_{t_{2,j−1}}^{t_{2,j}} f(x_{1}, x_{2}) dx_{2}) dx_{1} − (t_{2,j} − t_{2,j−1}) ∫_{a_{1}}^{b_{1}} f(x_{1}, t_{2,j}) dx_{1} (10.12.3)

= ∫_{a_{1}}^{b_{1}} ( ∫_{t_{2,j−1}}^{t_{2,j}}(f(x_{1}, x_{2}) − f(x_{1}, t_{2,j})) dx_{2}) dx_{1}

for each j = 1, …, k. This implies that

|∫_{a_{1}}^{b_{1}} ( ∫_{t_{2,j−1}}^{t_{2,j}} f(x_{1}, x_{2}) dx_{2}) dx_{1} − (t_{2,j} − t_{2,j−1}) ∫_{a_{1}}^{b_{1}} f(x_{1}, t_{2,j}) dx_{1}| (10.12.4)

≤ ∫_{a_{1}}^{b_{1}} ( ∫_{t_{2,j−1}}^{t_{2,j}} |f(x_{1}, x_{2}) − f(x_{1}, t_{2,j})| dx_{2}) dx_{1}
≤ (b_{1} − a_{1}) (t_{2,j} − t_{2,j−1}) a_{2,P_{2}}(j).

Similarly,

|∫_{t_{2,j−1}}^{t_{2,j}} ( ∫_{a_{1}}^{b_{1}} f(x_{1}, x_{2}) dx_{1}) dx_{2} − (t_{2,j} − t_{2,j−1}) ∫_{a_{1}}^{b_{1}} f(x_{1}, t_{2,j}) dx_{1}| (10.12.5)

≤ ∫_{t_{2,j−1}}^{t_{2,j}} ( ∫_{a_{1}}^{b_{1}} |f(x_{1}, x_{2}) − f(x_{1}, t_{2,j})| dx_{1}) dx_{2}
≤ (b_{1} − a_{1}) (t_{2,j} − t_{2,j−1}) a_{2,P_{2}}(j)

for each j = 1, …, k.

Using (10.12.3) and (10.12.5), we get that

|∫_{a_{1}}^{b_{1}} ( ∫_{t_{2,j−1}}^{t_{2,j}} f(x_{1}, x_{2}) dx_{2}) dx_{1} − ∫_{t_{2,j−1}}^{t_{2,j}} ( ∫_{a_{1}}^{b_{1}} f(x_{1}, x_{2}) dx_{1}) dx_{2}| (10.12.6)

≤ 2 (b_{1} − a_{1}) (t_{2,j} − t_{2,j−1}) a_{2,P_{2}}(j).

It follows that

|∫_{a_{1}}^{b_{1}} ( ∫_{a_{2}}^{b_{2}} f(x_{1}, x_{2}) dx_{2}) dx_{1} − ∫_{a_{2}}^{b_{2}} ( ∫_{a_{1}}^{b_{1}} f(x_{1}, x_{2}) dx_{1}) dx_{2}| (10.12.7)

≤ ∑_{j=1}^{k} 2 (b_{1} − a_{1}) (t_{2,j} − t_{2,j−1}) a_{2,P_{2}}(j)

≤ 2 (b_{1} − a_{1}) ∑_{j=1}^{k}(t_{2,j} − t_{2,j−1}) a_{2,P_{2}}
= 2 (b_{1} − a_{1}) (b_{2} − a_{2}) a_{2,P_{2}}.

The right side of (10.12.7) is as small as we want when (10.10.11) is sufficiently small, as before. This is another way to obtain (10.11.10), by taking P_{2} to be a partition of [a_{2}, b_{2}] for which (10.10.11) is arbitrarily small again.

10.13 Another approximation

Let us continue with the same notation and hypotheses as in the previous three sections. Also let P_{1} = {t_{1,l}}_{l=0}^{m} be a partition of [a_{1}, b_{1}], so that

a_{1} = t_{1,0} < ⋯ < t_{1,m} = b_{1}. (10.13.1)

Of course,

∫_{a_{1}}^{b_{1}} ( ∫_{a_{2}}^{b_{2}} f(x_{1}, x_{2}) dx_{2}) dx_{1} (10.13.2)

= ∑_{l=1}^{m} ∑_{j=1}^{k} ∫_{t_{1,l−1}}^{t_{1,l}} ( ∫_{t_{2,j−1}}^{t_{2,j}} f(x_{1}, x_{2}) dx_{2}) dx_{1}.

Similarly,

∫_{a_{2}}^{b_{2}} ( ∫_{a_{1}}^{b_{1}} f(x_{1}, x_{2}) dx_{1}) dx_{2} (10.13.3)

= ∑_{l=1}^{m} ∑_{j=1}^{k} ∫_{t_{2,j−1}}^{t_{2,j}} ( ∫_{t_{1,l−1}}^{t_{1,l}} f(x_{1}, x_{2}) dx_{1}) dx_{2}.

If 1 ≤ l ≤ m and 1 ≤ j ≤ k, then we can approximate

∫_{t_{1,l−1}}^{t_{1,l}} ( ∫_{t_{2,j−1}}^{t_{2,j}} f(x_{1}, x_{2}) dx_{2})dx_{1} (10.13.4)

and

∫_{t_{2,j−1}}^{t_{2,j}} ( ∫_{t_{1,l−1}}^{t_{1,l}} f(x_{1}, x_{2}) dx_{1}) dx_{2} (10.13.5)

by (t_{1,l} − t_{1,l−1}) (t_{2,j} − t_{2,j−1}) f(t_{1,l}, t_{2,j}). (10.13.6)

If t_{1,l} − t_{1,l−1} and t_{2,j} − t_{2,j−1} are sufficiently small, then the errors in these approximations will be small compared to

(t_{1,l} − t_{1,l−1}) (t_{2,j} − t_{2,j−1}), (10.13.7)

because of the uniform continuity of f on X. This means that the difference of (10.13.4) and (10.13.5) is small compared to (10.13.7) in this case.

Note that

∑_{l=1}^{m} ∑_{j=1}^{k}(t_{1,l} − t_{1,l−1}) (t_{2,j} − t_{2,j−1}) (10.13.8)

= ( ∑_{l=1}^{m}(t_{1,l} − t_{1,l−1})) ( ∑_{j=1}^{k}(t_{2,j} − t_{2,j−1}))
= (b_{1} − a_{1}) (b_{2} − a_{2}).

It follows that the difference of (10.13.2) and (10.13.3) is as small as we like when P_{1} and P_{2} are sufficiently fine partitions of [a_{1}, b_{1}] and [a_{2}, b_{2}], respectively. This implies that the iterated integrals are equal, as in (10.11.10), because they do not depend on P_{1} or P_{2}. Basically, we are approximating the iterated integrals on the left sides of (10.13.2) and (10.13.3) by two-dimensional Riemann sums, and these two-dimensional Riemann sums are equal to each other.

10.14 Partitions of unity

Let (X, d_{X}) be a metric space. As in Section 3.7, the support supp f of a real or complex-valued function f on X is defined to be the closure in X of the set of x ∈ X such that f(x) ≠ 0.

It is often helpful to be able to find finite collections of continuous real-valued functions ψ_{1}, …, ψ_{r} on X with properties like the following. First,

0 ≤ ψ_{j}(x) ≤ 1 (10.14.1)

for every j = 1, …, r. Second,

∑_{j=1}^{r} ψ_{j}(x) = 1 (10.14.2)

for all x in a particular subset of X, which may be X itself. If (10.14.2) holds on a proper subset of X, then one may ask that

∑_{j=1}^{r} ψ_{j}(x) ≤ 1 (10.14.3)

for every x ∈ X. Of course, the first inequality in (10.14.1) together with (10.14.3) implies the second inequality in (10.14.1).

In addition, one may want to have restrictions on the supports of the ψ_{j}’s. One may ask that the supports of the ψ_{j}’s be contained in open sets in a particular family, for instance. A collection of functions like this is called a partition of unity on the set where (10.14.2) holds.

Let a, b be real numbers with a < b, and let P = {t_{j}}_{j=0}^{k} be a partition of [a, b]. If 0 ≤ l ≤ r, then there is a unique nonnegative real-valued function τ_{l} on 10.14. PARTITIONS OF UNITY 187 [a, b] that is piecewise linear with breakpoints in P such that

(10.14.4)  τ_{l}(t_{j}) = 1 when j = l
= 0 when j ≠ l.

It is easy to see that

∑_{l=0}^{k} τ_{l}(x) = 1 (10.14.5)

for every x ∈ [a, b], because the left side is a piecewise-linear function on [a, b] with breakpoints in P that is equal to 1 at t_{j} for each j = 0, …, k. The support of τ_{l}, as a real-valued function on [a, b], is given by

(10.14.6) supp τ_{l} = [t_{0}, t_{1}] when l = 0
= [t_{l−1}, t_{l+1}] when 1 ≤ l ≤ k − 1
= [t_{k−1}, t_{k}] when l = k.

Let (X, d_{X}) be any metric space again, and let ϕ_{1}, …, ϕ_{r} be nonnegative continuous real-valued functions on X. There are a couple of common ways to get a partition of unity ψ_{1}, …, ψ_{r} from ϕ_{1}, …, ϕ_{r}, under suitable conditions.

In the first approach, we put

Φ(x) = ∑_{j=1}^{r} ϕ_{j}(x) (10.14.7)

for each x ∈ X, and suppose that (10.14.8) Φ(x) > 0 for every x ∈ X. In this case, ψ_{j}(x) = ϕ_{j}(x)/Φ(x) (10.14.9) defines a continuous real-valued function on X for each j = 1, …, r, and (10.14.2) holds for every x ∈ X, by construction. We also have that (10.14.10) supp ψ_{j} = supp ϕ_{j} for every j = 1, …, r.

In the second approach, we suppose that 0 ≤ ϕ_{j}(x) ≤ 1 (10.14.11) for every j = 1, …, r and x ∈ X. Put ψ_{1} = ϕ_{1}, and

ψ_{l} = ( ∏_{j=1}^{l−1}(1 − ϕ_{j})) ϕ_{l} (10.14.12)

for l = 2, …, r, as on p251 of [155]. Clearly (10.14.1) holds for every j = 1, …, r, and one can check that

∑_{j=1}^{l} ψ_{j} = 1 − ∏_{j=1}^{l}(1 − ϕ_{j}) (10.14.13)

for every l = 1, …, r, by induction. This implies that (10.14.3) holds for every x ∈ X, and that (10.14.2) holds for every x ∈ X such that

ϕ_{j}(x) = 1 for some j = 1, …, r. (10.14.14)

Note that (10.14.15) supp ψ_{j} ⊆ supp ϕ_{j}

for every j = 1, …, r.

10.15 Another approximation argument

Let (X_{1}, d_{X_{1}}), (X_{2}, d_{X_{2}}) be metric spaces, and put X = X_{1} × X_{2}. Also let d_{X,1}, d_{X,2} and d_{X,∞} be the corresponding metrics defined on X as in Section 10.1. As before, we shall refer to functions on X as being continuous if they are continuous with respect to any of these three metrics, and thus with respect to the other two metrics.

Let ψ_{2,1}, …, ψ_{2,r} be finitely many continuous nonnegative real-valued functions on X_{2} that form a partition of unity on X_{2}, so that

∑_{j=1}^{r} ψ_{2,j}(x_{2}) = 1 (10.15.1)

for every x_{2} ∈ X_{2}. If f is a real-valued function on X, then

f(x_{1}, x_{2}) = ∑_{j=1}^{r} f(x_{1}, x_{2}) ψ_{2,j}(x_{2}) (10.15.2)

for every x_{1} ∈ X_{1} and x_{2} ∈ X_{2}.

Suppose that for each j = 1, …, r, x_{2,j} ∈ X_{2} and

ψ_{2,j}(x_{2,j}) > 0. (10.15.3)

Put

(A_{2}(f))(x_{1}, x_{2}) = ∑_{j=1}^{r} f(x_{1}, x_{2,j}) ψ_{2,j}(x_{2}) (10.15.4)

for every x_{1} ∈ X_{1} and x_{2} ∈ X_{2}. Thus

f(x_{1}, x_{2}) − (A_{2}(f))(x_{1}, x_{2}) = ∑_{j=1}^{r}(f(x_{1}, x_{2}) − f(x_{1}, x_{2,j})) ψ_{2,j}(x_{2}) (10.15.5)

for every x_{1} ∈ X_{1} and x_{2} ∈ X_{2}, by (10.15.2). It follows that

|f(x_{1}, x_{2}) − (A_{2}(f))(x_{1}, x_{2})| ≤ ∑_{j=1}^{r} |f(x_{1}, x_{2}) − f(x_{1}, x_{2,j})| ψ_{2,j}(x_{2}) (10.15.6)

for every x_{1} ∈ X_{1} and x_{2} ∈ X_{2}.

Put

a_{2,j}(x_{1}) = sup{|f(x_{1}, x_{2}) − f(x_{1}, x_{2,j})| : x_{2} ∈ X_{2}, ψ_{2,j}(x_{2}) > 0} (10.15.7)

for each j = 1, …, r. The right side is allowed to be +∞ here, although we shall typically be concerned with situations where it is finite. Observe that

|f(x_{1}, x_{2}) − f(x_{1}, x_{2,j})| ψ_{2,j}(x_{2}) ≤ a_{2,j}(x_{1}) ψ_{2,j}(x_{2}) (10.15.8)

for every x_{1} ∈ X_{1} and x_{2} ∈ X_{2}, where the right side may be interpreted as being equal to 0 when ψ_{2,j}(x_{2}) = 0, even if (10.15.7) is +∞. This implies that

|f(x_{1}, x_{2}) − (A_{2}(f))(x_{1}, x_{2})| ≤ ∑_{j=1}^{r} a_{2,j}(x_{1}) ψ_{2,j}(x_{2}) (10.15.9)

for every x_{1} ∈ X_{1} and x_{2} ∈ X_{2}, by (10.15.6).

Put

a_{2}(x_{1}) = max_{1≤j≤r} a_{2,j}(x_{1}) (10.15.10)

for each x_{1} ∈ X_{1}. Using (10.15.9), we get that

|f(x_{1}, x_{2}) − (A_{2}(f))(x_{1}, x_{2})| ≤ a_{2}(x_{1}) (10.15.11)

for every x_{1} ∈ X_{1}, because of (10.15.1).

We may frequently be interested in situations where X_{2} is compact, or at least totally bounded, and

{x_{2} ∈ X_{2} : ψ_{2,j}(x_{2}) > 0} (10.15.12)

is a small subset of X_{2} for each j = 1, …, r. This may mean that (10.15.12) is contained in a ball in X_{2} centered at x_{2,j} with small radius, or that (10.15.12) has small diameter with respect to d_{X_{2}}, which is nearly the same thing. If (10.15.12) is sufficiently small in this way, then we may be able to show that (10.15.7) is small, using suitable continuity properties of f.

Suppose that E_{2} = {f(x_{1}, ·) : x_{1} ∈ X_{1}} is uniformly equicontinuous as a collection of real-valued functions on X_{2}. In this case, (10.15.10) is as small as we like when (10.15.12) is contained in a ball centered at x_{2,j} of sufficiently small radius for each j = 1, …, r, or the diameter of (10.15.12) is sufficiently small for each j = 1, …, r, which is almost the same thing, as before.

If X_{1} is compact, then E_{2} is equicontinuous at every x_{2} ∈ X_{2}, as in Section 10.6. If X_{2} is compact, then the equicontinuity of E_{2} at every point in X_{2} implies that E_{2} is uniformly equicontinuous on X_{2}, as in Section 5.2. If f is uniformly continuous on X with respect to d_{X,1}, d_{X,2}, or d_{X,3}, then it is easy to see that E_{2} is uniformly equicontinuous on X_{2}. If X_{1} and X_{2} are compact, then X is compact with respect to d_{X,1}, d_{X,2}, and d_{X,3}, as in Section 10.4. In this case, continuity of f on X implies uniform continuity, as usual.

10.16 Some continuous functions

If c is any real number, then one can check that

(10.16.1) max(t, c) and min(t, c)

are Lipschitz functions of t ∈ R, with constant 1. Of course, this uses the standard Euclidean metric on the real line.

Let (X, d_{X}) be a metric space, and let A be a nonempty subset of X. If x ∈ X, then the distance from x to A with respect to d_{X} is defined by

dist(x, A) = inf{d(x, a) : a ∈ A}. (10.16.2)

One can check that dist(x, A) = 0 (10.16.3)

if and only if x is an element of the closure \overline{A} of A in X. One can also verify that (10.16.4) dist(x, \overline{A}) = dist(x, A)

for every x ∈ X.

It is easy to see that

(10.16.5) dist(x, A) ≤ d(x, w) + dist(w, A)

for every x, w ∈ X. Using this, one can check that dist(x, A) is Lipschitz with constant 1, as a real-valued function on X.

Let A, B be nonempty disjoint closed subsets of X. Observe that

(10.16.6) dist(x, A) + dist(x, B) > 0

for every x ∈ X. It follows that

\frac{dist(x, A)}{dist(x, A) + dist(x, B)}

(10.16.7)

defines a continuous real-valued function on X. This function is equal to 0 exactly on A, and it is equal to 1 exactly on B. It also takes values in [0, 1] on X, by construction.

Let r > 0 be given, and note that

min(dist(x, A), r) (10.16.8)

is Lipschitz with constant 1, as a real-valued function of x ∈ X. This implies that (1/r) min(dist(x, A), r) (10.16.9)

is Lipschitz with constant 1/r on X.

Suppose for the moment that

d(a, b) ≥ r (10.16.10)

10.16. SOME CONTINUOUS FUNCTIONS 191

for every a ∈ A and b ∈ B. Equivalently, this means that dist(b, A) ≥ r (10.16.11) for every b ∈ B. This implies that (10.16.8) is equal to r when x ∈ B, so that (10.16.9) is equal to 1.

Conversely, suppose that there is a uniformly continuous real-valued function f on X such that (10.16.12) f(a) = 0 for every a ∈ A, and f(b) = 1 for every b ∈ B. Under these conditions, it is easy to see that there is an r > 0 such that (10.16.10) holds for every a ∈ A and b ∈ B.

Let a_{0} ∈ X and r_{0} > 0 be given, and note that d(x, a_{0}) (10.16.13) is Lipschitz with constant 1 as a real-valued function of x ∈ X. This is the same as (10.16.2), with A = {a_{0}}. It follows that r_{0} − d(x, a_{0}) (10.16.14) is Lipschitz with constant 1 on X, so that max(r_{0} − d(x, a_{0}), 0) (10.16.15) is Lipschitz with constant 1 on X as well. Of course, (10.16.15) is equal to 0 when d(x, a_{0}) ≥ r_{0}, (10.16.16) and otherwise (10.16.15) is strictly positive.

Let r_{1} be a nonnegative real number with r_{1} < r_{0}, (10.16.17) and consider min(max(r_{0} − d(x, a_{0}), 0), r_{0} − r_{1}). (10.16.18) This is another real-valued Lipschitz function on X with constant 1, so that (1/(r_{0} − r_{1})) min(max(r_{0} − d(x, a_{0}), 0), r_{0} − r_{1}) (10.16.19) is Lipschitz with constant 1/(r_{0} − r_{1}) on X. This function is equal to 0 when (10.16.16) holds, and it is equal to 1 when d(x, a_{0}) ≤ r_{1}. (10.16.20) If r_{1} < d(x, a_{0}) < r_{0}, (10.16.21) then (10.16.19) is in (0, 1).

10.17 Products and reciprocals

Let (X, d_{X}) be a metric space, and let f, g be real-valued functions on X. If f and g are uniformly continuous on X, then it is easy to see that f + g is uniformly continuous on X too. If f and g are Lipschitz functions on X, then f + g is a Lipschitz function on X as well, as in Section 7.7.

Suppose for the moment that f and g are bounded on X. If f and g are also uniformly continuous on X, then one can check that their product f g is uniformly continuous on X. Similarly, if f and g are Lipschitz functions on X, then one can verify that f g is Lipschitz on X.

If t_{1}, t_{2} are nonzero real numbers, then

1/t_{1} − 1/t_{2} = (t_{2} − t_{1})/(t_{1} t_{2}). (10.17.1)

This implies that |1/t_{1} − 1/t_{2}| = |t_{1} − t_{2}|/(|t_{1}| |t_{2}|). (10.17.2)

If r is a positive real number, and |t_{1}|, |t_{2}| ≥ r, then we get that

|1/t_{1} − 1/t_{2}| ≤ r^{−2} |t_{1} − t_{2}|. (10.17.3)

This means that t ↦ 1/t is Lipschitz with constant r^{−2} as a real-valued function on {t ∈ R : |t| ≥ r}. (10.17.4)

Of course, this uses the standard Euclidean metric on R, and its restriction to (10.17.4).

Suppose now that |f(x)| ≥ r (10.17.5)

for every x ∈ X. If f is uniformly continuous on X, then it is easy to see that

(10.17.6) 1/f is uniformly continuous on X.

Similarly, if f is Lipschitz on X, then

(10.17.7) 1/f is Lipschitz on X.

These statements can be verified directly, or by considering 1/f as the composition of f with t ↦ 1/t on (10.17.4). There are analogous statements for complex numbers, and complex-valued functions on X.

10.18 Graphs of mappings

Let X_{1}, X_{2} be nonempty sets, and let X = X_{1} ×X_{2} be their Cartesian product. If f is a mapping from X_{1} into X_{2}, then the graph of f is the subset of X given by {(x_{1}, f(x_{1})) : x_{1} ∈ X_{1}}, (10.18.1) as usual. Put F (x_{1}) = (x_{1}, f(x_{1})) (10.18.2)

for every x_{1} ∈ X_{1}, which defines F as a mapping from X_{1} into X. Note that (10.18.1) is the same as the image of X_{1} under F.

Let p_{1}, p_{2} be the usual coordinate projections from X onto X_{1}, X_{2}, respectively, as in Section 10.5. Thus p_{j}(x) = x_{j} for each x = (x_{1}, x_{2}) ∈ X and j = 1, 2. Clearly f = p_{2} ∘ F (10.18.3)

on X_{1}, and p_{1} ∘ F is the identity mapping on X_{1}.

Suppose now that (X_{1}, d_{X_{1}}) and (X_{2}, d_{X_{2}}) are metric spaces, so that we can define the metrics d_{X,1}, d_{X,2}, and d_{X,∞} on X as in Section 10.1. Remember that these metrics determine the same collections of open sets, closed sets, and compact sets in X, and that convergence of sequences in X with respect to these metrics are equivalent, as before.

If (10.18.4) f is continuous as a mapping from X_{1} into X_{2},

then one can check that

(10.18.5) F is continuous as a mapping from X_{1} into X,

with respect to each of d_{X,1}, d_{X,2}, and d_{X,∞} on X. Conversely, if (10.18.5) holds, with respect to any of the metrics d_{X,1}, d_{X,2}, or d_{X,∞} on X, then (10.18.4) holds. This can be seen using (10.18.3), and the fact that p_{2} is a continuous mapping from X onto X_{2} with respect to each of d_{X,1}, d_{X,2}, and d_{X,∞} on X, as in Section 10.5.

One can check that a subset E of a metric space M is a closed set if and only if for every sequence {w_{j}}_{j=1}^{∞} of elements of E that converges to an element w of M, we have that w ∈ E. (10.18.6)

More precisely, if {w_{j}}_{j=1}^{∞} is a sequence of elements of any set E ⊆ M that converges to an element w of M, and if w_{j} ≠ w for each j, then w is a limit point of E. If w ∈ M is a limit point of E, then one can find a sequence {w_{j}}_{j=1}^{∞} of elements of E that converges to w, with w_{j} ≠ w for each j.

It is well known that if (10.18.4) holds, then

(10.18.7) the graph of f is a closed set in X,

with respect to each of the metrics d_{X,1}, d_{X,2}, d_{X,∞}. As in the preceding paragraph, (10.18.7) holds if and only if for every sequence of elements of the graph of f that converges to an element of X, the limit of the sequence is in the graph of f. In this case, this means that (10.18.7) holds if and only if for every sequence {x_{1,j}}_{j=1}^{∞} of elements of X_{1} such that

{(x_{1,j}, f(x_{1,j}))}_{j=1}^{∞} (10.18.8) converges to an element (x_{1}, x_{2}) of X, we have that

f(x_{1}) = x_{2}. (10.18.9)

As in Section 10.3, the convergence of (10.18.8) to (x_{1}, x_{2}) ∈ X with respect to any of the metrics d_{X,1}, d_{X,2}, d_{X,∞} is equivalent to the convergence of {x_{1,j}}_{j=1}^{∞} to x_{1} in X_{1} and the convergence of {f(x_{1,j})}_{j=1}^{∞} to x_{2} in X_{2}. If f is continuous at x_{1}, then the convergence of {x_{1,j}}_{j=1}^{∞} to x_{1} in X_{1} implies that {f(x_{1,j})}_{j=1}^{∞} converges to f(x_{1}) in X_{2}. This implies that (10.18.9) holds when {f(x_{1,j})}_{j=1}^{∞} converges to x_{2} in X_{2}.

If (10.18.4) holds, and if

(10.18.10) X_{1} is compact,

then (10.18.11) the graph of f is a compact subset of X,

with respect to each of d_{X,1}, d_{X,2}, and d_{X,∞}. Remember that the graph of f is the same as F (X_{1}), and that F is continuous as a mapping from X_{1} into X in this case. This implies that F (X_{1}) is a compact subset of X when X_{1} is compact.

Conversely, let f be any mapping from X_{1} into X_{2}, and suppose that (10.18.11) holds, with respect to d_{X,1}, d_{X,2}, or d_{X,∞}. It is easy to see that (10.18.10) holds in this case, because p_{1} maps the graph of f onto X_{1}. This also use the fact that p_{1} is continuous as a mapping from X onto X_{1}, as in Section 10.5.

It is well known that (10.18.4) holds under these conditions as well. To see this, let x_{1} ∈ X_{1} be given, and suppose for the sake of a contradiction that f is not continuous at x_{1}. This means that there is an ϵ > 0 such that for every δ > 0 there is a point w_{1} ∈ X_{1} with

d_{X_{1}}(x_{1}, w_{1}) < δ (10.18.12)

and d_{X_{2}}(f(x_{1}), f(w_{1})) ≥ ϵ. (10.18.13)

One can use this to get a sequence {x_{1,j}}_{j=1}^{∞} of elements of X_{1} that converges to x_{1}, with d_{X_{2}}(f(x_{1}), f(x_{1,j})) ≥ ϵ (10.18.14)

for every j.

If (10.18.11) holds, then the graph of f is sequentially compact in X. Using this, we get that there is a subsequence {x_{1,j_{l}}}_{l=1}^{∞} of {x_{1,j}}_{j=1}^{∞} such that

{(x_{1,j_{l}}, f(x_{1,j_{l}}))}_{l=1}^{∞} (10.18.15)

converges to an element (y_{1}, f(y_{1})) of the graph of f in X. This means that

(10.18.16) {x_{1,j_{l}}}_{l=1}^{∞} converges to y_{1} in X_{1}, and that (10.18.17) {f(x_{1,j_{l}})}_{l=1}^{∞} converges to f(y_{1}) in X_{2},

as in Section 10.3. Note that {x_{1,j_{l}}}_{l=1}^{∞} converges to x_{1} in X_{1}, because {x_{1,j}}_{j=1}^{∞} converges to x_{1}, by construction. Thus

x_{1} = y_{1}. (10.18.18)

It follows that (10.18.19) {f(x_{1,j_{l}})}_{l=1}^{∞} converges to f(x_{1}) in X_{2}.

This contradicts (10.18.14), as desired.

10.19 Semicontinuity

Let us continue with the same notation and hypotheses as in the previous section, except that now we take X_{2} = R, with the standard Euclidean metric. If f is a real-valued function on X_{1}, then

{(x_{1}, x_{2}) ∈ X_{1} × R : f(x_{1}) > x_{2}} (10.19.1)

and {(x_{1}, x_{2}) ∈ X_{1} × R : f(x_{1}) ≤ x_{2}} (10.19.2)

are complementary subsets of X = X_{1} × R. In particular, (10.19.1) is an open set in X = X_{1} × R if and only if (10.19.2) is a closed set in X, with respect to any, and thus each, of the metrics d_{X,1}, d_{X,2}, and d_{X,∞}.

Similarly, {(x_{1}, x_{2}) ∈ X_{1} × R : f(x_{1}) ≥ x_{2}} (10.19.3)

and {(x_{1}, x_{2}) ∈ X_{1} × R : f(x_{1}) < x_{2}} (10.19.4)

are complementary subsets of X. It follows that (10.19.3) is a closed set in X if and only if (10.19.4) is an open set, as before. Note that the graph of f is the same as the intersection of (10.19.2) and (10.19.3).

If f is continuous on X, then one can show that (10.19.2) and (10.19.3) are closed sets in X, using the same type of argument as in the previous section. However, there are more precise statements, using notions of semicontinuity.

We say that f is upper semicontinuous at a point x_{1} ∈ X_{1} if for every ϵ > 0 there is a δ > 0 such that f(w_{1}) < f(x_{1}) + ϵ (10.19.5)

for every w_{1} ∈ X_{1} with d_{X_{1}}(x_{1}, w_{1}) < δ. Similarly, we say that f is lower semicontinuous at x_{1} if for every ϵ > 0 there is a δ′ > 0 such that

f(w_{1}) > f(x_{1}) − ϵ (10.19.6)

for every w_{1} ∈ X_{1} with d_{X_{1}}(x_{1}, w_{1}) < δ′.

Observe that f is continuous at x_{1} if and only if f is both upper and lower semicontinuous at x_{1}. It is easy to see that f is upper semicontinuous at x_{1} if and only if −f is lower semicontinuous at x_{1}.

Let us say that f is upper semicontinuous on X_{1} if f is upper semicontinuous at every x_{1} ∈ X_{1}. Similarly, we say that f is lower semicontinuous on X_{1} if f is lower semicontinuous at every x_{1} ∈ X_{1}.

One can check that f is upper semicontinuous on X_{1} if and only if for every real number b, {x_{1} ∈ X_{1} : f(x_{1}) < b} (10.19.7)

is an open subset of X_{1}. Similarly, f is lower semicontinuous on X_{1} if and only if for every a ∈ R, {x_{1} ∈ X_{1} : f(x_{1}) > a} (10.19.8)

is an open set in X_{1}.

One can also verify that f is upper semicontinuous on X_{1} if and only if (10.19.4) is an open set in X. Similarly, f is lower semicontinuous on X_{1} if and only if (10.19.1) is an open set in X.

Let K be a nonempty compact subset of X_{1}. If f is upper semi-continuous on X_{1}, then it is not too difficult to show that f attains its maximum on K. Similarly, if f is lower semicontinuous on X_{1}, then f attains its minimum on K. Of course, this is another version of the extreme value theorem.

Let {x_{1,j}}_{j=1}^{∞} be a sequence of elements of X_{1} that converges to x_{1} ∈ X_{1}. If f is upper semi-continuous at x_{1}, then one can check that

lim sup_{j→∞} f(x_{j}) ≤ f(x_{1}). (10.19.9)

Similarly, if f is lower semicontinuous at x_{1}, then

lim inf_{j→∞} f(x_{j}) ≥ f(x_{1}). (10.19.10)

One can verify that these properties characterize upper and lower semicontinuity of f at x_{1} too.

10.20 Homeomorphisms between metric spaces

Let (X, d_{X}) and (Y, d_{Y}) be metric spaces. A one-to-one mapping f from X onto Y is said to be a homeomorphism if f is continuous, and the corresponding inverse mapping f^{−1} from Y onto X is continuous. Of course, f^{−1} is a homeomorphism from Y onto X in this case.

Let (Z, d_{Z}) be another metric space. If f is a homeomorphism from X onto Y, and g is a homeomorphism from Y onto Z, then their composition g ∘ f is a homeomorphism from X onto Z.

Let f be a one-to-one continuous mapping from X onto Y. If X is compact, then it is well known that the inverse mapping f^{−1} is continuous, so that

(10.20.1) f is a homeomorphism.

One can show this using sequences, or by looking at closed sets.

To show this using sequences, let {y_{j}}_{j=1}^{∞} be a sequence of elements of Y that converges to a point y ∈ Y, and let us check that (10.20.2) {f^{−1}(y_{j})}_{j=1}^{∞} converges to f^{−1}(y) in X. Put x_{j} = f^{−1}(y_{j}) for each j, and x = f^{−1}(y), for convenience. Suppose for the sake of a contradiction that {x_{j}}_{j=1}^{∞} does not converge to x in X. This implies that there is an ϵ > 0 such that d_{X}(x, x_{j}) ≥ ϵ (10.20.3) for infinitely many j ≥ 1. Equivalently, this means that there is a subsequence {x_{j_{l}}}_{l=1}^{∞} of {x_{j}}_{j=1}^{∞} such that d_{X}(x, x_{j_{l}}) ≥ ϵ (10.20.4) for every l ≥ 1.

If X is compact, and thus sequentially compact, then there is a subsequence {x_{j_{ln}} }_{n=1}^{∞} of {x_{j_{l}}}_{l=1}^{∞} that converges to an element w of X. It follows that (10.20.5) {f(x_{j_{ln}})}_{n=1}^{∞} converges to f(w) in Y, because f is continuous at w, by hypothesis. However, {f(x_{j_{ln}})}_{n=1}^{∞} = {y_{j_{ln}} }_{n=1}^{∞} (10.20.6) is a subsequence of {y_{j}}_{j=1}^{∞}, which converges to y = f(x). This implies that f(w) = f(x), (10.20.7) so that w = x, because f is one-to-one. This means that {x_{j_{ln}} }_{n=1}^{∞} converges to x in X, contradicting (10.20.4), as desired.

Alternatively, let E be a closed set in X. In order to check that f^{−1} is continuous as a mapping from Y into X, we would like to verify that (f^{−1})^{−1}(E) = {y ∈ Y : f^{−1}(y) ∈ E} (10.20.8) is a closed set in Y. It is easy to see that (f^{−1})^{−1}(E) = f(E). (10.20.9) Note that (10.20.10) E is compact in X, because X is compact, and E is a closed set. This implies that (10.20.11) f(E) is a compact set in Y, because f is continuous. It follows that (10.20.12) f(E) is a closed set in Y. This means that f^{−1} is continuous, by a standard characterization of continuous mappings between metric spaces.

One might notice that the first argument is very similar to one in Section 10.18. This will be discussed further in the next section.

10.21 Graphs and homeomorphisms

Let (X_{1}, d_{X_{1}}), (X_{2}, d_{X_{2}}) be nonempty metric spaces, and put X = X_{1} ×X_{2}. As usual, we can define metrics d_{X,1}, d_{X,2}, and d_{X,∞} on X as in Section 10.1. Let f be a mapping from X_{1} into X_{2}, and let F be the mapping from X_{1} into X defined by putting F (x_{1}) = (x_{1}, f(x_{1})) for every x_{1} ∈ X_{1}, as in Section 10.18.

Let Y = F (X_{1}) (10.21.1)

be the graph of f in X, as before. We may consider Y as a metric space, using the restriction of d_{X,1}, d_{X,2}, or d_{X,∞} to Y. Note that F is a one-to-one mapping from X_{1} onto Y, by construction.

Let p_{1}, p_{2} be the usual coordinate projections from X onto X_{1}, X_{2}, respectively, as in Section 10.5. The restriction of p_{1} to Y is the same as the inverse mapping F^{−1} of F on Y. It follows that

(10.21.2) F^{−1} is continuous on Y,

with respect to the restriction of d_{X,1}, d_{X,2}, or d_{X,∞} to Y, as in Section 10.5.

If f is continuous as a mapping from X_{1} into X_{2}, then F is continuous as a mapping from X_{1} into X with respect to each of d_{X,1}, d_{X,2}, and d_{X,∞}, as in Section 10.18. In this case, we get that

(10.21.3) F is a homeomorphism from X_{1} onto Y,

with respect to the restriction of any of d_{X,1}, d_{X,2}, and d_{X,∞} to Y.

If Y is compact with respect to any of d_{X,1}, d_{X,2}, or d_{X,∞}, then one can get that (10.21.4) F is continuous,

because F^{−1} is continuous on Y, as in the previous section. Of course, this implies that (10.21.5) f = p_{1} ∘ F is continuous on X_{1},

because p_{1} is continuous on X, as in Section 10.5. This is another way to look at how the compactness of Y implies the continuity of f, as in Section 10.18.

Let m be a positive integer, and let us now take

X_{2} = R^{m}, (10.21.6)

equipped with the standard Euclidean metric, or the metric associated to a norm. Let f be a mapping from X_{1} into R^{m}, and let Φ be the mapping from X = X_{1} × R^{m} into itself defined by

Φ(x) = (x_{1}, x_{2} + f(x_{1})) (10.21.7)

for every x = (x_{1}, x_{2}) ∈ X. It is easy to see that Φ is a one-to-one mapping from X onto itself. More precisely, the inverse mapping is given by

Φ^{−1}(x) = (x_{1}, x_{2} − f(x_{1})) (10.21.8) for every x ∈ X.

If f is continuous on X_{1}, then one can check that Φ and Φ^{−1} are continuous on X, with respect to any of the metrics d_{X,1}, d_{X,2}, or d_{X,∞}. This means that Φ is a homeomorphism from X onto itself, with respect to any of these three metrics. It is easy to see that the continuity of f is necessary for the continuity of Φ or Φ^{−1}.

If x_{1} ∈ X_{1} and x = (x_{1}, 0), then Φ(x) = F (x_{1}). In particular,

Φ(X_{1} × {0}) = F (X_{1}). (10.21.9)

Equivalently, F (X_{1}) = (Φ^{−1})^{−1}(X_{1} × {0}). (10.21.10)

It is easy to see that X_{1} × {0} is a closed set in X, with respect to any of d_{X,1}, d_{X,2}, or d_{X,∞}. If f is continuous, then Φ^{−1} is continuous, and (10.21.10) gives another way to see that F (X_{1}) is a closed set in X, as in Section 10.18.

Suppose now that m = 1, so that f is a real-valued function on X_{1}. Observe that Φ(X_{1} × (0, +∞)) = {(x_{1}, x_{2}) ∈ X_{1} × R : x_{2} > f(x_{1})} (10.21.11)

and Φ(X_{1} × (−∞, 0)) = {(x_{1}, x_{2}) ∈ X_{1} × R : x_{2} < f(x_{1})}. (10.21.12)

It is easy to see that X_{1} × (0, +∞) and X × (0, −∞) are open sets in X, with respect to any of d_{X,1}, d_{X,2}, or d_{X,∞}. If f is continuous, then one can use the continuity of Φ^{−1} to get another way to see that (10.21.11) and (10.21.12) are open sets in X, as in Section 10.19.

Summable functions

Sums of real and complex-valued functions on arbitrary nonempty sets are considered in this chapter, extending sums of absolutely convergent series of real and complex numbers. The reader may choose to skip this chapter, at least initially.

11.1 Extended real numbers

As usual, the set of extended real numbers consists of the real numbers together with two additional elements, denoted +∞ and −∞. The standard ordering is extended to the set of extended real numbers by putting

−∞ < x < +∞ (11.1.1)

for every x ∈ R. Normally when we consider extended real numbers here, we shall only be concerned with nonnegative extended real numbers.

In some situations, addition and multiplication of extended real numbers can be defined in a natural way. In particular, we put

x + ∞ = ∞ + x = ∞ + ∞ = +∞ (11.1.2)

for every x ∈ R. Similarly, we put

x · ∞ = ∞ · x = ∞ · ∞ = ∞ (11.1.3)

for every positive real number x. Although 0 · ∞ is not necessarily defined, it will normally correspond to 0 here.

The notions of upper and lower bounds, supremum, and infimum can be defined for sets of extended real numbers, in the same way as for sets of real numbers. In particular, if A is a nonempty set of real numbers that does not have a finite upper bound in R, then the supremum of A can be defined as an extended real number to be +∞. Of course, if +∞ is an element of A, then the supremum of A is equal to +∞ automatically. If +∞ is the only element of A, then the infimum of A is equal to +∞.

Let t be a positive real number, and let E be a nonempty set of extended real numbers. Put t E = {t x : x ∈ E}, (11.1.4)

which is another nonempty set of extended real numbers. One can check that

sup(t E) = t (sup E). (11.1.5)

Let {x_{j}}_{j=1}^{∞} be a sequence of real numbers. If {x_{j}}_{j=1}^{∞} converges to a real number x with respect to the standard metric on R, then we may express this by x_{j} → x as j → ∞. As usual, we say that {x_{j}}_{j=1}^{∞} tends to +∞ as j → ∞, or x_{j} → +∞ as j → ∞, if for every positive real number R there is a positive integer L such that x_{j} > R (11.1.6)

for each j ≥ L.

Let {x_{j}}_{j=1}^{∞} be a monotonically increasing sequence of real numbers, so that

x_{j} ≤ x_{j+1} (11.1.7)

for every j ≥ 1. Put x = sup{x_{j} : j ∈ Z_{+}}, (11.1.8)

which is a real number when the set of x_{j}’s, j ∈ Z_{+}, has a finite upper bound in R, and otherwise is equal to +∞. It is well known that

x_{j} → x as j → ∞ (11.1.9)

under these conditions.

Now let {x_{j}}_{j=1}^{∞} be a sequence of real numbers such that x_{j} → +∞ as j → ∞. If {y_{j}}_{j=1}^{∞} is a sequence of real numbers with a finite lower bound in R, then it is easy to see that

x_{j} + y_{j} → +∞ as j → ∞. (11.1.10)

In particular, this holds when y_{j} → y as j → ∞, where y ∈ R or y = +∞.

11.2 Nonnegative sums

Let X be a nonempty set, and let f be a nonnegative real-valued function on X. If A is a nonempty finite subset of X, then the sum

∑_{x∈A} f(x) (11.2.1)

can be defined as a nonnegative real number in the usual way. Put

(11.2.2) ∑_{x∈X} f(x) = sup { ∑_{x∈A} f(x) : A is a nonempty finite subset of X}, where the supremum on the right side is defined as a nonnegative extended real number, as in the previous section. Of course, if X has only finitely many elements, then the supremum is attained with A = X. Similarly, if f has finite support in X, then the supremum is attained with any nonempty finite subset A of X such that A contains the support of f.

Suppose for the moment that X is the set Z_{+} of positive integers, and let f be a nonnegative real-valued function defined on Z_{+}. In this case, one may put

∑_{j=1}^{∞} f(j) = sup { ∑_{j=1}^{n} f(j) : n ∈ Z_{+}}, (11.2.3)

where the supremum on the right side is defined as a nonnegative extended real number again. Of course, the sequence of partial sums ∑_{j=1}^{n} f(j) increases monotonically in this case. The sequence of partial sums tends to its supremum as n → ∞, as in the previous section. If the sequence of partial sums has a finite upper bound in R, then the sequence of partial sums converges in R in the usual sense. This means that the infinite series on the left side of (11.2.3) converges in the usual sense, with sum equal to the right side of (11.2.3). If the sequence of partial sums does not have a finite upper bound, so that the right side of (11.2.3) is +∞, then one may interpret the sum on the left side of (11.2.3) as being +∞ as well.

If f is any nonnegative real-valued function on Z_{+}, then

∑_{j=1}^{∞} f(j) = ∑_{j∈Z_{+}} f(j), (11.2.4)

where these sums are as defined in (11.2.2) and (11.2.3). More precisely, one can check that

∑_{j=1}^{∞} f(j) ≤ ∑_{j∈Z_{+}} f(j), (11.2.5)

because each of the partial sums on the right side of (11.2.3) may be considered as one of the finite sums on the right side of (11.2.2). To get the other inequality, one can use the fact that every finite subset A of Z_{+} is contained in {1, …, n} for some n ∈ Z_{+}.

It is sometimes convenient to consider a nonnegative extended real-valued function f on a nonempty set X. If A is a nonempty finite subset of X and f(x) = +∞ for some x ∈ A, then the corresponding sum (11.2.1) is equal to +∞. If f(x) = +∞ for some x ∈ X, then the right side of (11.2.2) is equal to +∞. Similarly, if X = Z_{+}, then the partial sum ∑_{j=1}^{n} f(j) is +∞ when f(j) = +∞ for some j ≤ n, so that the right side of (11.2.3) is +∞ when f(j) = +∞ for some j. Thus (11.2.4) also holds in this situation.

11.3 Compositions and subsets

Let X and Y be nonempty sets, and let ϕ be a one-to-one mapping from X onto Y. Also let f be a nonnegative extended real-valued function on Y, so that f(ϕ(x)) defines a nonnegative extended real-valued function on Y. If A is a nonempty finite subset of X, then ϕ(A) is a nonempty finite subset of Y, and

∑_{x∈A} f(ϕ(x)) = ∑_{y∈ϕ(A)} f(y). (11.3.1)

If B is any nonempty finite subset of Y, then A = ϕ^{−1}(B) is a nonempty finite subset of X, and B = ϕ(A). It follows that

∑_{x∈X} f(ϕ(x)) = ∑_{y∈Y} f(y), (11.3.2)

because both sums are defined by taking the supremum of the corresponding finite subsums in (11.3.1).

Let f be a nonnegative extended real-valued function on a nonempty set X. If E is a nonempty subset of X, then the sum

∑_{x∈E} f(x) (11.3.3)

can be defined as a nonnegative extended real number in the same way as before, as the supremum of the corresponding collection of finite subsums. This is the same as applying the definition in the previous section to the restriction of f to E. If E_{1}, E_{2} are nonempty subsets of X and E_{1} ⊆ E_{2}, then

∑_{x∈E_{1}} f(x) ≤ ∑_{x∈E_{2}} f(x). (11.3.4)

This uses the fact that every finite subsum of the sum on the left is also a finite subsum of the sum on the right. If we also have that f(x) = 0 for every x ∈ E_{2} \ E_{1}, then it follows that

∑_{x∈E_{1}} f(x) = ∑_{x∈E_{2}} f(x). (11.3.5)

This is because finite subsums of the sum on the right side can be reduced to finite subsums of the sum on the left, except possibly for sums over nonempty finite subsets of E_{2} \ E_{1}, which are equal to 0 in this case.

Let {x_{j}}_{j=1}^{∞} be a sequence of distinct elements of X, and let

E = {x_{j} : j ∈ Z_{+}}. (11.3.6)

If f is a nonnegative extended real-valued function on X again, then f(x_{j}) may be considered as a nonnegative extended real-valued function on Z_{+}. Thus

∑_{j=1}^{∞} f(x_{j}) = ∑_{j∈Z_{+}} f(x_{j}), (11.3.7)

as in (11.2.4). We also have that

∑_{j∈Z_{+}} f(x_{j}) = ∑_{x∈E} f(x), (11.3.8)

as in (11.3.2). More precisely, this uses the fact that j ↦ x_{j} is a one-to-one mapping from Z_{+} onto E.

Let f, g be nonnegative extended real-valued functions on X, and suppose that f(x) ≤ g(x) (11.3.9)

for every x ∈ X. If A is a nonempty finite subset of X, then

∑_{x∈A} f(x) ≤ ∑_{x∈A} g(x). (11.3.10)

Using this, it is easy to see that

∑_{x∈X} f(x) ≤ ∑_{x∈X} g(x). (11.3.11)

11.4 Nonnegative summable functions

Let f be a nonnegative real-valued function on a nonempty set X. If

∑_{x∈X} f(x) < ∞, (11.4.1)

then f is said to be summable on X. Of course, if f has finite support in X, then f is summable on X. Suppose now that f is summable on X, and let us check that f vanishes at infinity on X, as in Section 2.5.

Let ϵ > 0 be given, and put

E_{ϵ}(f) = {x ∈ X : f(x) ≥ ϵ}. (11.4.2)

If A is a nonempty finite subset of E_{ϵ}(f), then

ϵ (#A) ≤ ∑_{x∈A} f(x) ≤ ∑_{x∈X} f(x), (11.4.3)

where #A denotes the number of elements of A. Thus

#A ≤ (1/ϵ) ∑_{x∈X} f(x). (11.4.4)

It follows that E_{ϵ}(f) has only finitely many elements, with

#E_{ϵ}(f) ≤ (1/ϵ) ∑_{x∈X} f(x). (11.4.5)

Let us continue to suppose that f is summable on X, and let ϵ > 0 be given again. Observe that there is a nonempty finite subset A(ϵ) of X such that

∑_{x∈X} f(x) − ϵ < ∑_{x∈A(ϵ)} f(x), (11.4.6)

by the definition (11.2.2) of the sum over X. If A is a finite subset of X that contains A(ϵ), then

∑_{x∈A(ϵ)} f(x) ≤ ∑_{x∈A} f(x) ≤ ∑_{x∈X} f(x) < ∑_{x∈A(ϵ)} f(x) + ϵ. (11.4.7)

Let B be a nonempty finite subset of X that is disjoint from A(ϵ), so that

∑_{x∈A(ϵ)} f(x) + ∑_{x∈B} f(x) = ∑_{x∈A(ϵ)∪B} f(x) ≤ ∑_{x∈X} f(x). (11.4.8)

This implies that

∑_{x∈B} f(x) ≤ ∑_{x∈X} f(x) − ∑_{x∈A(ϵ)} f(x). (11.4.9)

If A(ϵ) ≠ X, then it follows that

∑_{x∈X\A(ϵ)} f(x) ≤ ∑_{x∈X} f(x) − ∑_{x∈A(ϵ)} f(x) < ϵ. (11.4.10)

If A(ϵ) = X, then the sum on the left side of (11.4.10) may be interpreted as being equal to 0.

If f is summable on X and E is a nonempty subset of X, then the restriction of f to E is summable on E. More precisely, the sum of f(x) over x ∈ E is less than or equal to the sum of f(x) over x ∈ X, as in (11.3.4). Similarly, let f and g be nonnegative real-valued functions on X such that f(x) ≤ g(x) for every x ∈ X. If g is summable on X, then f is summable on X too, by (11.3.11).

11.5 Some linearity properties

Let X be a nonempty set, and let f be a nonnegative extended real-valued function on X. Also let t be a positive real number, so that t f is a nonnegative extended real-valued function on X as well. If A is a nonempty finite subset of X, then

∑_{x∈A} t f(x) = t ∑_{x∈A} f(x). (11.5.1)

Using this, one can check that

∑_{x∈X} t f(x) = t ∑_{x∈X} f(x). (11.5.2)

More precisely, this can be obtained from (11.1.5) and the definition (11.2.2) of the sum over X.

Let g be another nonnegative extended real-valued function on X, so that f+g is a nonnegative extended real-valued function on X too. If A is a nonempty finite subset of X, then

∑_{x∈A}(f(x) + g(x)) = ∑_{x∈A} f(x) + ∑_{x∈A} g(x). (11.5.3)

One can use this to show that

∑_{x∈X}(f(x) + g(x)) = ∑_{x∈X} f(x) + ∑_{x∈X} g(x). (11.5.4)

More precisely, if A is a nonempty finite subset of X, then

∑_{x∈A}(f(x) + g(x)) ≤ ∑_{x∈X} f(x) + ∑_{x∈X} g(x), (11.5.5)

by (11.5.3). This implies that

∑_{x∈X}(f(x) + g(x)) ≤ ∑_{x∈X} f(x) + ∑_{x∈X} g(x). (11.5.6)

We would like to verify that

∑_{x∈X} f(x) + ∑_{x∈X} g(x) ≤ ∑_{x∈X}(f(x) + g(x)), (11.5.7)

in order to get (11.5.4). This is trivial when the right side of (11.5.7) is +∞, and so we may suppose that it is finite, so that f + g is summable on X. It follows that f and g are summable on X, because f, g ≤ f +g, as in the previous section. In particular, f(x) and g(x) are finite for every x ∈ X. Let A and B be nonempty finite subsets of X, and observe that

∑_{x∈A} f(x) + ∑_{x∈B} g(x) ≤ ∑_{x∈A∪B} f(x) + ∑_{x∈A∪B} g(x)
(11.5.8)  = ∑_{x∈A∪B}(f(x) + g(x)) ≤ ∑_{x∈X}(f(x) + g(x)).

Thus

∑_{x∈A} f(x) ≤ ∑_{x∈X}(f(x) + g(x)) − ∑_{x∈B} g(x). (11.5.9)

This implies that

∑_{x∈X} f(x) ≤ ∑_{x∈X}(f(x) + g(x)) − ∑_{x∈B} g(x) (11.5.10)

11.6. ℓ^{1} SPACES 207

for every nonempty finite set B ⊆ X. Equivalently,

∑_{x∈B} g(x) ≤ ∑_{x∈X}(f(x) + g(x)) − ∑_{x∈X} f(x) (11.5.11)

for every nonempty finite set B ⊆ X, and hence

∑_{x∈X} g(x) ≤ ∑_{x∈X}(f(x) + g(x)) − ∑_{x∈X} f(x). (11.5.12)

This shows that (11.5.7) holds under these conditions, as desired.

Let f be a nonnegative extended real-valued function on X again, and let E_{1}, E_{2} be disjoint nonempty subsets of X. It is easy to see that

∑_{x∈E_{1}∪E_{2}} f(x) = ∑_{x∈E_{1}} f(x) + ∑_{x∈E_{2}} f(x), (11.5.13)

using (11.5.4). More precisely, this can be obtained by expressing f on E_{1} ∪ E_{2} as the sum of two functions, with supports contained in E_{1} and E_{2}.

11.6 ℓ^{1} Spaces

A real or complex–valued function f on a nonempty set X is said to be summable on X if |f| is summable as a nonnegative real-valued function on X. Let ℓ^{1}(X, R) and ℓ^{1}(X, C) be the spaces of real and complex-valued summable functions on X, respectively. Of course, if a real or complex-valued function f on X has finite support, then f is summable on X, so that

c_{00}(X, R) ⊆ ℓ^{1}(X, R), c_{00}(X, C) ⊆ ℓ^{1}(X, C). (11.6.1)

If f is any real or complex-valued summable function on X, then |f| vanishes at infinity on X, as in Section 11.4. This means that f vanishes at infinity on X too, so that

ℓ^{1}(X, R) ⊆ c_{0}(X, R), ℓ^{1}(X, C) ⊆ c_{0}(X, C). (11.6.2)

If f is a real or complex-valued summable function on X, then put

∥f∥_{1} = ∑_{x∈X} |f(x)|. (11.6.3)

This is a nonnegative real number, which is equal to 0 exactly when f(x) = 0 for every x ∈ X. If t ∈ R or C, as appropriate, then |t f| = |t| |f| is summable on X, by (11.5.2), and we have that

∥t f∥_{1} = ∑_{x∈X} |t f(x)| = |t| ∑_{x∈X} |f(x)| = |t| ∥f∥_{1}. (11.6.4)

Similarly, if g is another real or complex-valued summable function on X, as appropriate, then it is easy to see that f + g is summable on X as well, with

∥f + g∥_{1} = ∑_{x∈X} |f(x) + g(x)| ≤ ∑_{x∈X}(|f(x)| + |g(x)|) = ∥f∥_{1} + ∥g∥_{1}. (11.6.5)

Thus ℓ^{1}(X, R) and ℓ^{1}(X, C) are linear subspaces of the real and complex vector spaces of all real and complex-valued functions on X, respectively, and (11.6.3) defines a norm on each of ℓ^{1}(X, R) and ℓ^{1}(X, C).

One can verify that d_{1}(f, g) = ∥f − g∥_{1} (11.6.6)

defines a metric on each of ℓ^{1}(X, R) and ℓ^{1}(X, C), using (11.6.4) and (11.6.5), as usual. Remember that ∥f∥_{∞} denotes the supremum norm of a bounded real or complex-valued f on X, as in (1.13.4). If f is a real or complex-valued summable function on X, then

|f(x)| ≤ ∥f∥_{1} (11.6.7)

for every x ∈ X. This implies that f is bounded on X, with

∥f∥_{∞} ≤ ∥f∥_{1}. (11.6.8)

If f, g are real or complex-valued summable functions on X, then it follows that

d_{∞}(f, g) ≤ d_{1}(f, g), (11.6.9)

where d_{∞}(f, g) is the supremum metric, as in (1.13.8).

Let us check that c_{00}(X, R), c_{00}(X, C) are dense in ℓ^{1}(X, R), ℓ^{1}(X, C), respectively, with respect to the ℓ^{1} metric (11.6.6). Of course, this is trivial when X has only finitely many elements, and so we suppose that X is an infinite set. Let f be a real or complex-valued summable function on X, and let ϵ > 0 be given. Remember that there is a finite subset A(ϵ) of X such that

∑_{x∈X\A(ϵ)} |f(x)| < ϵ, (11.6.10)

as in (11.4.10). Let f_{ϵ} be the real or complex-valued function, as appropriate, defined on X by

(11.6.11)  f_{ϵ}(x) = f(x) when x ∈ A(ϵ)
= 0  when x ∈ X \ A(ϵ).

Thus f_{ϵ} has finite support in X. Note that f − f_{ϵ} is equal to 0 on A(ϵ), and to f on X \ A(ϵ). This implies that

∥f − f_{ϵ}∥_{1} = ∑_{x∈X\A(ϵ)} |f(x)| < ϵ, (11.6.12)

as desired.

11.7. REAL-VALUED SUMMABLE FUNCTIONS 209

11.7 Real-valued summable functions

Let f be a real-valued function on a nonempty set X, and put (11.7.1) f_{+}(x) = max(f(x), 0) and f_{−}(x) = max(−f(x), 0) for each x ∈ X. These are nonnegative real-valued functions on X that satisfy f_{+}(x) + f_{−}(x) = |f(x)| (11.7.2) and f_{+}(x) − f_{−}(x) = f(x) (11.7.3) for every x ∈ X. If f is summable on X, so that |f| is summable on X, then f_{+} and f_{−} are summable as nonnegative real-valued functions on X. In this case, we put

∑_{x∈X} f(x) = ∑_{x∈X} f_{+}(x) − ∑_{x∈X} f_{−}(x). (11.7.4)

Note that

|∑_{x∈X} f(x)| ≤ ∑_{x∈X} f_{+}(x) + ∑_{x∈X} f_{−}(x) = ∑_{x∈X} |f(x)|. (11.7.5)

Let f_{1}, f_{2} be nonnegative real-valued summable functions on X such that f(x) = f_{1}(x) − f_{2}(x) (11.7.6) for every x ∈ X. This implies that f_{1}(x) + f_{−}(x) = f_{+}(x) + f_{2}(x) (11.7.7) for every x ∈ X, because of (11.7.3). Hence

∑_{x∈X} f_{1}(x) + ∑_{x∈X} f_{−}(x) = ∑_{x∈X} f_{+}(x) + ∑_{x∈X} f_{2}(x), (11.7.8)

as in (11.5.7). It follows that

∑_{x∈X} f_{1}(x) − ∑_{x∈X} f_{2}(x) = ∑_{x∈X} f_{+}(x) − ∑_{x∈X} f_{−}(x), (11.7.9)

because the individual sums are all finite. This means that

∑_{x∈X} f(x) = ∑_{x∈X} f_{1}(x) − ∑_{x∈X} f_{2}(x), (11.7.10)

by (11.7.4).

If f is a real-valued summable function on X and t ∈ R, then t f is summable on X too, as in the previous section. One can check that

∑_{x∈X} t f(x) = t ∑_{x∈X} f(x). (11.7.11)

Of course, this is trivial when t = 0. If t > 0, then (11.7.11) can be derived from the analogous fact (11.5.2) for nonnegative real-valued functions on X, and the definition (11.7.4) of the sum for real-valued summable functions on X. If t = −1, then (11.7.11) can be obtained directly from (11.7.4).

If f, g are real-valued summable functions on X, then f + g is summable on X as well, as in the previous section again. One can verify that

∑_{x∈X}(f(x) + g(x)) = ∑_{x∈X} f(x) + ∑_{x∈X} g(x). (11.7.12)

We have already seen in Section 11.5 that this holds when f and g are nonnegative. Otherwise, f and g can be expressed as differences of nonnegative real-valued summable functions on X, as in (11.7.3). This leads to an expression of f + g as a difference of nonnegative real-valued summable functions on X. Each of the sums of f, g, and f + g over X can be given as the difference of the corresponding sums of nonnegative summable functions, as in (11.7.4) and (11.7.10). Using this, one can reduce (11.7.12) to the analogous statement for nonnegative functions.

11.8 Complex-valued summable functions

Let f be a complex-valued summable function on a nonempty set X, and let Re f(x), Im f(x) be the real and imaginary parts of f(x) for each x ∈ X, as usual. Note that

(11.8.1) | Re f(x)|, | Im f(x)| ≤ |f(x)| ≤ | Re f(x)| + | Im f(x)|

for every x ∈ X. This implies that f is summable as a complex-valued function on X if and only if its real and imaginary parts are summable as real-valued functions on X. In this case, we put

∑_{x∈X} f(x) = ∑_{x∈X} Re f(x) + i ∑_{x∈X} Im f(x), (11.8.2)

where the sums on the right side are defined as in the preceding section.

If f, g are complex-valued summable functions on X, then f +g is summable on X too, as in Section 11.6. It is easy to see that

∑_{x∈X}(f(x) + g(x)) = ∑_{x∈X} f(x) + ∑_{x∈X} g(x). (11.8.3)

Of course, the real and imaginary parts of f + g are the same as the sums of the real and imaginary parts of f and g, respectively. Thus (11.8.3) follows from the definition (11.8.2) of the sum in the complex case and the analogous statement for the sum in the real case.

If f is a complex-valued summable function on X and t ∈ C, then t f is summable on X as well, as in Section 11.6 again. Let us check that

∑_{x∈X} t f(x) = t ∑_{x∈X} f(x). (11.8.4)

If t ∈ R, then this can be obtained from the definition (11.8.2) of the sum in the complex case and the analogous property of the sum in the real case. Similarly, if t is imaginary, then (11.8.4) can be reduced to the analogous statement in the real case. If t ∈ C, then (11.8.4) can be derived from the previous two cases, using (11.8.3).

Let f be a complex-valued summable function on X again, and observe that

|∑_{x∈X} f(x)| ≤ |∑_{x∈X} Re f(x)| + |∑_{x∈X} Im f(x)|, (11.8.5)

by the definition (11.8.2) of the sum. This implies that

|∑_{x∈X} f(x)| ≤ ∑_{x∈X} | Re f(x)| + ∑_{x∈X} | Im f(x)|, (11.8.6)

because of (11.7.5). It follows that

|∑_{x∈X} f(x)| ≤ 2 ∑_{x∈X} |f(x)|, (11.8.7)

using the first step in (11.8.1). Of course, we would rather have that

|∑_{x∈X} f(x)| ≤ ∑_{x∈X} |f(x)|. (11.8.8)

This corresponds to the ordinary triangle inequality for the standard absolute value function on C in the case of finite sums. In this situation, one can get (11.8.8) by approximating by finite sums. Some more details about this type of approximation will be given in the next section.

11.9 Some properties of the sum

Let f be a real or complex-valued function on a nonempty set X, and suppose for the moment that f has finite support in X. In this case, the sum ∑_{x∈X} f(x) can be defined as a real or complex number by reducing to a finite sum. If f is a nonnegative real-valued function on X, then we have seen that the definition of the sum in Section 11.2 reduces to the same finite sum. If f is a real or complex-valued function on X, then we can apply this to |f(x)|, to get that f is summable on X. If f is a real-valued function on X, then the functions defined on X in (11.7.1) have finite support. This implies that the definition of the sum in Section 11.7 is the same as the finite sum, because of the analogous statement for nonnegative real-valued functions on X with finite support. Similarly, if f is a complex-valued function on X, then the real and imaginary parts of f have finite support in X. It follows that the definition of the sum in the preceding section is the same as the finite sum, because of the analogous statement for real-valued functions on X with finite support. In particular, (11.8.8) holds, because of the ordinary triangle inequality for finite sums.

Now let f, g be real or complex-valued summable functions on X. Observe that

|∑_{x∈X} f(x) − ∑_{x∈X} g(x)| = |∑_{x∈X}(f(x) − g(x))| ≤ 2 ∑_{x∈X} |f(x) − g(x)|. (11.9.1)

This uses the linearity of the sum in the first step, and (11.8.7) in the second step. It follows that the mapping from a real or complex-valued summable function f on X to its sum ∑_{x∈X} f(x) is uniformly continuous as a mapping from ℓ^{1}(X, R), ℓ^{1}(X, C) into R, C, respectively. Here we use the corresponding ℓ^{1} metric on the domain, as in (11.6.6), and the standard Euclidean metric on the range.

One can use these properties of the sum to check that (11.8.8) holds for all complex-valued summable functions f on X. More precisely, if f has finite support in X, then we have already seen that (11.8.8) holds. Otherwise, if f is any complex-valued summable function on X, then f can be approximated by complex-valued functions on X with finite support with respect to the ℓ^{1} metric, as in Section 11.6. To get that f satisfies (11.8.8) as well, one can use the continuity condition (11.9.1).

Once we have that (11.8.8) holds for every complex-valued summable function on X, we get that

|∑_{x∈X} f(x) − ∑_{x∈X} g(x)| ≤ ∑_{x∈X} |f(x) − g(x)| (11.9.2)

for all complex-valued summable functions f, g on X. This is basically the same as (11.9.1), except that we use (11.8.8) in the second step. Of course, if f and g are real-valued summable functions on X, then we could already get this using (11.7.5).

Let f be a real or complex-valued summable function on X again. If E is a nonempty subset of X, then the restriction of f to E is summable as a real or complex-valued function on E, as appropriate. This follows from the analogous statement for nonnegative real-valued summable functions, which was mentioned in Section 11.4. In particular, this means that ∑_{x∈E} f(x) can be defined as a real or complex number, as appropriate, as in the previous two sections.

Note that

∑_{x∈E} f(x) = ∑_{x∈X} f(x) (11.9.3)

when f(x) = 0 for every x ∈ X \ E. This was mentioned in Section 11.3 when f is a nonnegative real-valued function on X. If f is a real-valued summable function on X with support contained in E, then the functions defined in (11.7.1) have support contained in E too. In this case, (11.9.3) can be reduced to the corresponding statement for nonnegative real-valued functions, because of the way that the sum was defined in Section 11.7. Similarly, if f is a complexvalued summable function on X with support contained in E, then (11.9.3) can be obtained from the corresponding statement for real-valued functions, applied to the real and imaginary parts of f.

If E_{1} and E_{2} are nonempty disjoint subsets of X, then

∑_{x∈E_{1}∪E_{2}} f(x) = ∑_{x∈E_{1}} f(x) + ∑_{x∈E_{2}} f(x). (11.9.4)

This was mentioned in Section 11.5 when f is nonnegative. If f is any real or complex-valued summable function on X, then (11.9.4) can be obtained from the linearity of the sum, by expressing f on E_{1} ∪E_{2} as a sum of functions supported in E_{1} and E_{2}, as before. Alternatively, if f is a real-valued summable function on X, then one can reduce to the case of nonnegative real-valued summable functions on X, because of the way that the sum is defined in Section 11.7. Similarly, if f is a complex-valued summable function on X, then one can apply the previous statement to the real and imaginary parts of f.

11.10 Generalized convergence

Let f be a real or complex-valued function on a nonempty set X. The sum

∑_{x∈X} f(x) (11.10.1)

is said to converge in the generalized sense if there is a real or complex number λ, as appropriate, such that for every ϵ > 0 there is a nonempty finite subset A_{ϵ} of X with the property that

|∑_{x∈A} f(x) − λ| < ϵ (11.10.2)

for every nonempty finite subset A of X with A_{ϵ} ⊆ A. In this case, the value of the sum (11.10.1) is defined to be λ. More precisely, one can check that λ is unique when it exists. Of course, if X has only finitely many elements, then one can take A_{ϵ} = X for every ϵ > 0, to get that this condition holds trivially with λ equal to the finite sum (11.10.1).

Suppose that f is summable on X, and let us check that the sum (11.10.1) converges in the generalized sense, with the same value of the sum as defined in Sections 11.2, 11.7, and 11.8. This is trivial when X has only finitely many elements, and so we suppose now that X is an infinite set. If A is a nonempty finite subset of X, then

∑_{x∈X} f(x) = ∑_{x∈A} f(x) + ∑_{x∈X\A} f(x), (11.10.3)

as in (11.9.4). Note that X \ A ≠ ∅ in this situation, and that the restriction of f to X \ A is summable, as in the previous section. It follows that

|∑_{x∈A} f(x) − ∑_{x∈X} f(x)| = | ∑_{x∈X\A} f(x)| ≤ ∑_{x∈X\A} |f(x)|, (11.10.4)

using (11.7.5) or (11.8.8) in the second step, as appropriate. Remember that for each ϵ > 0 there is a nonempty finite subset A(ϵ) of X such that

∑_{x∈X\A(ϵ)} |f(x)| < ϵ, (11.10.5)

as in (11.4.10). If A(ϵ) ⊆ A, then X \ A ⊆ X \ A(ϵ), and hence

∑_{x∈X\A} |f(x)| ≤ ∑_{x∈X\A(ϵ)} |f(x)|, (11.10.6)

as in (11.3.4). Thus

|∑_{x∈A} f(x) − ∑_{x∈X} f(x)| ≤ ∑_{x∈X\A} |f(x)| ≤ ∑_{x∈X\A(ϵ)} |f(x)| < ϵ (11.10.7)

when A is a nonempty finite subset of X with A(ϵ) ⊆ A, as desired.

Alternatively, if f is a nonnegative real-valued summable function on X, then the convergence of the sum (11.10.1) in the generalized sense can be obtained from (11.4.7). If f is a real-valued summable function on X, then f can be expressed as a difference of nonnegative real-valued summable functions on X, as in (11.7.3). In this case, the convergence of the sum (11.10.1) in the generalized sense can be reduced to the previous statement. Similarly, if f is a complexvalued summable function on X, then the real and imaginary parts of f are summable on X too. This permits one to reduce the convergence of (11.10.1) in the generalized sense to the analogous statements for the real and imaginary parts of f.

Let f be a real or complex-valued function on X again, and suppose that the sum (11.10.1) converges in the generalized sense. Applying the earlier definition with ϵ = 1, we get that there is a real or complex number λ, as appropriate, and a nonempty finite subset A_{1} of X such that

|∑_{x∈A} f(x) − λ| < 1 (11.10.8)

for every nonempty finite subset A of X with A_{1} ⊆ A. In particular,

|∑_{x∈A} f(x)| < |λ| + 1 (11.10.9)

for every nonempty finite subset A of X with A_{1} ⊆ A. Let us check that

|∑_{x∈B} f(x)| ≤ |λ| + 1 + ∑_{x∈A_{1}} |f(x)| (11.10.10)

for every nonempty finite subset B of X. Of course, (11.10.10) follows from (11.10.9) when A_{1} ⊆ B. Otherwise, we can apply (11.10.9) to A = B ∪ A_{1}, to get that

| ∑_{x∈B∪A_{1}} f(x)| < |λ| + 1. (11.10.11)

This implies that

(11.10.12)  |∑_{x∈B} f(x)| = | ∑_{x∈B∪A_{1}} f(x) − ∑_{x∈A_{1}\B} f(x)|

≤ | ∑_{x∈B∪A_{1}} f(x)| + | ∑_{x∈A_{1}\B} f(x)|
≤ |λ| + 1 + ∑_{x∈A_{1}\B} |f(x)|.

This shows that (11.10.10) also holds when A_{1} \ B ≠ ∅, as desired.

If f is a nonnegative real-valued function on X, then (11.10.10) implies that f is summable on X. If f is a real-valued function on X, then (11.10.10) implies that the positive and negative parts of f are summable on X. More precisely, one can get this by applying (11.10.10) to nonempty finite subsets B of X on which f has constant sign. It follows that f is summable on X too in this case. If f is a complex-valued function on X, then one can use (11.10.10) to get that the real and imaginary parts of f are summable on X, and hence that f is summable on X.

11.11 Compositions and sums

Let X and Y be nonempty sets, and let ϕ be a one-to-one mapping from X onto Y. Also let f be a real or complex-valued function on Y, so that f(ϕ(x)) defines a real or complex-valued function on X, as appropriate. Note that

∑_{x∈X} |f(ϕ(x))| = ∑_{y∈Y} |f(y)|, (11.11.1)

as in (11.3.2). Thus f is summable on Y if and only if f(ϕ(x)) is summable on X. Let us check that

∑_{x∈X} f(ϕ(x)) = ∑_{y∈Y} f(y) (11.11.2)

in this case. If f is a nonnegative real-valued function on Y, then (11.11.2) is the same as (11.11.1). If f is a real-valued function on Y, then one can get (11.11.2) by expressing f as a difference of nonnegative real-valued summable functions on Y, as in Section 11.7. If f is a complex-valued summable function on Y, then (11.11.2) can be obtained by applying the previous statement to the real and imaginary parts of f.

Similarly, if f is a real or complex-valued function on Y, then one can check directly that ∑_{x∈X} f(ϕ(x)) converges in the generalized sense if and only if ∑_{y∈Y} f(y) converges in the generalized sense, with the same value of the sums. This uses the fact that A ↦ ϕ(A) defines a one-to-one correspondence between nonempty finite subsets of X and nonempty finite subsets of Y.

Now let X be a countably infinite set, and let {x_{j}}_{j=1}^{∞} be a sequence of elements of X in which every element of X occurs exactly once. Thus j ↦ x_{j} is a one-to-one mapping from the set Z_{+} of positive integers onto X. Let f be a real or complex-valued function on X, so that f(x_{j}) may be considered as a real or complex-valued function of j ∈ Z_{+}, as appropriate. As before, f is summable on X if and only if f(x_{j}) is summable on Z_{+}, in which case

∑_{j∈Z_{+}} f(x_{j}) = ∑_{x∈X} f(x). (11.11.3)

Alternatively, the sum on the left converges in the generalized sense if and only if the sum on the right converges in the generalized sense, with the same value of the sum, as in the preceding paragraph.

Remember that

∑_{j=1}^{∞} |f(x_{j})| = ∑_{j∈Z_{+}} |f(x_{j})|, (11.11.4)

as in (11.2.4). Thus f(x_{j}) is summable on Z_{+} if and only if the infinite series on the left side of (11.11.4) converges in the usual sense, which means that ∑_{j=1}^{∞} f(x_{j}) converges absolutely. Under these conditions,

∑_{j=1}^{∞} f(x_{j}) = ∑_{j∈Z_{+}} f(x_{j}). (11.11.5)

More precisely, (11.11.5) is the same as (11.11.4) when f is real-valued and nonnegative. If f is real-valued, then (11.11.5) can be obtained from the previous statement by expressing f as a difference of nonnegative real-valued summable functions. If f is complex-valued, then one can get (11.11.5) by considering the real and imaginary parts separately. Alternatively, if the sum on the right side of (11.11.5) converges in the generalized sense, then it is easy to see that the sum on the left side of (11.11.5) converges, and with the same value of the sum.

11.12 Completeness of ℓ^{1} spaces

11.12. COMPLETENESS OF ℓ^{1} SPACES 217

Let X be a nonempty set, and let {f_{j}}_{j=1}^{∞} be a sequence of real or complexvalued functions on X that converge pointwise to a real or complex-valued function f on X, as appropriate. Suppose also that the f_{j}’s are summable on X, with bounded ℓ^{1} norms, so that there is a nonnegative real number C such that

∑_{x∈X} |f_{j}(x)| ≤ C (11.12.1)

for every j ≥ 1. We would like to show that f is summable on X too under these conditions, with

∑_{x∈X} |f(x)| ≤ C. (11.12.2)

If A is any nonempty finite subset of X, then

∑_{x∈A} |f(x)| = lim_{j→∞} ∑_{x∈A} |f_{j}(x)| ≤ C, (11.12.3)

using basic properties of limits in the first step, and (11.12.1) in the second step. This implies (11.12.2), as desired.

Let us now check that ℓ^{1}(X, R) and ℓ^{1}(X, C) are complete as metric spaces, with respect to the ℓ^{1} metric (11.6.6). Let {f_{j}}_{j=1}^{∞} be a sequence of real or complex-valued summable functions on X that is a Cauchy sequence with respect to the ℓ^{1} metric. This means that for each ϵ > 0 there is an L(ϵ) ∈ Z_{+} such that

∑_{x∈X} |f_{j}(x) − f_{l}(x)| = ∥f_{j} − f_{l}∥_{1} < ϵ (11.12.4)

for every j, l ≥ L(ϵ). In particular, if x ∈ X, then it follows that

|f_{j}(x) − f_{l}(x)| < ϵ (11.12.5)

for every j, l ≥ L(ϵ). Thus {f_{j}(x)}_{j=1}^{∞} is a Cauchy sequence in R or C, as appropriate, with respect to the standard Euclidean metric.

It is well known that R and C are complete as metric spaces with respect to the corresponding Euclidean metric. Thus {f_{j}(x)}_{j=1}^{∞} converges in R or C, as appropriate, for each x ∈ X. Put

f(x) = lim_{j→∞} f(x) (11.12.6)

for every x ∈ X, which defines f as a real or complex-valued function on X, as appropriate. We would like to show that f is summable on X, and that {f_{j}}_{j=1}^{∞} converges to f with respect to the ℓ^{1} metric. Of course, if X has only finitely many elements, then this follows easily from pointwise convergence.

Let ϵ > 0 and l ≥ L(ϵ) be given, and consider {f_{j} − f_{l}}_{j=L(ϵ)}^{∞} as a sequence of summable functions on X that converges pointwise to f − f_{l}. Using (11.12.4) and the remarks at the beginning of the section, we get that f − f_{l} is summable on X, with

∑_{x∈X} |f(x) − f_{l}(x)| ≤ ϵ. (11.12.7)

In particular, f is summable on X, because f_{l} is summable on X. More precisely, we can simply take ϵ = 1 and l = L(1) in this step. It follows from (11.12.7) that {f_{j}}_{j=1}^{∞} converges to f with respect to the ℓ^{1} metric, as desired.

11.13 Monotone convergence

Let X be a nonempty set, and let {f_{j}}_{j=1}^{∞} be a sequence of nonnegative realvalued functions on X. Suppose that the f_{j}’s are monotonically increasing in j, so that f_{j}(x) ≤ f_{j+1}(x) (11.13.1)

for every x ∈ X and j ≥ 1. Put

f(x) = sup_{j≥1} f_{j}(x) (11.13.2)

for each x ∈ X, which defines f as a nonnegative extended real-valued function on X. Equivalently, this means that

f_{j}(x) → f(x) as j → ∞ (11.13.3)

for every x ∈ X, as in Section 11.1. Note that

∑_{x∈X} f_{j}(x) ≤ ∑_{x∈X} f_{j+1}(x) (11.13.4)

for every j ≥ 1, because of (11.13.1), as in (11.3.11).

Similarly,

∑_{x∈X} f_{j}(x) ≤ ∑_{x∈X} f(x) (11.13.5)

for every j ≥ 1, because f_{j}(x) ≤ f(x) for every x ∈ X and j ≥ 1, by construction. Thus

sup_{j≥1} ( ∑_{x∈X} f_{j}(x)) ≤ ∑_{x∈X} f(x). (11.13.6)

We would like to show that

sup_{j≥1} ( ∑_{x∈X} f_{j}(x)) = ∑_{x∈X} f(x) (11.13.7)

under these conditions. This is basically the same as saying that

∑_{x∈X} f_{j}(x) → ∑_{x∈X} f(x) as j → ∞, (11.13.8)

because of (11.13.4). More precisely, if f_{j} is summable on X for each j ≥ 1, then (11.13.7) is equivalent to (11.13.8), as in Section 11.1. Otherwise, if f_{j} is not summable on X for some j ≥ 1, then f_{j} is not summable on X for all sufficiently large j, because of (11.13.1). In this case, it is easy to see that f is not summable on X, so that (11.13.7) holds, and that (11.13.8) holds in a suitable sense.

In order to show (11.13.7) or (11.13.8), let a be a real number such that

a < ∑_{x∈X} f(x). (11.13.9)

By definition of the sum on the right, there is a nonempty finite subset A of X such that

a < ∑_{x∈A} f(x). (11.13.10)

Observe that

∑_{x∈A} f_{j}(x) → ∑_{x∈A} f(x) as j → ∞, (11.13.11)

because of (11.13.3). It follows that

a < ∑_{x∈A} f_{j}(x) (11.13.12)

for all but finitely many j ≥ 1. This implies that

a < ∑_{x∈X} f_{j}(x) (11.13.13)

for the same j’s, and hence all but finitely many j ≥ 1. Using this, one can check that (11.13.7) or (11.13.8) holds, as desired. This is the monotone convergence theorem for sums. Of course, if X has only finitely many elements, then this is more elementary, as in (11.13.11).

11.14 Dominated convergence

Let X be a nonempty set, and let {f_{j}}_{j=1}^{∞} be a sequence of real or complexvalued functions on X that converges pointwise to a real or complex-valued function f on X, as appropriate. Suppose that there is a nonnegative realvalued summable function g on X such that

|f_{j}(x)| ≤ g(x) (11.14.1)

for every x ∈ X and j ≥ 1. This implies that

|f(x)| ≤ g(x) (11.14.2)

for every x ∈ X, because {f_{j}(x)}_{j=1}^{∞} converges to f(x), by hypothesis. In particular, f_{j} is summable on X for each j ≥ 1, because of (11.14.1), and f is summable on X too, because of (11.14.2). We would like to show that

lim_{j→∞} ∑_{x∈X} |f_{j}(x) − f(x)| = 0 (11.14.3)

under these conditions. Of course, if X has only finitely many elements, then this follows from standard results about sums of convergent sequences. Thus we may suppose that X has infinitely many elements.

Let ϵ > 0 be given. Because g is summable on X, there is a nonempty finite subset A_{0} of X such that

∑_{x∈X\A_{0}} g(x) < ϵ/3, (11.14.4)

as in (11.4.10). Note that

|f_{j}(x) − f(x)| ≤ |f_{j}(x)| + |f(x)| ≤ 2 g(x) (11.14.5)

for every x ∈ X and j ≥ 1, by (11.14.1) and (11.14.2). It follows that

∑_{x∈X\A_{0}} |f_{j}(x) − f(x)| ≤ ∑_{x∈X\A_{0}} 2 g(x) < 2 ϵ/3 (11.14.6)

for every j ≥ 1.

As before,

lim_{j→∞} ∑_{x∈A_{0}} |f_{j}(x) − f(x)| = 0, (11.14.7)

because A_{0} has only finitely many elements. Hence there is an L ∈ Z_{+} such that

∑_{x∈A_{0}} |f_{j}(x) − f(x)| < ϵ/3 (11.14.8)

for every j ≥ L. Combining this with (11.14.6), we get that

∑_{x∈X} |f_{j}(x) − f(x)| = ∑_{x∈A_{0}} |f_{j}(x) − f(x)| + ∑_{x∈X\A_{0}} |f_{j}(x) − f(x)|
(11.14.9)  < ϵ/3 + 2 ϵ/3 = ϵ

for every j ≥ L, as desired.

Observe that

(11.14.10)  |∑_{x∈X} f_{j}(x) − ∑_{x∈X} f(x)| = |∑_{x∈X}(f_{j}(x) − f(x))|
≤ ∑_{x∈X} |f_{j}(x) − f(x)|

for every j ≥ 1. This uses (11.7.5) or (11.8.8), as appropriate, in the second step. Thus (11.14.3) implies that

lim_{j→∞} ∑_{x∈X} f_{j}(x) = ∑_{x∈X} f(x). (11.14.11)

The fact that this holds under these conditions is the dominated convergence theorem for sums. If X has only finitely many elements, then this is a standard result about sums of convergent sequences, as before.

11.15 Nonnegative sums of sums

Let X be a nonempty set, and let f be a nonnegative real-valued function on X. Also let I be a nonempty set, and let E_{j} be a nonempty subset of X for each j ∈ I. Suppose that E_{j} ∩ E_{l} = ∅ (11.15.1) for every j, l ∈ I with j ≠ l, and put

E = ⋃_{j∈I} E_{j}. (11.15.2)

As usual,

∑_{x∈E_{j}} f(x) (11.15.3)

is defined as a nonnegative extended real number for each j ∈ I, so that (11.15.3) may be considered as a nonnegative extended real-valued function of j on I. Thus

∑_{j∈I} ( ∑_{x∈E_{j}} f(x)) (11.15.4)

can also be defined as a nonnegative extended real number, as in Section 11.2. Of course,

∑_{x∈E} f(x) (11.15.5)

can be defined as a nonnegative extended real number as well. We would like to show that (11.15.4) is equal to (11.3.3). This is elementary when E is a finite set, which is the same as saying that I is a finite set, and that E_{j} is a finite set for each j ∈ I.

Let us first verify that (11.15.5) is less than or equal to (11.15.4). Let A be a nonempty finite subset of E. Put

I_{A} = {j ∈ I : A ∩ E_{j} ≠ ∅}, (11.15.6)

which is a nonempty finite subset of I, and observe that

A = ⋃_{j∈I_{A}} A ∩ E_{j}. (11.15.7)

Thus

∑_{x∈A} f(x) = ∑_{j∈I_{A}} ( ∑_{x∈A∩E_{j}} f(x)), (11.15.8)

as mentioned earlier. If j ∈ I_{A}, then

∑_{x∈A∩E_{j}} f(x) ≤ ∑_{x∈E_{j}} f(x), (11.15.9)

because A ∩ E_{j} ⊆ E_{j}. Combining this with (11.15.8), we get that

∑_{x∈A} f(x) ≤ ∑_{j∈I_{A}} ( ∑_{x∈E_{j}} f(x)). (11.15.10)

It follows that

∑_{x∈A} f(x) ≤ ∑_{j∈I} ( ∑_{x∈E_{j}} f(x)), (11.15.11)

because I_{A} ⊆ I. This implies that

∑_{x∈E} f(x) ≤ ∑_{j∈I} ( ∑_{x∈E_{j}} f(x)), (11.15.12)

by the definition of the sum over E on the left.

Now let us check that (11.15.4) is less than or equal to (11.15.5). Let I_{0} be a nonempty finite subset of I, and put

E(I_{0}) = ⋃_{j∈I_{0}} E_{j}. (11.15.13)

Observe that

∑_{j∈I_{0}} ( ∑_{x∈E_{j}} f(x)) = ∑_{x∈E(I_{0})} f(x). (11.15.14)

This follows from the analogous statement (11.5.13) for the union of two disjoint nonempty subsets of X. Hence

∑_{j∈I_{0}} ( ∑_{x∈E_{j}} f(x)) ≤ ∑_{x∈E} f(x), (11.15.15)

because E(I_{0}) ⊆ E. It follows that

∑_{j∈I} ( ∑_{x∈E} f(x)) ≤ ∑_{x∈E} f(x), (11.15.16)

by the definition of the sum over I on the left. Combining (11.15.12) and (11.15.16), we get that (11.15.4) is equal to (11.15.5), as desired.

11.16 Real and complex sums

Let X be a nonempty set again, and let f be a real or complex-valued summable function on X. As in the previous section, we let I be a nonempty set, and we let E_{j} be a nonempty subset of X for each j ∈ I. We suppose that the E_{j}’s are pairwise disjoint, as in (11.15.1), and we let E be their union, as in (11.15.2). The sum (11.15.3) of f(x) over x ∈ E_{j} is now defined as a real or complex number for each j ∈ I, as appropriate. Similarly, the sum (11.15.5) of f(x) over x ∈ E is defined as a real or complex number, as appropriate.

If j ∈ I, then

| ∑_{x∈E_{j}} f(x)| ≤ ∑_{x∈E_{j}} |f(x)|, (11.16.1)

as in (11.7.5) or (11.8.8), as appropriate. Thus

∑_{j∈I}| ∑_{x∈E_{j}} f(x)| ≤ ∑_{j∈I} ( ∑_{x∈E_{j}} |f(x)|), (11.16.2)

where these sums over I are defined as nonnegative extended real numbers in the usual way. The remarks in the previous section imply that

∑_{j∈I} ( ∑_{x∈E_{j}} |f(x)|) = ∑_{x∈E} |f(x)|. (11.16.3)

We also have that

∑_{x∈E} |f(x)| ≤ ∑_{x∈X} |f(x)| < ∞, (11.16.4)

because f is summable on X, by hypothesis. It follows that

∑_{j∈I}| ∑_{x∈E_{j}} f(x)| < ∞. (11.16.5)

This means that the sum (11.15.3) of f(x) over x ∈ E_{j} is summable as a real or complex-valued function of j ∈ I, as appropriate. Hence the sum (11.15.4) of this sum over j ∈ I can be defined as a real or complex number, as appropriate. We would like to check that this sum is equal to the sum (11.15.5) of f(x) over x ∈ E. If f is a nonnegative real-valued function on X, then this follows from the remarks in the previous section. If f is a real-valued summable function on X, then f can be expressed as the difference of two nonnegative real-valued summable functions on X, as in (11.7.3). In this case, the equality of (11.15.4) and (11.15.5) follows from the analogous statement for nonnegative real-valued summable functions on X. If f is a complex-valued summable function on X, then one can apply the previous statement to the real and imaginary parts of f.

Let Y and Z be nonempty sets, and suppose that X = Y × Z is their Cartesian product. If f(y, z) is a nonnegative real-valued function on Y × Z, then

∑_{z∈Z} f(y, z) (11.16.6)

defines a nonnegative extended real-valued function of y ∈ Y, and

∑_{y∈Y} f(y, z) (11.16.7)

defines a nonnegative extended real-valued function of z ∈ Z. Thus

∑_{y∈Y} ( ∑_{z∈Z} f(y, z)) (11.16.8)

and

∑_{z∈Z} ( ∑_{y∈Y} f(y, z)) (11.16.9)

are defined as nonnegative extended real numbers, as is

∑_{(y,z)∈Y ×Z} f(y, z). (11.16.10)

The remarks in the previous section imply that (11.16.8) is equal to (11.16.10). This corresponds to expressing Y × Z as the union of the pairwise-disjoint nonempty subsets of the form {y} × Z, with y ∈ Y. Similarly, (11.16.9) is equal to (11.16.10), which corresponds to expressing Y ×Z as the uion of the pairwisedisjoint nonempty subsets of the form Y × {z}, with z ∈ Z. In particular, it follows that (11.16.8) is equal to (11.16.9).

Suppose now that f(y, z) is a real or complex-valued summable function on Y ×Z. In this case, (11.16.6) defines a real or complex-valued function of y ∈ Y, and (11.16.7) defines a real or complex-valued function of z ∈ Z, as appropriate. These functions are summable on Y and Z, respectively, as in (11.16.5). Hence (11.16.8) and (11.16.9) are defined as real or complex numbers, as appropriate. The sum (11.16.10) is also defined as a real or complex number, as appropriate, and it is equal to (11.16.8) and (11.16.9), as before.

11.17 Square-summable functions

Let X be a nonempty set, and let f be a real or complex-valued function on X. If |f(x)|^{2} is summable on X, then we say that f is square-summable on X. In this case, we put

∥f∥_{2} = ( ∑_{x∈X} |f(x)|^{2})^{1/2}, (11.17.1)

using the nonnegative square root on the right side, as usual. If t ∈ R or C, as appropriate, then t f is square-summable on X too, because |t f(x)|^{2} = |t|^{2} |f(x)|^{2} is summable on X. We also have that

∥t f∥_{2} = ( ∑_{x∈X} |t|^{2} |f(x)|^{2})^{1/2} = |t| ∥f∥_{2}. (11.17.2)

Remember that

a b ≤ \frac{1}{2} (a^{2} + b^{2}) (11.17.3)

for all nonnegative real numbers a, b, as in (2.3.6). Let g be another real or complex-valued square-summable function on X, as appropriate. Observe that

|f(x)| |g(x)| ≤ \frac{1}{2} (|f(x)|^{2} + |g(x)|^{2}) (11.17.4)

for every x ∈ X, by (11.17.3). This implies that

∑_{x∈X} |f(x)| |g(x)| ≤ ∑_{x∈X} \frac{1}{2} (|f(x)|^{2} + |g(x)|^{2})

(11.17.5)  = \frac{1}{2} ∑_{x∈X} |f(x)|^{2} + \frac{1}{2} ∑_{x∈X} |g(x)|^{2} = \frac{1}{2} ∥f∥_{2}^{2} + \frac{1}{2} ∥g∥_{2}^{2}.

In particular, |f(x)| |g(x)| is summable on X when f and g are square-summable on X.

More precisely,

∑_{x∈X} |f(x)| |g(x)| ≤ ∥f∥_{2} ∥g∥_{2} (11.17.6)

when f and g are square-summable on X. This is another version of the Cauchy– Schwarz inequality. If ∥f∥_{2} = ∥g∥_{2} = 1, then (11.17.6) follows from (11.17.5). If ∥f∥_{2}, ∥g∥_{2} > 0, then one can reduce to the previous case, using (11.17.2). Otherwise, if f(x) = 0 for every x ∈ X, or g(x) = 0 for every x ∈ X, then (11.17.6) is trivial.

Note that

(11.17.7)  |f(x) + g(x)|^{2} ≤ (|f(x)| + |g(x)|)^{2}
= |f(x)|^{2} + 2 |f(x)| |g(x)| + |g(x)|^{2}

for every x ∈ X. Hence

∑_{x∈X} |f(x) + g(x)|^{2} (11.17.8)

≤ ∑_{x∈X} |f(x)|^{2} + 2 ∑_{x∈X} |f(x)| |g(x)| + ∑_{x∈X} |g(x)|^{2}.

If f and g are square-summable on X, then it follows that f + g is squaresummable on X, because |f(x)| |g(x)| is summable on X, as before. Using (11.17.6), we get that

∥f + g∥_{2}^{2} ≤ ∥f∥_{2}^{2} + 2 ∥f∥_{2} ∥g∥_{2} + ∥g∥_{2}^{2} = (∥f∥_{2} + ∥g∥_{2})^{2}. (11.17.9)

Equivalently, this means that

∥f + g∥_{2} ≤ ∥f∥_{2} + ∥g∥_{2}. (11.17.10)

11.18 ℓ^{2} Spaces

Let X be a nonempty set again, and let ℓ^{2}(X, R) and ℓ^{2}(X, C) be the spaces of real and complex-valued square-summable functions on X, respectively. These are linear subspaces of the real and complex vector spaces of all real and complex-valued functions on X, respectively, as in the previous section. Note that (11.17.1) defines a norm on each of ℓ^{2}(X, R) and ℓ^{2}(X, C), because of (11.17.2), (11.17.10), and the fact that ∥f∥_{2} = 0 if and only if f(x) = 0 for every x ∈ X. This implies that

d_{2}(f, g) = ∥f − g∥_{2} (11.18.1)

defines a metric on each of ℓ^{2}(X, R) and ℓ^{2}(X, C), as usual.

If f is a real or complex-valued square-summable function on X, then

|f(x)| ≤ ∥f∥_{2} (11.18.2) for every x ∈ X, by the definition (11.17.1) of ∥f∥_{2}. It follows that f is bounded on X, with ∥f∥_{∞} ≤ ∥f∥_{2}. (11.18.3)

Here ∥f∥_{∞} denotes the supremum norm of f, as in (1.13.4). If g is another real or complex-valued square-summable function on X, as appropriate, then we get that d_{∞}(f, g) ≤ d_{2}(f, g), (11.18.4)

where d_{∞}(f, g) is the supremum metric, as in (1.13.8).

Let f be a real or complex-valued square-summable function on X again, so that |f|^{2} is summable on X. This implies that |f|^{2} vanishes at infinity on X, as in Section 11.4. Using this, it is easy to see that f vanishes at infinity on X as well. Thus

ℓ^{2}(X, R) ⊆ c_{0}(X, R), ℓ^{2}(X, C) ⊆ c_{0}(X, C). (11.18.5)

Let f be a real or complex-valued summable function on X, and remember that f is bounded on X, as in Section 11.6. Observe that

∑_{x∈X} |f(x)|^{2} ≤ ∥f∥_{∞} ∑_{x∈X} |f(x)| = ∥f∥_{∞} ∥f∥_{1} ≤ ∥f∥_{1}^{2}, (11.18.6)

where ∥f∥_{1} is the ℓ^{1} norm of f, as in (11.6.3). This implies that f is squaresummable on X, with ∥f∥_{2} ≤ ∥f∥_{1}. (11.18.7)

Hence ℓ^{1}(X, R) ⊆ ℓ^{2}(X, R), ℓ^{1}(X, C) ⊆ ℓ^{2}(X, C). (11.18.8)

If g is another real or complex-valued summable function on X, as appropriate, then we have that d_{2}(f, g) ≤ d_{1}(f, g), (11.18.9)

where d_{1}(f, g) is the ℓ^{1} metric, as in (11.6.6).

In particular, real or complex-valued functions on X with finite support are square-summable. Let us verify that c_{00}(X, R), c_{00}(X, C) are dense in ℓ^{2}(X, R), ℓ^{2}(X, C), respectively, with respect to the ℓ^{2} metric (11.18.1). This is very similar to the analogous argument for ℓ^{1} spaces, in Section 11.6. As before, there is nothing to do when X has only finitely many elements, and so we suppose that X is an infinite set. Let f be a real or complex-valued squaresummable function on X, and let ϵ > 0 be given. Because |f|^{2} is summable on X, there is a finite subset A(ϵ) of X such that

∑_{x∈X\A(ϵ)} |f(x)|^{2} < ϵ^{2}, (11.18.10)

as in (11.4.10). Let f_{ϵ} be the real or complex-valued function on X, as appropriate, that is equal to f on A(ϵ) and to 0 on X \ A(ϵ). Thus f_{ϵ} has finite support in X, and f − f_{ϵ} is equal to 0 on A(ϵ), and to f on X \ A(ϵ). It follows that

∑_{x∈X} |f(x) − f_{ϵ}(x)|^{2} = ∑_{x∈X\A(ϵ)} |f(x)|^{2} < ϵ^{2}, (11.18.11)

so that ∥f − f_{ϵ}∥_{2} < ϵ, (11.18.12)

as desired.

11.19 Completeness of ℓ^{2} spaces

Let X be a nonempty set, and let {f_{j}}_{j=1}^{∞} be a sequence of real or complexvalued functions on X that converges pointwise to a real or complex-valued function f on X, as appropriate. Suppose that the f_{j}’s are square-summable on X, with bounded ℓ^{2} norms, so that

∥f_{j}∥_{2} ≤ C (11.19.1)

for some nonnegative real number C and every j ≥ 1. Equivalently, this means that

∑_{x∈X} |f_{j}(x)|^{2} ≤ C^{2} (11.19.2)

for every j ≥ 1. Of course, {|f_{j}(x)|^{2}}_{j=1}^{∞} converges to |f(x)|^{2} with respect to the standard Euclidean metric on R for every x ∈ X, by well-known results about convergent sequences of real or complex numbers. Under these conditions, f is a square-summable function on X, with

∑_{x∈X} |f(x)|^{2} ≤ C^{2}. (11.19.3)

This follows from the remarks at the beginning of Section 11.12, applied to |f_{j}|^{2}. Of course, (11.19.3) is the same as saying that

∥f∥_{2} ≤ C. (11.19.4)

We would like to show that ℓ^{2}(X, R), ℓ^{2}(X, C) are complete with respect to the ℓ^{2} metric (11.18.1). Let {f_{j}}_{j=1}^{∞} be a sequence of real or complex-valued square-summable functions on X that is a Cauchy sequence with respect to the ℓ^{2} metric. Thus for each ϵ > 0 there is an L(ϵ) ∈ Z_{+} such that

∥f_{j} − f_{l}∥_{2} < ϵ (11.19.5)

for every j, l ≥ L(ϵ). If x ∈ X, then it follows that

|f_{j}(x) − f_{l}(x)| < ϵ (11.19.6) for every j, l ≥ L(ϵ), by (11.18.2). This implies that {f_{j}(x)}_{j=1}^{∞} is a Cauchy sequence in R or C, as appropriate, with respect to the standard Euclidean metric.

Hence {f_{j}(x)}_{j=1}^{∞} converges in R or C, as appropriate, for every x ∈ X, because R and C are complete as metric spaces. Let f(x) be the limit of this sequence for each x ∈ X, so that f defines a real or complex-valued function on X, as appropriate. We want to verify that f is square-summable on X, and that {f_{j}}_{j=1}^{∞} converges to f with respect to the ℓ^{2} metric. If X has only finitely many elements, then f is automatically square-summable on X, and convergence with respect to the ℓ^{2} metric can be obtained from pointwise convergence.

Let ϵ > 0 and l ≥ L(ϵ) be given, and let us consider {f_{j} − f_{l}}_{j=L(ϵ)}^{∞} as a sequence of square-summable functions on X that converges to f − f_{l} pointwise on X. The remarks at the beginning of the section imply that f − f_{l} is squaresummable on X, with ∥f − f_{l}∥_{2} ≤ ϵ, (11.19.7)

because of (11.19.5). In particular, this holds with ϵ = 1 and l = L(1). This implies that f is square-summable on X, because f_{L(1)} is square-summable on X. Using (11.19.7) again, we get that {f_{j}}_{j=1}^{∞} converges to f with respect to the ℓ^{2} metric, as desired.

11.20 Inner products on ℓ^{2} spaces

Let X be a nonempty set, and let f, g be square-summable real-valued functions on X. Under these conditions, |f(x)| |g(x)| is summable as a nonnegative real-valued function on X, as in Section 11.17. This means that f(x) g(x) is summable as a real-valued function on X, so that

⟨f, g⟩ = ⟨f, g⟩_{ℓ^{2}(X,R)} = ∑_{x∈X} f(x) g(x) (11.20.1)

can be defined as a real number, as in Section 11.7. This is the standard inner product on ℓ^{2}(X, R). In particular,

⟨f, f⟩_{ℓ^{2}(X,R)} = ∑_{x∈X} f(x)^{2} = ∥f∥_{2}^{2}, (11.20.2)

where ∥f∥_{2} is the ℓ^{2} norm of f, as in (11.17.1). As before,

|⟨f, g⟩_{ℓ^{2}(X,R)}| = |∑_{x∈X} f(x) g(x)| ≤ ∑_{x∈X} |f(x)| |g(x)| ≤ ∥f∥_{2} ∥g∥_{2}, (11.20.3)

using the Cauchy–Schwarz inequality (11.17.6) in the third step. Of course, it is much easier to define this inner product on the space c_{00}(X, R) of real-valued functions on X with finite support, by reducing the sum on the right side of (11.20.1) to a finite sum.

11.20. INNER PRODUCTS ON ℓ^{2} SPACES 229

Note that

⟨f, g⟩_{ℓ^{2}(X,R)} = ∑_{x∈X} f(x) g(x) = ∑_{x∈X} g(x) f(x) = ⟨g, f⟩_{ℓ^{2}(X,R)}. (11.20.4)

If t ∈ R, then t f is square-summable on X too, and

(11.20.5)  ⟨t f, g⟩_{ℓ^{2}(X,R)} = ∑_{x∈X} t f(x) g(x)
= t ∑_{x∈X} f(x) g(x) = t ⟨f, g⟩_{ℓ^{2}(X,R)}.

If f_{1} and f_{2} are square-summable real-valued functions on X, then we have seen in Section 11.17 that f_{1} + f_{2} is square-summable on X as well. In this case, we have that

(11.20.6) ⟨f_{1} + f_{2}, g⟩_{ℓ^{2}(X,R)} = ∑_{x∈X}(f_{1}(x) + f_{2}(x)) g(x)
= ∑_{x∈X} f_{1}(x) g(x) + ∑_{x∈X} f_{2}(x) g(x)
= ⟨f_{1}, g⟩_{ℓ^{2}(X,R)} + ⟨f_{2}, g⟩_{ℓ^{2}(X,R)},

using the linearity of the sum in the second step. The analogous linearity properties of the inner product (11.20.1) in g can be obtained in the same way, or using (11.20.4).

Now let f, g be square-summable complex-valued functions on X. As before, |f(x)| |g(x)| is summable as a nonnegative real-valued function on X, so that f(x) \overline{g(x)} is summable as a complex-valued function on X. Hence

⟨f, g⟩ = ⟨f, g⟩_{ℓ^{2}(X,C)} = ∑_{x∈X} f(x) \overline{g(x)} (11.20.7)

can be defined as a complex number, as in Section 11.8, which is the standard inner product on ℓ^{2}(X, C). As usual,

⟨f, f⟩_{ℓ^{2}(X,C)} = ∑_{x∈X} |f(x)|^{2} = ∥f∥_{2}^{2}, (11.20.8)

where ∥f∥_{2} is the ℓ^{2} norm of f. We also have that

|⟨f, g⟩_{ℓ^{2}(X,C)}| = |∑_{x∈X} f(x) \overline{g(x)}| ≤ ∑_{x∈X} |f(x)| |g(x)| ≤ ∥f∥_{2} ∥g∥_{2}, (11.20.9)

using the Cauchy–Schwarz inequality (11.17.6) in the third step again. In this situation, we have that

\overline{⟨f, g⟩_{ℓ^{2}(X,C)}} = \overline{( ∑_{x∈X} f(x) \overline{g(x)})} = ∑_{x∈X} g(x) \overline{f(x)} = ⟨g, f⟩_{ℓ^{2}(X,C)}. (11.20.10)

230 CHAPTER 11. SUMMABLE FUNCTIONS If t ∈ C, then t f is square-summable on X, and it is easy to see that ⟨t f, g⟩_{ℓ^{2}(X,C)} = t ⟨f, g⟩_{ℓ^{2}(X,C)}, (11.20.11) as before. Similarly, t g is square-summable on X, and ⟨f, t g⟩_{ℓ^{2}(X,C)} = \overline{t} ⟨f, g⟩_{ℓ^{2}(X,C)}, (11.20.12) by the same type of argument, or using (11.20.10). If f_{1} and f_{2} are squaresummable complex-valued functions on X, then f_{1} + f_{2} are square-summable on X, and ⟨f_{1} + f_{2}, g⟩_{ℓ^{2}(X,C)} = ⟨f_{1}, g⟩_{ℓ^{2}(X,C)} + ⟨f_{2}, g⟩_{ℓ^{2}(X,C)}, (11.20.13) as in the real case. The analogous additivity property of the inner product (11.20.7) in g can be obtained in the same way, or using (11.20.10).

Some additional topics

12.1 Lebesgue measure and integration

Let a, b be real numbers with a < b, and remember that

d_{1}(f, g) = ∫_{a}^{b} |f(x) − g(x)| dx (12.1.1)

defines a metric on the space C([a, b], R) of continuous real-valued functions on [a, b], as in Section 3.1. It is not difficult to see that this space is not complete with respect to (12.1.1), as in Section A.5. In order to get completeness, one can use the Lebesgue integral. Although we shall not discuss this in detail here, let us mention a few other topics related to Lebesgue’s theory of measure and integration.

Let {f_{j}}_{j=1}^{∞} be a sequence of continuous real-valued functions on [a, b], and let f be another continuous real-valued function on [a, b]. If {f_{j}}_{j=1}^{∞} converges to f uniformly on [a, b], then it is easy to see that {f_{j}}_{j=1}^{∞} converges to f with respect to (12.1.1). In particular, this implies that

lim_{j→∞} ∫_{a}^{b} f_{j}(x) dx = ∫_{a}^{b} f(x) dx, (12.1.2)

because

|∫_{a}^{b} f_{j}(x) dx − ∫_{a}^{b} f(x) dx| ≤ ∫_{a}^{b} |f_{j}(x) − f(x)| dx (12.1.3)

for every j ≥ 1. However, one can give examples to show that this does not work if we only ask that {f_{j}}_{j=1}^{∞} converge to f pointwise on [a, b], as in Section A.4.

Suppose now that {f_{j}}_{j=1}^{∞} is uniformly bounded on [a, b], in the sense that there is a nonnegative real number A such that

|f_{j}(x)| ≤ A (12.1.4) for every x ∈ [a, b] and j ≥ 1. If {f_{j}}_{j=1}^{∞} converges to f pointwise on [a, b], then a classical theorem of Arzelà and Osgood implies that (12.1.2) holds, as in [37, 69, 122, 126, 180]. More precisely, this works when the f_{j}’s and f are Riemann integrable on [a, b], instead of continuous. One can also get that {f_{j}}_{j=1}^{∞} converges to f with respect to (12.1.1), by considering {|f_{j} − f|}_{j=1}^{∞} in the previous statement. Questions like these can be treated more extensively using the Lebesgue integral.

If f is a bounded real-valued function on [a, b], then a famous theorem states that f is Riemann integrable on [a, b] if and only if f is continuous at “almost every” point in [a, b] with respect to Lebesgue measure. This means that the set of points in [a, b] at which f is not continuous has Lebesgue measure equal to 0. This is discussed in many textbooks, as well as the articles [24, 147]. Some related results can be found in [121, 134], and also in in [177], for Riemann– Stieltjes integrability.

Now let f be a monotonically increasing real-valued function on [a, b]. It is well known that the one-sided limits of f exist at every point in (a, b), as well as the appropriate one-sided limits at the endpoints a, b. This can be used to show that f is continuous at all but finitely or countably many points in [a, b]. A famous theorem states that f is also differentiable at almost every point in [a, b], with respect to Lebesgue measure. Of course, the derivative is nonnegative when it exists, because of monotonicity.

One can show that

∫_{a}^{b} f′(x) dx ≤ f(b) − f(a), (12.1.5)

where the integral on the left is defined as a Lebesgue integral. If f has a jump discontinuity at any point in [a, b], then the inequality in (12.1.5) is strict. There are also examples where f is both continuous and monotonically increasing on [a, b], and the inequality in (12.1.5) is still strict.

Suppose that f is a real-valued Lipschitz function on [a, b], with respect to the standard Euclidean metric on R and its restriction to [a, b], but not necessarily monotonic. Another famous theorem states that f is differentiable almost everywhere with respect to Lebesgue measure on [a, b]. It is easy to see that |f′(x)| ≤ Lip(f), (12.1.6)

when the derivative exists. This permits the integral on the left side of (12.1.5) to be defined as a Lebesgue integral, and in fact one has

∫_{a}^{b} f′(x) dx = f(b) − f(a) (12.1.7)

in this situation. If fis a real-valued Lipschitz function on R^{n} for some n ∈ Z_{+}, with respect to the standard Euclidean metrics on R^{n} and R, then another famous theorem states that f is differentiable almost everywhere with respect to n-dimensional Lebesgue measure on R^{n}.

12.2 Banach spaces

We shall now suppose that the reader has some familiarity with abstract vector spaces and related notions. Let V be a vector space over the real or complex numbers. Thus V is a set, on which operations of addition and scalar multiplication over R or C have been defined. These operations should satisfy a number of standard conditions, such as associativity and commutativity of addition, and compatibility of scalar multiplication with addition on V and addition and multiplication on R or C, as appropriate. There should also be a distinguished additive element in V, denoted 0.

As usual, a nonnegative real-valued function N on V is said to be a norm on V if it satisfies the following three conditions. First, N(v) = 0 if and only if v = 0. Second, N(t v) = |t| N(v) (12.2.1)

for every v ∈ V and t ∈ R or C, as appropriate. Third,

N(v + w) ≤ N(v) + N(w) (12.2.2)

for every v, w ∈ V. In this case, it is easy to see that

d_{N}(v, w) = N(v − w) (12.2.3)

defines a metric on V. If V is also complete as a metric space with respect to (12.2.3), then V is said to be a Banach space with respect to N. Otherwise, one can pass to a suitable completion of V, but we shall get into that now.

Suppose that {v_{j}}_{j=1}^{∞}, {w_{j}}_{j=1}^{∞} are sequences of elements of V that converge to v, w ∈ V, respectively, with respect to (12.2.3). Under these conditions, one can check that {v_{j} + w_{j}}_{j=1}^{∞} converges to v + w in V with respect to (12.2.3). Similarly, suppose that {t_{j}}_{j=1}^{∞} is also a sequence of real or complex numbers that converges to a real or complex number t, with respect to the standard metric on R or C, as appropriate. In this case, one can verify that {t_{j} v_{j}}_{j=1}^{∞} converges to t v in V with respect to (12.2.3). The proofs of these statements are analogous to those for the corresponding facts about sums and products of convergent sequences of real and complex numbers.

Let W be a linear subspace of V. This means that W is a subset of V that contains 0 and satisfies the following two properties. First, if v, w ∈ W, then

v + w ∈ W. (12.2.4)

Second, if v ∈ W and t ∈ R or C, as appropriate, then

t v ∈ W. (12.2.5)

It follows that W is also a vector space over R or C, as appropriate, with respect to the restriction of the vector space operations on V to W. If N is a norm on V, then it is easy to see that the restriction of N to W defines a norm on W. Of course, the metric on W associated to the restriction of N to W is the same as the restriction to W of the metric (12.2.3) associated to N on V. If V is a Banach space with respect to N and W is a closed set in V with respect to (12.2.3), then W is a Banach space with respect to the restriction of N to W. More precisely, W is complete with respect to the restriction of (12.2.3) to W in this situation.

12.3 Hilbert spaces

Let V be a vector space over the real numbers. A real-valued function ⟨v, w⟩ defined for v, w ∈ V is said to be an inner product on V if it satisfies the following three conditions. First, for each w ∈ V, ⟨v, w⟩ should be linear in V, as a mapping from V into R. This means that

⟨v + v′, w⟩ = ⟨v, w⟩ + ⟨v′, w⟩ (12.3.1)

for every v, v′ ∈ V, and that

⟨t v, w⟩ = t ⟨v, w⟩ (12.3.2)

for every v ∈ V and t ∈ R. Second, ⟨v, w⟩ should be symmetric in v and w, so that ⟨v, w⟩ = ⟨w, v⟩ (12.3.3)

for every v, w ∈ V. This implies that ⟨v, w⟩ is linear in w for every v ∈ V, because of the linearity in v. Third,

⟨v, v⟩ > 0 (12.3.4)

for every v ∈ V with v ≠ 0. Of course, ⟨v, w⟩ = 0 whenever v = 0 or w = 0, because of linearity in v and w.

Similarly, if V is a vector space over the complex numbers, then a complexvalued function ⟨v, w⟩ defined for v, w ∈ V is said to be an inner product if it satisfies the following three conditions. First, for every w ∈ V, ⟨v, w⟩ should be (complex) linear in v, as a mapping from V into C. This means that (12.3.1) should hold for every v, v′ ∈ V, as before, and that (12.3.2) should hold for every v ∈ V and t ∈ C. Second, ⟨v, w⟩ should be Hermitian symmetric in v and w, which means that ⟨w, v⟩ = \overline{⟨v, w⟩} (12.3.5)

for every v, w ∈ V, where \overline{a} is the complex conjugate of a complex number a. Combining this with the first condition, we get that ⟨v, w⟩ is conjugate-linear in w for each v ∈ V, so that

⟨v, w + w′⟩ = ⟨v, w⟩ + ⟨v, w′⟩ (12.3.6)

for every w, w′ ∈ V, and ⟨v, t w⟩ = \overline{t} ⟨v, w⟩ (12.3.7) for every w ∈ V and t ∈ C. Observe that

\overline{⟨v, v⟩} = ⟨v, v⟩ (12.3.8)

for every v ∈ V, which means that ⟨v, v⟩ ∈ R. The third condition is that (12.3.4) hold for every v ∈ V with v ≠ 0 again. As before, ⟨v, w⟩ = 0 whenever v = 0 or w = 0.

Let (V, ⟨v, w⟩) be a real or complex inner product space. Put

∥v∥ = ⟨v, v⟩^{1/2} (12.3.9)

for every v ∈ V, using the nonnegative square root on the right side. It is well known that |⟨v, w⟩| ≤ ∥v∥ ∥w∥ (12.3.10)

for every v, w ∈ V, which is another version of the Cauchy–Schwarz inequality. More precisely, this can be shown using the fact that

⟨v + t w, v + t w⟩ = ∥v + t w∥^{2} ≥ 0 (12.3.11)

for every v, w ∈ V amd t ∈ R or C, as appropriate. Note that

∥t v∥ = |t| ∥v∥ (12.3.12)

for every v ∈ V and t ∈ R or C, as appropriate. One can also check that

∥v + w∥ ≤ ∥v∥ + ∥w∥ (12.3.13)

for every v, w ∈ V, using (12.3.10). Thus (12.3.9) defines a norm on V as a vector space over R or C, as appropriate.

If V is complete with respect to the metric associated to the norm (12.3.9), then V is said to be a Hilbert space with respect to the inner product ⟨v, w⟩. Otherwise, one can pass to a completion of V, as before.

12.4 Infinite series in Banach spaces

Let V be a vector space over the real or complex numbers, and let N be a norm on V. An infinite series

∑_{j=1}^{∞} v_{j} (12.4.1)

with terms v_{j} in V for each j ≥ 1 is said to converge in V with respect to N if the corresponding sequence of partial sums

∑_{j=1}^{l} v_{j} (12.4.2)

converges to an element of V with respect to the metric d_{N} associated to N. In this case, the value of the sum (12.4.1) is defined to be the limit of the sequence (12.4.2). If (12.4.1) converges in V, and if ∑_{j=1}^{∞} w_{j} is another infinite series of elements of V that converges in V, then ∑_{j=1}^{∞}(v_{j} +w_{j}) converges in V too, with

∑_{j=1}^{∞}(v_{j} + w_{j}) = ∑_{j=1}^{∞} v_{j} + ∑_{j=1}^{∞} w_{j}. (12.4.3)

This follows from the analogous statement for sums of convergent sequences in V, applied to the partial sums of these series. Similarly, if (12.4.1) converges in V, and t ∈ R or C, as appropriate, then ∑_{j=1}^{∞} t v_{j} converges in V as well, with

∑_{j=1}^{∞} t v_{j} = t ∑_{j=1}^{∞} v_{j}. (12.4.4)

This uses the fact that t times a convergent sequence in V converges to t times the limit of the sequence.

The condition that the sequence of partial sums (12.4.2) of an infinite series (12.4.1) be a Cauchy sequence with respect to the metric d_{N} associated to N is equivalent to saying that for each ϵ > 0 there is a positive integer L such that

N( ∑_{j=k}^{l} v_{j}) < ϵ (12.4.5)

for every l ≥ k ≥ L. In particular, this holds when (12.4.1) converges in V, because a convergent sequence in any metric space is a Cauchy sequence. Note that the Cauchy condition for the partial sums implies that

lim_{l→∞} N(v_{l}) = 0, (12.4.6)

by taking k = l in (12.4.5). This is the same as saying that {v_{j}}_{j=1}^{∞} converges to 0 in V, with respect to d_{N}. If V is a Banach space with respect to N, then the Cauchy condition for the partial sums implies that the series converges in V.

An infinite series (12.4.1) with terms in V is said to be absolutely convergent with respect to N if

∑_{j=1}^{∞} N(v_{j}) (12.4.7)

converges as an infinite series of nonnegative real numbers. Observe that

N( ∑_{j=k}^{l} v_{j}) ≤ ∑_{j=k}^{l} N(v_{j}) (12.4.8)

for every l ≥ k ≥ 1, by the triangle inequality for N. If (12.4.1) converges absolutely with respect to N, then the sequence of partial sums (12.4.2) is a Cauchy sequence with respect to d_{N}. More precisely, one verify (12.4.5) using (12.4.8) and the analogous Cauchy condition for the partial sums of (12.4.7). If V is a Banach space with respect to N, then it follows that (12.4.1) converges in V.

Now let (V, ⟨v, w⟩) be a real or complex inner product space, and let us use the corresponding norm ∥ · ∥, as in the previous section. A pair v, w of vectors in V are said to be orthogonal if

⟨v, w⟩ = 0. (12.4.9)

In this case, it is easy to see that

∥v + w∥^{2} = ∥v∥^{2} + ∥w∥^{2}. (12.4.10)

Let (12.4.1) be an infinite series of pairwise-orthogonal vectors in V, so that v_{j} is orthogonal to v_{k} when j ≠ k. This implies that

∥∑_{j=k}^{l} v_{j}∥^{2} = ∑_{j=k}^{l} ∥v_{j}∥^{2} (12.4.11)

for every l ≥ k ≥ 1. If

∑_{j=1}^{∞} ∥v_{j}∥^{2} (12.4.12)

converges as an infinite series of nonnegative real numbers, then the sequence (12.4.2) is a Cauchy sequence in V. This uses (12.4.11) to get the Cauchy condition (12.4.5) from the analogous Cauchy condition for the partial sums of (12.4.12). If V is a Hilbert space with respect to ⟨·, ·⟩, then it follows that (12.4.1) converges in V.

12.5 Bounded linear mappings

Let V and W be vector spaces, where more precisely V and W should both be defined over the real numbers, or both defined over the complex numbers. Also let N_{V} and N_{W} be norms on V and W, respectively. Thus

d_{V} (v, v′) = N_{V} (v − v′) and d_{W} (w, w′) = N_{W} (w, w′) (12.5.1)

define metrics on V and W, respectively. A linear mapping T from V into W is said to be bounded with respect to N_{V} and N_{W} if there is a nonnegative real number C such that N_{W} (T (v)) ≤ C N_{V} (v) (12.5.2)

for every v ∈ V. In this case, we have that

d_{W} (T (u), T (v)) = N_{W} (T (u) − T (v)) = N_{W} (T (u − v))
(12.5.3)  ≤ C N_{V} (u − v) = C d_{V} (u, v)

for every u, v ∈ V, so that T is Lipschitz with respect to the corresponding metrics on V and W.

Conversely, if a linear mapping T from V into W is Lipschitz with respect to the metrics d_{V} (·, ·) and d_{W} (·, ·), then it is easy to see that T is bounded with respect to N_{V} and N_{W}. More precisely, if a linear mapping T from V into W has the property that N_{W} (T (v)) is bounded on a ball in V of positive radius, then one can check that T is bounded as a linear mapping. This uses scalar multiplication to obtain (12.5.2) from the boundedness of N_{W} (T (v)) when N_{V} (v) is less than a fixed radius. In particular, if T is continuous at 0 with respect to the metrics d_{V} (·, ·) and d_{W} (·, ·) on V and W, then there is a δ > 0 such that N_{W} (T (v)) < 1 when v ∈ V satisfies N_{V} (v) < 1. This implies that T is a bounded as a linear mapping, as before.

If V is a finite-dimensional vector space, then every linear mapping T from V into W is bounded. Of course, this is trivial when V = {0}. Otherwise, there is a positive integer n such that V is isomorphic to R^{n} or C^{n} as a vector space over the real or complex numbers, as appropriate. This permits one to reduce to the same type of arguments as in Section 7.3.

Suppose that V ≠ {0}, and that T is a bounded linear mapping from V into W with respect to N_{V} and N_{W}. The corresponding operator norm is defined by

∥T ∥_{op} = sup \{\frac{N_{W} (T (v))}{N_{V} (v)} : v ∈ V, v ≠ 0\}, (12.5.4)

where the finiteness of the supremum follows from the boundedness of T. If V = {0}, then one can simply take ∥T ∥_{op} = 0. Equivalently, ∥T ∥_{op} is the same as the infimum of the nonnegative real numbers C such that (12.5.2) holds. As in Section 7.7, ∥T ∥_{op} is also the same as Lip(T), defined with respect to the metrics d_{V} (·, ·) and d_{W} (·, ·) associated to N_{V} and N_{W}.

If a is a real or complex number, as appropriate, then a T also defines a linear mapping from V into W, where (a T)(v) = a T (v) for every v ∈ V. It is easy to see that a T is a bounded linear mapping when T is, with

∥a T ∥_{op} = |a| ∥T ∥_{op}. (12.5.5)

Note that ∥T ∥_{op} = 0 if and only if T = 0, which is to say that T (v) = 0 for every v ∈ V. If T_{1} and T_{2} are linear mappings from V into W, then T_{1} + T_{2} defines a linear mapping from V into W as well, with

(T_{1} + T_{2})(v) = T_{1}(v) + T_{2}(v). (12.5.6)

If T_{1}, T_{2} are bounded as linear mappings with respect to N_{V} and N_{W}, then one can check that T_{1} + T_{2} is bounded as well, with

∥T_{1} + T_{2}∥_{op} ≤ ∥T_{1}∥_{op} + ∥T_{2}∥_{op}. (12.5.7)

The space of linear mappings from V into W is a vector space over the real or complex numbers, as appropriate, with respect to pointwise addition and scalar multiplication of mappings. The space of bounded linear mappings from V into W is a linear subspace of the space of all linear mappings from V into W, as in the preceding paragraph. The operator norm (12.5.4) defines a norm on the space of bounded linear mappings from V into W, as a vector space over the real or complex numbers, as appropriate.

Let Z be another vector space, which is defined over the real or complex numbers, depending on whether V and W are defined over the real or complex numbers. Also let N_{Z} be a norm on Z, let T_{1} be a bounded linear mapping from V into W, and let T_{2} be a bounded linear mapping from W into Z. Under these conditions, one can verify that the composition T_{2} ∘ T_{1} is bounded as a linear mapping from V into Z, with

∥T_{2} ∘ T_{1}∥_{op} ≤ ∥T_{1}∥_{op} ∥T_{2}∥_{op}. (12.5.8)

Here the various operator norms are defined with respect to the given norms on V, W, and Z, as appropriate.

Appendix A

Some more on mappings, metrics, and norms

A.1 A nice inequality

Let a be a positive real number, with a ≤ 1. If r and t are nonnegative real numbers, then it is well known that (r + t)^{a} ≤ r^{a} + t^{a}. (A.1.1) To see this, observe first that max(r, t) ≤ (r^{a} + t^{a})^{1/a}. (A.1.2) Using this, we get that r + t ≤ max(r, t)^{1−a} (r^{a} + t^{a}) ≤ (r^{a} + t^{a})^{((1−a)/a)+1} = (r^{a} + t^{a})^{1/a}. (A.1.3) This is equivalent to (A.1.1).

Let (X, d(x, y)) be a metric space. If 0 < a ≤ 1, then it is easy to see that d(x, y)^{a} (A.1.4) is a metric on X as well, using (A.1.1). This was mentioned in Section 1.1 when a = 1/2.

Let us use B_{d}(x, r), \overline{B}_{d}(x, r) to denote the open and closed balls in X centered at x ∈ X with radius r with respect to d(·, ·), respectively, as in Section 1.9, and B_{d^{a}}(x, r), \overline{B}_{d^{a}}(x, r) for the open and closed balls in X centered at x with radius r with respect to d(·, ·)^{a}, respectively. Observe that B_{d^{a}}(x, r^{a}) = B_{d}(x, r) (A.1.5) for every r > 0, and that \overline{B}_{d^{a}}(x, r^{a}) = \overline{B}_{d}(x, r) (A.1.6)

A.2. SOME MORE LIPSCHITZ CONDITIONS 241 for every r ≥ 0.

It is easy to see that the identity mapping on X is uniformly continuous as a mapping from X equipped with d(·, ·) into X equipped with d(·, ·)^{a}. Similarly, the identity mapping on X is uniformly continuous as a mapping from X equipped with d(·, ·)^{a} into X equipped with d(·, ·). These statements were also mentioned in Section 1.1 when a = 1/2.

If a > 1, then one can verify that |x − y|^{a} (A.1.7) is not a metric on the real line. This corresponds to the first part of Exercise 11 at the end of Chapter 2 in [155] when a = 2.

A.2 Some more Lipschitz conditions

Let (X, d_{X}) and (Y, d_{Y}) be metric spaces, and let α be a positive real number. A mapping f from X into Y is said to be Lipschitz of order α if there is a nonnegative real number C such that d_{Y} (f(x), f(w)) ≤ C d_{X}(x, w)^{α} (A.2.1) for every x, w ∈ X. This is the same as a Lipschitz mapping as in Section 1.1 when α = 1. One may also say that f is Hölder continuous of order α in this case.

Lipschitz mappings of any order are uniformly continuous, as before. Of course, (A.2.1) holds with C = 0 if and only if f is constant on X.

Let a be a positive real number, and suppose that (A.2.2) d_{X}(x, w)^{a} is a metric on X. This holds automatically when a ≤ 1, as in the previous section. Note that (A.2.1) is the same as saying that d_{Y} (f(x), f(w)) ≤ C (d_{X}(x, w)^{a})^{α/a} (A.2.3) for every x, y ∈ X. This means that (A.2.4) f is Lipschitz of order α with respect to d_{X} on X if and only if (A.2.5) f is Lipschitz of order α/a with respect to d_{X}(·, ·)^{a} on X, and with the same constant C.

Similarly, let b be a positive real number, and suppose that (A.2.6) d_{Y} (·, ·)^{b} is a metric on Y.

242APPENDIX A. SOME MORE ON MAPPINGS, METRICS, AND NORMS

Clearly (A.2.1) is the same as saying that d_{Y} (f(x), f(w))^{b} ≤ C^{b} d_{X}(x, w)^{α b} (A.2.7) for every x, y ∈ X. Thus (A.2.8) f is Lipschitz of order α with constant C with respect to d_{Y} on Y if and only if (A.2.9) f is Lipschitz of order α b with constant C^{b}

with respect to d_{Y} (·, ·)^{b} on Y.

Suppose that X = Y = R, with the standard Euclidean metric. If f is Lipschitz of order α > 1, then (A.2.10) f is constant on R. More precisely, it is easy to see that the derivative of f is equal to 0 at every point in R under these conditions.

A.3 Another nice inequality

Let X be a nonempty set, and let f be a real or complex-valued function on X with finite support, as in Section 1.6. If p is a positive real number, then put

∥f∥_{p} = ( ∑_{x∈X} |f(x)|^{p})^{1/p}. (A.3.1)

This is the same as in Section 1.6 when p = 1 or 2. If X = {1, …, n} for some positive integer n, then this corresponds to an analogous expression in Section 1.4. We also put

∥f∥_{∞} = max_{x∈X} |f(x)|, (A.3.2)

as before.

Observe that ∥f∥_{∞} ≤ ∥f∥_{p}. (A.3.3) If 0 < p_{1} ≤ p_{2} < ∞, then we would like to check that ∥f∥_{p_{2}} ≤ ∥f∥_{p_{1}}. (A.3.4) Clearly

∥f∥_{p_{2}}^{p_{2}} = ∑_{x∈X} |f(x)|^{p_{2}} ≤ ∥f∥_{∞}^{p_{2}−p_{1}} ∑_{x∈X} |f(x)|^{p_{1}} = ∥f∥_{∞}^{p_{2}−p_{1}} ∥f∥_{p_{1}}^{p_{1}}. (A.3.5)

This implies that ∥f∥_{p_{2}}^{p_{2}} ≤ ∥f∥_{p_{1}}^{p_{2}}, (A.3.6)

A.4. SOME FUNCTIONS ON [0, 1] 243

because of (A.3.3). It follows that (A.3.4) holds, as desired.

It is easy to see that (A.1.1) follows from (A.3.4), with p_{1} = a, p_{2} = 1, and where X has two elements.

Observe that ∥f∥_{p} ≤ (# supp f)^{1/p} ∥f∥_{∞}, (A.3.7)

where # supp f is the number of elements in the support of f. One can use this and (A.3.3) to get that

∥f∥_{p} → ∥f∥_{∞} as p → ∞. (A.3.8)

Of course, ∥f∥_{p} = 0 if and only if f = 0 on X, and

∥t f∥_{p} = |t| ∥f∥_{p} (A.3.9)

for every t ∈ R or C, as appropriate. If g is another real or complex-valued function on X with finite support and 0 < p ≤ 1, then

∥f + g∥_{p}^{p} = ∑_{x∈X} |f(x) + g(x)|^{p} ≤ ∑_{x∈X}(|f(x)| + |g(x)|)^{p}
(A.3.10)  ≤ ∑_{x∈X}(|f(x)|^{p} + |g(x)|^{p}) = ∑_{x∈X} |f(x)|^{p} + ∑_{x∈X} |g(x)|^{p}
= ∥f∥_{p}^{p} + ∥g∥_{p}^{p},

using (A.1.1) in the third step, with a = p. This implies that

∥f − g∥_{p}^{p} (A.3.11)

defines a metric on c_{00}(X, R) and c_{00}(X, C) when 0 < p ≤ 1.

A.4 Some functions on [0, 1]

Let j be a positive integer, and let f_{j} be the real-valued function defined on [0, 1] by

(A.4.1)  f_{j}(x) = 2 j x  when 0 ≤ x ≤ 1/(2 j)
= 2 − 2 j x when 1/(2 j) ≤ x ≤ 1/j
= 0  when 1/j ≤ x ≤ 1.

Thus f_{j}(0) = 0, f_{j}(1/(2 j)) = 1, and f(1/j) = 0, (A.4.2)

244APPENDIX A. SOME MORE ON MAPPINGS, METRICS, AND NORMS

with the overlapping definitions of f_{j}(x) agreeing at x = 1/(2 j), 1/j. Equivalently, f_{j} is defined to be linear on [0, 1/(2 j)] and [1/(2 j), 1/j], with these values at the endpoints. Note that f_{j} is continuous on [0, 1]. One can check that (A.4.3) {f_{j}}_{j=1}^{∞} converges to 0 pointwise on [0, 1], and not uniformly.

In fact, ∥f_{j}∥_{∞} = 1 (A.4.4) for every j, where ∥ · ∥_{∞} is the supremum norm on C([0, 1], R), as before. Observe that

∥f_{j}∥_{1} = ∫_{0}^{1} f_{j}(x) dx = 2 ∫_{0}^{1/(2 j)} f_{j}(x) dx = 2 j (1/(2 j))^{2} = 1/(2 j) (A.4.5)

for every j, where ∥ · ∥_{1} is as in (1.15.4). More precisely,

f_{j}(x) = f_{j}((1/j) − x) when 0 ≤ x ≤ 1/j, (A.4.6)

which is to say that f_{j}(x) is symmetric about x = 1/(2 j) on [0, 1/j]. This implies that the integrals of f_{j}(x) over [0, 1/(2 j)] and [1/(2 j), 1/j] are the same. Of course, (A.4.5) is the same as the area of the triangle determined by the graph of f_{j} on [0, 1/j].

Similarly,

(A.4.7) ∥f_{j}∥_{2}^{2} = ∫_{0}^{1} f_{j}(x)^{2} dx = 2 ∫_{0}^{1/(2 j)} f_{j}(x)^{2} dx
= (2/3) (2 j)^{2} (1/(2 j))^{3} = 1/(3 j)

for every j, where ∥ · ∥_{2} is as in (1.15.6). This uses (A.4.6) in the second step, to get that the integrals of f_{j}(x)^{2} over [0, 1/(2 j)] and [1/(2 j), 1/j] are the same. It follows that ∥f_{j}∥_{2} = 1/(3 j)^{1/2} (A.4.8) for each j.

If α ∈ R, then one can verify that

(A.4.9) {j^{−α} f_{j}}_{j=1}^{∞} converges to 0 pointwise on [0, 1].

Note that ∥j^{−α} f_{j}∥_{∞} = j^{−α} (A.4.10) for every j, by (A.4.4). This implies that {j^{−α} f_{j}}_{j=1}^{∞} is bounded with respect to the supremum metric exactly when α ≥ 0, and that {f_{j}}_{j=1}^{∞} converges to 0 with respect to the supremum metric exactly when α > 0.

Similarly, ∥j^{−α} f_{j}∥_{1} = (1/2) j^{−1−α} (A.4.11) for every j, by (A.4.5). This means that {j^{−α} f_{j}}_{j=1}^{∞} is bounded with respect to the metric d_{1} on C([0, 1], R) associated to ∥ · ∥_{1} as in (1.15.10) exactly when α ≥ −1, and that {j^{−α} f_{j}}_{j=1}^{∞} converges to 0 with respect to d_{1} exactly when α > −1. We also have that

∥j^{−α} f_{j}∥_{2} = (1/\sqrt{3}) j^{−(1/2)−α} (A.4.12)

for every j, by (A.4.8). It follows that {j^{−α} f_{j}}_{j=1}^{∞} is bounded with respect to the metric d_{2} on C([0, 1], R) associated to ∥ · ∥_{2} as in (1.15.11) if and only if α ≥ −1/2, and that {j^{−α} f_{j}}_{j=1}^{∞} converges to 0 with respect to d_{2} if and only if α > −1/2.

A.5 Some Cauchy sequences

Let j be a positive integer, and let f_{j} be the real-valued function defined on [0, 1] by

f_{j}(x) = 0  when 0 ≤ x ≤ 1/2 − 1/(2 j)
(A.5.1)  = 2 j (x − (1/2 − 1/(2 j))) when 1/2 − 1/(2 j) ≤ x ≤ 1/2
= 1  when 1/2 ≤ x ≤ 1.

The second case says that f_{j} is linear in that range, with the same values at the endpoints as in the other two cases. In particular, f_{j} is continuous on [0, 1] for each j. It is easy to see that {f_{j}}_{j=1}^{∞} converges pointwise on [0, 1] to the real-valued function f defined on [0, 1] by

(A.5.2)  f(x) = 0 when 0 ≤ x < 1/2
= 1 when 1/2 ≤ x ≤ 1.

Note that {f_{j}}_{j=1}^{∞} does not converge to f uniformly on [0, 1].

However, one can check that {f_{j}}_{j=1}^{∞} is a Cauchy sequence in C([0, 1], R) with respect to the metric d_{1} defined in (1.15.10). Basically, {f_{j}}_{j=1}^{∞} converges to f with respect to a metric like d_{1} on a larger space. Similarly, one can verify that {f_{j}}_{j=1}^{∞} is a Cauchy sequence in C([0, 1], R) with respect to the metric d_{2} defined in (1.15.11).

Let j be a positive integer again, and let f_{j} be the real-valued function defined on [0, 1] by f_{j}(x) = min(1/\sqrt{x}, j), (A.5.3)

where the right side is interpreted as being equal to j when x = 0. Note that f_{j} is continuous on [0, 1] for each j. One can check that {f_{j}}_{j=1}^{∞} is a Cauchy sequence in C([0, 1], R) with respect to d_{1}. However, {f_{j}}_{j=1}^{∞} is not bounded as a sequence in C([0, 1], R) with respect to d_{2}.

Now let f be a real or complex-valued function on X = Z_{+}. If j is a positive integer, then let f_{j} be the real or complex-valued function, as appropriate, defined on Z_{+} by

(A.5.4)  f_{j}(l) = f(l) when l ≤ j
= 0  when l > j.

Clearly {f_{j}}_{j=1}^{∞} converges to f pointwise on Z_{+}. If

lim_{l→∞} f(l) = 0, (A.5.5)

then {f_{j}}_{j=1}^{∞} converges to f uniformly on Z_{+}.

By construction, the support of f_{j} has only finitely many elements for each j. Suppose for the moment that

∑_{l=1}^{∞} |f(l)| (A.5.6)

converges as an infinite series of nonnegative real numbers. In this case, one can check that {f_{j}}_{j=1}^{∞} is a Cauchy sequence in c_{00}(Z_{+}, R) or c_{00}(Z_{+}, C), as appropriate, with respect to the metric d_{1}(·, ·) defined in Section 1.6. Similarly, suppose instead that

∑_{l=1}^{∞} |f(l)|^{2} (A.5.7)

converges as an infinite series of nonnegative real numbers. One can verify that {f_{j}}_{j=1}^{∞} is a Cauchy sequence in c_{00}(Z_{+}, R) or c_{00}(Z_{+}, C), as appropriate, with respect to the metric d_{2}(·, ·) defined in Section 1.6 in this situation.

A.6 Norms and convexity

If the reader is familiar with the abstract notion of a vector space, then let V be a vector space over the real numbers. Otherwise, one can consider particular situations like V = R^{n} for some positive integer n, or a vector space of realvalued functions on a nonempty set. As usual, a subset E of V is said to be convex if for every v, w ∈ E and t ∈ R with 0 ≤ t ≤ 1 we have that

t v + (1 − t) w ∈ E. (A.6.1)

Let N be a nonnegative real-valued function on V such that

N(t v) = |t| N(v) (A.6.2)

for every t ∈ R and v ∈ V. If N also satisfies the triangle inequality

N(v + w) ≤ N(v) + N(w) (A.6.3)

for every v, w ∈ V, then N is said to be a seminorm on V. Thus a norm on V is the same as a seminorm N such that N(v) > 0 for every v ∈ V with v ≠ 0. Put \overline{B}_{N} = \{v ∈ V : N(v) ≤ 1\}, (A.6.4)

which is the closed unit ball in V with respect to N. If N is a seminorm on V, then it is easy to see that \overline{B}_{N} is a convex subset of V. Conversely, if N is a nonnegative real-valued function on V that satisfies (A.6.2), and if \overline{B}_{N} is a convex set in V, then N satisfies (A.6.3), and hence is a seminorm on V. This is not too difficult to show, directly from the definitions.

Let X be a nonempty set, and let c_{00}(X, R) be the space of real-valued functions on X with finite support, as in Section 1.6. If f ∈ c_{00}(X, R) and p is a positive real number, then ∥f∥_{p} may be defined as in Section A.3. This satisfies the first two conditions in the definition of a norm on c_{00}(X, R), as before. If p ≥ 1, then one can show that the corresponding closed unit ball

\overline{B}_{p} = \{f ∈ c_{00}(X, R) : ∥f∥_{p} ≤ 1\} (A.6.5) is a convex set in c_{00}(X, R). More precisely, (A.6.5) is the same as taking

\overline{B}_{p} = \{f ∈ c_{00}(X, R) : ∥f∥_{p}^{p} = ∑_{x∈X} |f(x)|^{p} ≤ 1\}. (A.6.6)

One can show that this is a convex subset of c_{00}(X, R) when p ≥ 1, using the convexity of r^{p} for r ≥ 0 when p ≥ 1. This implies that ∥f∥_{p} defines a norm on c_{00}(X, R) when p ≥ 1, as in the preceding paragraph. If 0 < p < 1, then (A.6.5) is not convex in c_{00}(X, R) when X has at least two elements.

A vector space over the complex numbers may be considered as a vector space over the real numbers as well, to which the earlier remarks about convexity can be applied. In particular, there are analogues of the statements in the preceding paragraph for the space c_{00}(X, C) of complex-valued functions on X with finite support.

Let n be a positive integer, and let N be a seminorm on R^{n}. The closed unit ball \overline{B}_{N} in R^{n} with respect to N is convex, as before, as is the open unit ball B_{N} = {v ∈ R^{n} : N(v) < 1}. (A.6.7)

Note that B_{N} and \overline{B}_{N} are also symmetric about the origin, which is to say that they are invariant under the mapping v ↦ −v on R^{n}. One can show that N is continuous as a real-valued function on R^{n}, and in fact it is Lipschitz with respect to the standard Euclidean metric on R^{n}, as in Section 6.7. This implies that B_{N} is an open set, and \overline{B}_{N} is a closed set, with respect to the standard Euclidean metric on R^{n}. If N is a norm on R^{n}, then the standard Euclidean norm on R^{n} is bounded by a constant multiple of N, as in Section 6.7. It follows that B_{N} and \overline{B}_{N} are bounded sets with respect to the standard Euclidean metric on R^{n} in this case.

A.7 Path-connected sets

Let (X, d_{X}) be a metric space, and let a, b be real numbers with a < b. Suppose that

(A.7.1) p is a continuous mapping from the closed interval [a, b] into X,

with respect to the restriction of the standard Euclidean metric on R to [a, b]. Under these conditions, one can check that

(A.7.2) p([a, b]) is a connected subset of X.

More precisely, it is well known that [a, b] is a connected subset of the real line, with respect to the standard metric. This implies that [a, b] is connected as a subset of itself, with respect to the restriction of the standard metric on R to [a, b], as in Section 8.12. It follows that (A.7.2) holds, because of the wellknown theorem that continuous mappings send connected sets to connected 248APPENDIX A. SOME MORE ON MAPPINGS, METRICS, AND NORMS

sets. Alternatively, one can extend p to a continuous mapping from R into X, by putting p(t) = p(a) when t < a, and p(t) = p(b) when t > b.

A subset E of X is said to be path connected if for every pair of points x, w ∈ E there are real numbers a, b with a < b and a continuous mapping p from [a, b] into X such that p(a) = x, p(b) = w, and p([a, b]) ⊆ E. (A.7.3) It is well known and not too difficult to show that (A.7.4) path-connected sets are connected, using (A.7.2).

If E is a convex set in R^{n} for some positive integer n, then (A.7.5) E is path connected, with respect to the standard Euclidean metric on R^{n}. Note that connected subsets of R are convex.

If U is a connected open set in R^{n}, then it is well known that (A.7.6) U is path connected. To see this, let x ∈ U be given, and let U_{x} be the set of w ∈ U for which there is a continuous path in U from x to w, as before. One can check that (A.7.7) U_{x} is an open set in R^{n}. Similarly, one can verify that (A.7.8) U \ U_{x} is an open set in R^{n}. If U is connected, then it follows that U_{x} = U. (A.7.9)

It is well known that there are connected subsets of R^{2} that are not path connected.

If E is a connected set in any metric space X, then it is well known and not too difficult to show that (A.7.10) \overline{E} is connected in X too. However, there are path-connected subsets of R^{2} whose closures are not path connected.

Let (Y, d_{Y}) be another metric space, and let f be a continuous mapping from X into Y. If E is a path-connected subset of X, then it is easy to see that (A.7.11) f(E) is path connected in Y.

Bibliography

[1] R. Agarwal, C. Flaut, and D. O’Regan, An Introduction to Real Analysis, CRC Press, 2018.

[2] T. Apostol, Mathematical Analysis, 2nd edition, Addison-Wesley, 1974.

[3] T. Archibald, Connectivity and smoke-rings: Green’s second identity in its first fifty years, Mathematics Magazine 62 (1989), 219–232.

[4] L. Badger, Notes: Generating the measures of n-balls, American Mathematical Monthly 107 (2000), 256–258.

[5] J. Baker, Isometries in normed spaces, American Mathematical Monthly 78 (1971), 655–658.

[6] J. Baker, Integration over spheres and the divergence theorem for balls, American Mathematical Monthly 104 (1997), 36–47.

[7] J. Baker, Integration of radial functions, Mathematics Magazine 72 (1999), 392–395.

[8] R. Beals, Advanced Mathematical Analysis: Periodic Functions and Distributions, Complex Analysis, Laplace Transform and Applications, Springer-Verlag, 1973.

[9] R. Beals, Analysis: An Introduction, Cambridge University Press, 2004.

[10] R. Beals, The scope of the power series method, Mathematics Magazine 86 (2013), 56–62.

[11] P. Beesack, On improper multiple integrals, Mathematics Magazine 43 (1970), 113–123.

[12] M. Berger, Convexity, American Mathematical Monthly 97 (1990), 650– 678.

[13] L. Bers, Classroom notes: On avoiding the mean value theorem, American Mathematical Monthly 74 (1967), 583.

[14] R. Boas, A Primer of Real Functions, 4th edition, revised and updated by H. Boas, Mathematical Association of America, 1996.

[15] S. Bochner, Fourier series came first, American Mathematicsl Monthly 86 (1979), 197–199.

[16] M. Botsko and R. Gosser, The teaching of mathematics: On the differentiability of functions of several variables, American Mathematical Monthly 92 (1985), 663–665.

[17] D. Bressoud, A Radical Approach to Real Analysis, 2nd edition, Mathematical Association of America, 2007.

[18] D. Bressoud, A Radical Approach to Lebesgue’s Theory of Integration, MAA Textbooks, Cambridge University Press, 2008.

[19] D. Bressoud, Historical reflections on teaching the fundamental theorem of integral calculus, American Mathematical Monthly 118 (2011), 99–115.

[20] D. Bressoud, Calculus Reordered: A History of the Big Ideas, Princeton University Press, 2019.

[21] A. Browder, Mathematical Analysis: An Introduction, Springer-Verlag, 1996.

[22] A. Browder, Topology in the complex plane, American Mathematical Monthly 107 (2000), 393–401.

[23] A. Browder, Complex numbers and the ham sandwich theorem, American Mathematical Monthly 113 (2006), 935–937.

[24] A. Brown, A proof of the Lebesgue condition for Riemann integrability, American Mathematical Monthly 43 (1936), 396–398.

[25] A. Brown, On transformation of multiple integrals, American Mathematical Monthly 48 (1941), 29–33.

[26] A. Brown, Mathematical notes: Extensions of the Brouwer fixed point theorem, American Mathematical Monthly 69 (1962), 643.

[27] A. Bruckner, Derivatives: why they elude classification, Mathematics Magazine 49 (1976), 5–11.

[28] G. Bullock, The teaching of mathematics: A geometric interpretation of the Riemann–Stieltjes integral, American Mathematical Monthly 95 (1988), 448-455.

[29] R. Burckel and C. Goffman, Rectifiable curves are of zero content, Mathematics Magazine 44 (1971), 179–180.

[30] J. Burkill and H. Burkill, A Second Course in Mathematical Analysis, Cambridge University Press, 1970.

[31] T. Cameron, The determinant from signed volume to the Laplace expansion, American Mathematical Monthly 126 (2019), 437–447.

BIBLIOGRAPHY 251

[32] D. Cannell, Geoge Green: an enigmatic mathematician, American Mathematical Monthly 106 (1999), 136–151.

[33] D. Cannell, George Green, Mathematician & Physicist 1793–1841, 2nd edition, Society for Industrial and Applied Mathematics (SIAM), 2001.

[34] L. Cohen, Classroom notes: On being mean to the mean-value theorem, American Mathematical Monthly 74 (1967), 581–582.

[35] R. Cohen, Set isometries and their extensions in the max metric, American Mathematical Monthly 97 (1990), 795–808.

[36] J. Conway, A First Course in Analysis, Cambridge University Press, 2018.

[37] N. de Silva, A concise, elementary proof of Arzelà’s bounded convergence theorem, American Mathematical Monthly 117 (2010), 918–920.

[38] A. Devinatz, Advanced Calculus, Holt, Rinehart and Winston, 1968.

[39] T. Drammatico, A new proof of the equality of mixed second partial derivatives, American Mathematical Monthly 122 (2015), 602-603.

[40] R. Dressler and K. Stromberg, The Tonelli integral, American Mathematical Monthly 81 (1974), 67–68.

[41] W. Dunham, Touring the calculus gallery, American Mathematical Monthly 112 (2005), 1-19.

[42] W. Dunham, The Calculus Gallery: Masterpieces from Newton to Lebesgue, Princeton University Press, 2018.

[43] P. Duren, Invitation to Classical Analysis, American Mathematical Society, 2012.

[44] A. Fadell, Classroom notes: A proof of the chain rule for derivatives in n-space, American Mathematical Monthly 80, 1134–1135.

[45] W. Felscher, Bolzano, Cauchy, epsilon, delta, American Mathematical Monthly 107 (2000), 844–862.

[46] L. Flatto, Advanced Calculus, Williams & Wilkins, 1976.

[47] T. Flett, On transformations in R^{n} and a theorem of Sard, American Mathematical Monthly 71 (1964), 623-629.

[48] G. Folland, Fourier Analysis and its Applications, Wadsworth & Brooks / Cole, 1992.

[49] G. Folland, How to integrate a polynomial over a sphere, American Mathematical Monthly 108 (2001), 446–448.

252 BIBLIOGRAPHY

[50] G. Folland, A Guide to Advanced Real Analysis, Mathematical Association of America, 2009.

[51] M. Fort, Jr., Some properties of continuous functions, American Mathematical Monthly 59 (1952), 372–375.

[52] P. Franklin, A Treatise on Advanced Calulus, Dover, 1964.

[53] A. Friedman, Advanced Calculus, Holt, Rinehart and Winston, 1971.

[54] T. Gamelin and R. Greene, Introduction to Topology, 2nd edition, Dover, 1999.

[55] C. Gardner, Another elementary proof of Peano’s existence theorem, American Mathematical Monthly 83 (1976), 556-560.

[56] H. Gaskill and P. Narayanaswami, Foundations of Analysis: The Theory of Limits, Harper & Row, 1989.

[57] B. Gelbaum and J. Olmsted, Counterexamples in Analysis, corrected reprint of the 2nd (1965) edition, Dover, 2003.

[58] C. Goffman, Definition of the Lebesgue integral, American Mathematical Monthly 60 (1953), 251–252.

[59] C. Goffman, Real Functions, Rinehart & Co., 1953.

[60] C. Goffman, Classroom notes: Arc length, American Mathematical Monthly 71 (1964), 303–304.

[61] C. Goffman, Calculus of Several Variables, Harper & Row, 1965.

[62] C. Goffman, Introduction to Real Analysis, Harper & Row, 1966.

[63] C. Goffman, A note on integration, Mathematics Magazine 44 (1971), 1–4.

[64] C. Goffman, A bounded derivative which is not Riemann integrable, American Mathematical Monthly 84 (1977), 205–206.

[65] R. Goldberg, Methods of Real Analysis, 2nd edition, Wiley, 1976.

[66] E. González-Velasco, Connections in mathematical analysis: the case of Fourier series, American Mathematical Monthly 99 (1992), 427–441.

[67] E. González-Velasco, James Gregory’s calculus in the Geometriæ pars universalis, American Mathematical Monthly 114 (2007), 565–576.

[68] E. González-Velasco, Journey Through Mathematics: Creative Episodes in its History, Springer, 2011.

[69] R. Gordon, A convergence theorem for the Riemann integral, Mathematics Magazine 73 (2000), 141–147.

[70] R. Gordon, A bounded derivative that is not Riemann integrable, Mathematics Magazine 89 (2016), 364–370.

[71] F. Gouvêa, Was Cantor surprised?, American Mathematical Monthly 118 (2011), 198–209.

[72] J. Grabiner, The Origins of Cauchy’s Rigorous Calculus, MIT Press, 1981.

[73] J. Grabiner, Who gave you the epsilon? Cauchy and the origins of rigorous calculus, American Mathematical Monthly 90 (1983), 185–194.

[74] J. Grabiner, The changing concept of change: the derivative from Fermat to Weierstrass, Mathematics Magazine 56 (1983), 195–206.

[75] J. Grabiner, Was Newton’s calculus a dead end? The continental influence of Maclauren’s treatise of fluxions, American Mathematical Monthly 104 (1997), 393–410.

[76] J. Grabiner, A Historian Looks Back: The Calculus as Algebra and selected writings, MAA Spectrum, Mathematical Association of America, 2010.

[77] I. Grattan-Guinness, Why did George Green write his essay of 1828 on electricity and magnetism?, American Mathematical Monthly 102 (1995), 387–396.

[78] J. Gray, Miscellanea: Clear theory, American Mathematical Monthly 92 (1985), 158–159.

[79] J. Gray, The Real and the Complex: A History of Analysis in the 19th Century, Springer, 2015.

[80] J. Gray, Change and Variations — A History of Differential Equations to 1900, Springer, 2021.

[81] J. Green, The theory of functions of a real variable, Mathematics Magazine 24 (1951), 209–217.

[82] J. Green and F. Valentine, On the Arzelà–Ascoli theorem, Mathematics Magazine 34 (1960/61), 199–202.

[83] R. Gunning, An Introduction to Analysis, Princeton University Press, 2018.

[84] J. Hannah, A geometric approach to determinants, American Mathematical Monthly 103 (1996), 401–409.

[85] E. Hewitt and K. Stromberg, Real and Abstract Analysis, Springer-Verlag, 1975.

[86] T. Hildebrandt, Definitions of Stieltjes integrals of the Riemann type, American Mathematical Monthly 45 (1938), 265–278.

254 BIBLIOGRAPHY

[87] K. Hoffman, Analysis in Euclidean Space, Prentice-Hall, 1975.

[88] F. Jones, Lebesgue Integration on Euclidean Space, Jones and Bartlett, 1993.

[89] H. Junghenn, A Course in Real Analysis, CRC Press, 2015.

[90] W. Kaczor and M. Nowak, Problems in Mathematical Analysis I: Real Numbers, Sequences and Series, translated and revised from the 1996 Polish original by the authors, American Mathematical Society, 2000.

[91] W. Kaczor and M. Nowak, Problems in Mathematical Analysis II: Continuity and Differentiation, translated from the 1998 Polish original, revised, and augments by the authors, American Mathematical Society, 2001.

[92] W. Kaczor and M. Nowak, Problems in Mathematical Analysis III: Integration, American Mathematical Society, 2003.

[93] I. Kaplansky, Set Theory and Metric Spaces, 2nd edition, AMS Chelsea, 1977.

[94] H. Kaptanoğlu, In praise of x^{α} sin(\frac{1}{x}), American Mathematical Monthly 108 (2001), 144–150.

[95] H. Kennedy, Research problems: Is there an elementary proof of Peano’s existence theorem for first order differential equations?, American Mathematical Monthly 76 (1969), 1043–1045.

[96] H. Kennedy, Peano: Life and Works of Giuseppe Peano, Reidel, 1980.

[97] G. Klambauer, Mathematical Analysis, Dekker, 1975.

[98] A. Knapp, Basic Real Analysis, Birkhäuser, 2005.

[99] J. Koliha, Mean, meaner, and meanest mean value theorem, American Mathematical Monthly 116 (2009), 356–361.

[100] T. Körner, Differentiable functions on the rationals, Bulletin of the London Mathematical Society 23 (1991), 557–562.

[101] T. Körner, Exercises for Fourier Analysis ([102]). Cambridge University Press, 1993.

[102] T. Körner, Butterfly hunting and the symmetry of mixed partial derivatives, American Mathematical Monthly 105 (1998), 756–758.

[103] T. Körner, A Companion to Analysis: A Second First and First Second Course in Analysis, American Mathematical Society, 2004.

[104] T. Körner, Fourier Analysis, Cambridge University Press, 2022.

[105] S. Krantz, A Guide to Real Variables, Mathematical Association of America, 2009.

[106] S. Krantz, Real Analysis and Foundations, 4th edition, CRC Press, 2017.

[107] S. Krantz, Elementary Introduction to the Lebesgue Integral, CRC Press, 2018.

[108] S. Krantz and H. Parks, The Implicit Function Theorem: History, Theory, and Applications, Birkhäuser, 2002.

[109] K. Kreith, Geometric interpretation of the Implicit Function Theorem, Mathematics Magazine 36 (1963), 64–65.

[110] W. Kulpa, The Poincaré–Miranda theorem, American Mathematical Monthly 104 (1997), 545–550.

[111] M. Laczkovich and V. Sós, Real Analysis — Foundations and Functions of One Variable, 5th edition, translated from the third (2005) Hungarian edition by the authors, Springer, 2015.

[112] S. Lang, Undergraduate Analysis, 2nd edition, Springer-Verlag, 1997.

[113] R. Langer, Fourier’s Series: The Genesis and Evolution of a Theory, The first Herbert Ellsworth Slaught Memorial Paper, American Mathematical Monthly 54 (1947), no. 7, part II.

[114] J. LaVita, Classroom notes: A necessary and sufficient condition for Riemann integration, American Mathematical Monthly 71 (1964), 193–196.

[115] P. Lax, Change of variables in multiple integrals, American Mathematical Monthly 106 (1999), 497–501.

[116] P. Lax, Change of variables in multiple integrals II, American Mathematical Monthly 108 (2001), 115–119.

[117] P. Lax, Rethinking the Lebesgue integral, American Mathematical Monthly 116 (2009), 863–881.

[118] T. Lehrer, The Derivative Song, American Mathematical Monthly 81 (1974), 490.

[119] T. Lehrer, There’s a Delta for Every Epsilon (Calypso), American Mathematical Monthly 81 (1974), 612.

[120] T. Lehrer, The Professor’s Song, American Mathematical Monthly 81 (1974), 745.

[121] L. Levine, On a necessary and sufficient condition for Riemann integrability, American Mathematical Monthly 84 (1977), 205.

[122] J. Lewin, The teaching of mathematics: A truly elementary approach to the bounded convergence theorem, American Mathematical Monthly 93 (1986), 395–397.

[123] J. Lewin, The teaching of mathematics: Some applications of the bounded convergence theorem for an introductory course in analysis, American Mathematical Monthly 94 (1987), 988–993.

[124] C.-K. Li, Norms, isometries, and isometry groups, American Mathematical Monthly 107 (2000), 334–340.

[125] C.-K. Li and W. So, Isometries of l_{p}-norm, American Mathematical Monthly 101 (1994), 452–453.

[126] W. Luxemburg, Arzelà’s dominated convergence theorem for the Riemann integral, American Mathematical Monthly 78 (1971), 970-979.

[127] S. Malik and S. Arora, Mathematical Analysis, 2nd edition, Wiley, 1992.

[128] S. Mandelbrojt, Obituary: Emile Picard, 1856–1941, American Mathematical Monthly 49 (1942), 277–278.

[129] S. Mandelbrojt, The mathematical work of Jacques Hadamard, American Mathematical Monthly 60 (1953), 599–604.

[130] J. Mawhin, Simple proofs of the Hadamard and Poincaré–Miranda theorems using the Brouwer fixed point theorem, American Mathematical Monthly 126 (2019), 260–263.

[131] P. McGrath, Another proof of Clairaut’s theorem, American Mathematical Monthly 121 (2014), 165–166.

[132] J. McKnight, Jr., Brown’s method of extending fixed point theorems, American Mathematical Monthly 72 (1965), 152–155.

[133] B. Mendelson, Introduction to Topology, 3rd edition, Dover, 1990.

[134] R. Metzler, On Riemann integrability, American Mathematical Monthly 78 (1971), 1129–1131.

[135] J. Milnor, Analytic proofs of the “hairy ball theorem” and the Brouwer fixed-point theorem, American Mathematical Monthly 85 (1978), 521-524.

[136] F. Morgan, Real Analysis, American Mathematical Society, 2005.

[137] F. Morgan, Real Analysis and Applications: Including Fourier Series and the Calculus of Variations, American Mathematical Society, 2005.

[138] C. Morrison and M. Stynes, An intuitive proof of Brouwer’s fixed point theorem in R^{2}, Mathematics Magazine 56 (1983), 38–41.

BIBLIOGRAPHY 257

[139] R. Mortini, A short proof of the chain rule for differentiable mappings in R^{n}, Mathematics Magazine 85 (2012), 136–141.

[140] S. Nadler, Jr., A proof of Darboux’s theorem, American Mathematical Monthly 117 (2010), 174–175.

[141] A. Nijenhuis, Strong derivatives and inverse mappings, American Mathematical Monthly 81 (1974), 969–980; addendum, 83 (1975), 22.

[142] L. Olsen, A new proof of Darboux’s theorem, American Mathematical Monthly 111 (2004), 713–715.

[143] S. Ponnusamy, Foundations of Mathematical Analysis, Birkhäuser / Springer, 2012.

[144] M. Protter, Basic Elements of Real Analysis, Springer-Verlag, 1998.

[145] M. Protter and C. Morrey, Jr., A First Course in Real Analysis, 2nd edition, Springer-Verlag, 1991.

[146] C. Pugh, Real Mathematical Analysis, 2nd edition, Springer, 2015.

[147] H. Rademacher, On the condition of Riemann integrability, American Mathematical Monthly 61 (1954), 1–8.

[148] M. Reed, Fundamental Ideas in Analysis, Wiley, 1998.

[149] H. Reiter and A. Reiter, The space of closed subsets of a convergent sequence, Mathematics Magazine 69 (1996), 217–221.

[150] C. Rogers, A less strange version of Milnor’s proof of Brouwer’s fixed-point theorem, American Mathematical Monthly 87 (1980), 525–527.

[151] M. Rosenlicht, Introduction to Analysis, Dover, 1986.

[152] C. Rosentrater, Varieties of Integration, Mathematical Association of America, 2015.

[153] K. Ross, Another approach to Riemann–Stieltjes integrals, American Mathematical Monthly 87 (1980), 660-662.

[154] K. Ross, Elementary Analysis: The Theory of Calculus, 2nd edition, in collaboration with J. López, Springer, 2013.

[155] W. Rudin, Principles of Mathematical Analysis, 3rd edition, McGraw-Hill, 1976.

[156] W. Rudin, Real and Complex Analysis, 3rd edition, McGraw-Hill, 1987.

[157] W. Rudin, Functional Analysis, 2nd edition, McGraw-Hill, 1991.

[158] D. Sanderson, Advanced plane topology from an elementary standpoint, Mathematics Magazine (1980), 81–89.

[159] U. Satyanarayana, A note on Riemann-Stieltjes integrals, American Mathematical Monthly 87 (1980), 477–478.

[160] D. Schattschneider, A multiplicative metric, Mathematics Magazine 49 (1976), 203–205.

[161] D. Schattschneider, The taxicab group, American Mathematical Monthly 91 (1984), 423–428.

[162] R. Shakarchi, Problems and Solutions for Undergraduate Analysis ([112]), Springer-Verlag, 1998.

[163] O. Shisha, An approach to Darboux-Stieltjes integration, American Mathematical Monthly 72 (1965), 890–892.

[164] H. Siegberg, Some historical remarks concerning degree theory, American Mathematical Monthly 88 (1981), 125–139.

[165] D. Smith and M. Vamanamurthy, How small is a unit ball?, Mathematics Magazine 62 (1989), 101–107.

[166] E. Stein and R. Shakarchi, Fourier Analysis: An Introduction, Princeton University Press, 2003.

[167] E. Stein and R. Shakarchi, Real Analysis: Measure Theory, Integration, and Hilbert Spaces, Princeton University Press, 2005.

[168] P. Stein, Classroom notes: A note on the volume of a simplex, American Mathematical Monthly 73 (1966), 299-301.

[169] P. Stein, A proof of a classical theorem in multiple integration, American Mathematical Monthly 75 (1968), 160–163.

[170] G. Stoica, Components used in the Stone–Weierstrass theorem, American Mathematical Monthly 127 (2020), 658.

[171] R. Strichartz, The Way of Analysis, Revised edition, Jones and Bartlett, 2000.

[172] K. Stromberg, The Banach–Tarski paradox, American Mathematical Monthly 86 (1979), 151–161.

[173] K. Stromberg, An Introduction to Classical Real Analysis, AMS Chelsea, 1981.

[174] A. Taylor, A note on an inequality for integrals, American Mathematical Monthly 57 (1950), 93–96.

[175] A. Taylor and W. Mann, Advanced Calculus, 3rd edition, Wiley, 1983.

[176] A. Taylor and S. Wagon, A paradox arising from the elimination of a paradox, American Mathematical Monthly 126 (2019), 306–318.

BIBLIOGRAPHY 259

[177] H. ter Horst, Riemann–Stieltjes and Lebesgue–Stieltjes integrability, American Mathematical Monthly 91 (1984), 551–559.

[178] A. Thompson, Minkowski Geometry, Cambridge University Press, 1996.

[179] B. Thomson, Monotone convergence for the Riemann integral, American Mathematical Monthly 117 (2010), 547–550.

[180] B. Thomson, The bounded convergence theorem, American Mathematical Monthly 127 (2020), 483–503.

[181] T. Tokieda, A mean value theorem, American Mathematical Monthly 106 (1999), 673–674.

[182] G. Tomkowicz and S. Wagon, The Banach–Tarski Paradox, 2nd edition, Cambridge University Press, 2016.

[183] J. Väisälä, A proof of the Mazur–Ulam theorem, American Mathematical Monthly 110 (2003), 633–635.

[184] D. Varberg, On differentiable transformations in R^{n}, American Mathematical Monthly 73 (1966), no. 4, part II, 111-114.

[185] D. Varberg, Change of variables in multiple integrals, American Mathematical Monthly 78 (1971), 42–45.

[186] J. Walter, On elementary proofs of Peano’s existence theorems, American Mathematical Monthly 80 (1973), 282-286.

[187] W. Walter, There is an elementary proof of Peano’s existence theorem, American Mathematical Monthly 78 (1971), 170–173.

[188] W. Walter, Ordinary Differential Equations, translated from the sixth German (1996) edition by R. Thompson, Springer-Verlag, 1998.

[189] G. Weiss, Complex methods in harmonic analysis, American Mathematical Monthly 77 (1970), 465–474.

[190] D. Widder, What is the Laplace transform?, American Methematical Monthly 52 (1945), 410–425.

[191] D. Widder, Advanced Calculus, 3rd edition, Dover, 1989.

[192] B. Youse, Introduction to Real Analysis, Allyn and Bacon, 1972.

[193] H. Zatzkis, Volume and surface of a sphere in N-dimensional Euclidean space, Mathematics Magazine 30 (1957), 155–157.

Index

absolute convergence, 24, 236 absolute value, 2

B(X, Y), 16 B_{N}(X, C^{m}), 104 B_{N}(X, R^{m}), 104 Baire category theorem, 66 Banach spaces, 233 bases, 59 bilipschitz mappings, 118 bounded convergence theorem, 232 bounded linear mappings, 237 bounded mappings, 16 bounded sequences, 14 bounded sets, 13 Brouwer’s fixed-point theorem, 122

C, 4 c(X, C), 47 c(X, C^{m}), 103 c(X, R), 47 c(X, R^{m}), 103 C(X, Y), 19 C^{n}, 4 C_{0}(X, C), 53 c_{0}(X, C), 30 C_{0}(X, R), 53 c_{0}(X, R), 30 c_{00}(X, C), 8 c_{00}(X, R), 8 C_{b}(X, Y), 19 C_{com}(X, C), 43 C_{com}(X, R), 43 C_{N}(X, C^{m}), 105 C_{N}(X, R^{m}), 105 C_{b,N}(X, C^{m}), 105 C_{b,N}(X, R^{m}), 105 Cartesian products, 167 Cauchy products, 90 Cauchy sequences, 10 Cauchy subsequence property, 63 Cauchy–Schwarz inequality, 4, 21, 27, 35, 40, 225, 235

cells, 102

closed balls, 14

compact support, 42

complementary linear subspaces, 160 complete metric spaces, 10 complex conjugate, 96 complex linear mappings, 107 connected sets, 141 continuous differentiability, 130 contraction mapping theorem, 121 convergent series, 23, 235 convex sets, 246 Cramer’s rule, 159

dense sets, 46

derivative of a function, 2 determinants, 113, 158 diameter of a set, 55 differentiable mappings, 2, 133, 138 differential of a mapping, 133 dimensions of vector spaces, 159 directional derivatives, 132 discrete metric, 42 dominated convergence theorem, 220

eigenvalues, 114 eigenvectors, 114 equicontinuity, 71 equiconvergence of limits, 82 Euclidean metric, 4, 6 Euclidean norm, 3, 5 extended real numbers, 200 extreme value theorem, 196

first category, 68

generalized convergence, 213 GL(R^{n}), 148 gradient of a function, 134 graphs of mappings, 192

Hilbert spaces, 235 Hilbert–Schmidt norm, 107, 111 Hölder continuity, 241 homeomorphisms, 196

implicit function theorem, 154 inner products, 96, 97, 100, 101, 228,

229, 234 interior of a set, 68 inverse function theorem, 149 invertible linear mappings, 148 isometries, 116, 117 iterated integrals, 178

joint continuity, 175

kernels of linear mappings, 159

L(C^{n}, C^{m}), 110 L(R^{n}), 147 L(R^{n}, R^{m}), 139 L(R^{n}, R^{m}), 110 ℓ^{1}(X, C), 207 ℓ^{1}(X, R), 207 ℓ^{1}(Z_{+}, C), 25 ℓ^{1}(Z_{+}, R), 25 ℓ^{2}(X, C), 225 ℓ^{2}(X, R), 225 ℓ^{2}(Z_{+}, C), 26 ℓ^{2}(Z_{+}, R), 26 ℓ^{∞}(X, C), 18 ℓ^{∞}(X, R), 18 ℓ_{N}^{∞}(X, C^{m}), 104 ℓ_{N}^{∞}(X, R^{m}), 104 Lebesgue integrals, 231 limit point property, 61 Lindelöf property, 60 Lindelöf’s theorem, 60 linear mappings, 107 linear subspaces, 47, 159, 233 Lip(f), 115 Lip(X, Y), 114 Lipschitz mappings, 1, 114 of order α, 241 local compactness, 51, 52 locally constant mappings, 142 locally Lipschitz mappings, 144 lower semicontinuity, 195, 196

matrices, 106 meager sets, 68 Minkowski’s inequality, 6 monotone convergence theorem, 219

nonmeager sets, 68

norms, 3, 5, 8, 19, 20, 33, 44, 106, 111,

233 nowhere dense sets, 68 nowhere vanishing, 48 null spaces of linear mappings, 159

open balls, 13

open mappings, 124 operator norms, 111, 238 orthogonal vectors, 237

partial derivatives, 130

partial Lipschitz conditions, 128, 174 partitions of intervals, 36, 178 partitions of unity, 186 path-connected sets, 248 piecewise-linear functions, 178 pointwise boundedness, 77 pointwise convergence, 11 pointwise Lipschitz conditions, 134 projections, 160

R, 2 R^{n}, 3 radius of convergence, 87 rank of a linear mapping, 156 rank theorem, 158, 162 rearrangements, 92 Riemann–Stieltjes integrals, 36 second category, 68 semicontinuity, 195 seminorms, 37, 246 separable metric spaces, 58 separate continuity, 175 separated sets, 141 separates points, 48 sequential compactness, 63 small subsequence property, 63 square-summable functions, 224 Stone–Weierstrass theorem, 47, 48 subalgebras, 47 summable functions, 204, 207 sums of linear subspaces, 159 supports of functions, 8, 42 supremum metric, 17 supremum norm, 19, 33, 44, 104

totally bounded sets, 57 transverse linear subspaces, 159 triangle inequality, 3

uniform boundedness, 78 uniform continuity, 175 uniform convergence, 11 uniform equicontinuity, 71 uniformly Cauchy sequences, 72 upper semicontinuity, 195, 196

vanishing at infinity, 30, 52

Weierstrass’ criterion, 85 Weierstrass’ theorem, 46

Z_{+}, 8