Author's Notes on Convex Analysis and Optimization

PMA: Principles of Mathematical Analysis (Rudin)
RCA: Real and Complex Analysis (Rudin)
CA: Convex Analysis (Rockafellar)
NO: Numerical Optimization (Jocedal and Wright)

1. Differentiation

In this chapter we define and study the properties of differentiation on euclidean spaces. The material here is mostly based on chapter 5 and 9 of PMA, with a few alterations that I believe helps with the exposition.

For instance, I find that it helps to deal with univariate and multivariate differentiation at the same time, which allows, among other things, for the mean value theorem can immediately be generalized to multidimensional spaces, and for Taylor's theorem to be presented alongside its multivariate counterpart.

In addition, I develop elementary fundamental theorems of calculus alongside the theory of differentiation; this has a couple of advantages. One of these advantages is that we are able to exploit the convenient relationship between integrals and derivatives early on, for instance with the derivation of the Lagrange form of the remainder for Taylor's theorem.

Following the lead of PMA, I also present the contraction mapping theorem in this chapter instead of at an earlier point (say, in the measure theory text) so that it can immediately be used in the proof of the inverse function theorem. To demonstrate the power of the theorem, I present two applications to the theory of ordinary differential equations, where the theorem proves instrumental to showing the local and global existence of solutions to systems of differential equations.

2. Convex Analysis

This chapter offers a bare-bones introduction to the elements of convex analysis that appear frequently in microeconomics.

The first part of the chapter is devoted to hyperplane separation theorems. We start with the basic separation theorem, which allows us to separate a point and a closed convex set, and gradually work with more and more general kinds of sets until we arrive at the separating hyperplane theorem for aribtrary non-empty convex sets. In particular, we present a number of applications of the basic separation theorem, including Farkas' lemma, which proves vital to the proof of the KKT theorem in the subsequent chapter.

The rest of the chapter deals with convex functions. Again, we start from the most fundamental properties, such as the continuity of convex functions, and gradually move onto more sophisticated results. The exposition here is based mostly on CO.

3. Static Optimization

Necessary and sufficient conditions for unconstrained and constrained optimization are the main objects of interest in this chapter.

In the case of constrained optimization, we use the proof of the KKT theorem found in NO. There, the authors present a classical but nevertheless clever argument that uses the implicit function theorem in conjunction with Farkas' lemma. I streamline the proof so that it requires only the knowledge of directional derivatives and the gradient of differentiable functions. The rest of the material in the chapter are standard.

4. Correspondences and Fixed Point Theorems

Here we introduce set-valued functions and the notion of hemicontinuity associated with them, along with fixed point theorems involving correspondences.

Instead of starting off in euclidean spaces and adopting the sequential characterization of hemicontinuity, I start from more abstract topological spaces and definitions of hemicontinuity that are suitably more general. Throughout, I try to be as general as possible, for instance, proving Berge's maximum theorem for arbitrary topological spaces instead of metric spaces.

We also present a measurable selection theorem in this chapter, which proves useful later down the line when working with stochastic versions of Taylor's theorem, which is used to derive the Delta method and the asymptotic distribution of GMM/MLE estimators.

Finally, even though it is not usually a topic associated with correspondences, I decided to dedicate the last section to fixed point theorems. This is so I could deal with Brouwer's theorem and Kakutani's theorem in quick succession, instead of proving Brouwer's theorem first and having to put off the latter until after studying correspondences. The proof of Kakutani's theorem is exactly the one in his original 1941 paper A Generalization of Brouwer's Fixed Point Theorem.

I owe my proofs of Sperner's lemma and Brouwer's theorem to the lecture notes on combinatorics used by Stanford Mathematics Professor Jacob Fox; you can find them in his website here. Here's a fun little tidbit: I've been indebted to these notes for a few years now, ever since I first took the core microeconomics course in Korea University. It turns out Professor Fox had taught combinatorics using these lecture notes all the way back when he was a graduate student at, you guessed it, Princeton. I guess some things are just meant to be!