PMA: Principles of Mathematical Analysis (Rudin)
RCA: Real and Complex Analysis (Rudin)
PS: Probability and Stochastics (Cinlar)
PM: Probability and Measure (Bilingsley)
CPM: Convergence of Probability Measures (Bilingsley)

1. Basic Elements of Probability Theory

Like its namesake, this chapter presents what I believe are the basic elements of probability theory. We start with the mathematically formal way of defining the distribution and distribution function of a random variable.

Some core concepts are then introduced in quick succession: the Doob-Dynkin lemma as a way of formalizing the concept of the "information" contained in a random variable; expected values as integrals and various useful inequalities; characteristic functions as a means of summarizing the distribution of a random variable; multivariate distributions; and the concept of probabilistic independence.

Despite the central role the normal distribution plays in probability theory and convergence theory in particular, I found many treatments of the normal distribution to be relegated to exercises at best, or entirely omitted at worst. For this reason I dedicated the penultimate section to the development of the univariate and multivariate normal distributions.

2. Conditioning and Information

This chapter formally develops the concepts of conditional expectations and conditional probabilities, and is based primarily on chapter 4 of PS.

Instead of focusing solely on conditional distributions, however, this chapter also rigorously defines conditional densities and presents Bayes' rule in terms of these densities. Although less general than what is presented in Cinlar's textbook, I find the density-based approach to be of more practical use, especially when it comes to Bayesian econometrics.

3. Convergence of Random Variables

Here we study the convergence of random variables taking values in a metric space. Almost sure convergence and convergence in probability are our main focus, and we conclude with a theorem that characterizes convergence in L^p in terms of convergence in probability and uniform integrability.

The beginning of the chapter emphasizes the role of toplogical separability in metric spaces, which has gone overlooked in every probability theory textbook I have read. I also present a detailed discussion of the Big and Little O notations in probability, since they are liberally used in econometric theory papers.

4. Convegence of Measures

This chapter focuses on the weak convergence of measures, more popularly known as convergence in distribution, based primarily on CPM.

Many of the most important topics related to weak convergence are present here; starting with the Portmanteau theorem, which characterizes weak convergence; Slutsky's theorem, which is an indispensible tool in econometric proofs; Prohorov's theorem, which characterizes weak convergence in terms of a concept called tightness; and Levy's continuity theorem, the workhorse for proofs of LLNs and CLTs.

Also addressed are some more esoteric topics, such as Skorokhod's representation theorem, which relates weak convergence to almost sure convergence, and the Prohorov metric, a metric under which weak convergence can be treated as convergence in a metric space. The latter is especially useful for a formal treatment of consumer behavior under uncertainty in microeconomic theory.

5. The Law of Large Numbers and Central Limit Theorems

Here we prove various LLNs and CLTs using the machinery developed in the preceding chapters. Following what is taught in elementary statistics courses, I prove the LLNs and CLTs for i.i.d. sequences via Taylor expansions of characteristic functions.

Of the various CLTs for dependent processes, I focus on martingale difference CLTs because they play a central role in the development of asymptotic theory in time series analysis. To prove LLNs and CLTs for martingale difference sequences, I consulted a variety of sources ranging from papers by Don Andrews to Bilingsley's PM.

6. Continuous Function Spaces

In the final chapter, we study random functions taking values in continuous function spaces and conclude with a treatment of the Functional CLT. The proof of the completeness and separability of continuous function spaces under the supremum norm follows chapter 7 of PMA, while the rest of the material is drawn straight from CPM.