In this dissertation we address theoretical topics related to the mixing time or efficiency of certain common types of stochastic algorithms. In the first chapter we present a first hitting time analysis of the Independence Metropolis Sampler. Closed form formulas for the expected value and the variance of its first hitting times as well as simple bounds on this quantities are determined.
The second chapter focuses on the updating scheme of the Gibbs Sampler. In the first part, we present a Kullback-Leibler divergence updating criterion and discuss its feasibility. The main part of the chapter deals with the systematic scan Gibbs sampler and the problem of how many non-equivalent visiting orders there are. We prove that this quantity is bounded above by (n-1)!/2 and moreover, show that when the implicit
neighboring structure is acyclic all visiting orders are equivalent. In the last chapter we review a general framework for auxiliary variables samplers. Such algorithms follow a Gibbs sampler approach, sampling from conditional kernels that can be constructed in a variety of ways. Within this setup we show that an Inde-
pendence Metropolis conditional kernel induces an auxiliary variables sampler that is inferior to the one based on a “standard” conditional kernel.