Human real-time communication creates a limitation on the flow of information, which requires the transfer of carefully chosen and concise data in various situations. Although pointing is sparse, overloaded, and indirect, it allows humans to effectively decode shared information, (ex)change their minds, and plan accordingly. I introduce a model that explains how humans choose information for communication and understand communication by utilizing the linguistics concept of “relevance'' derived from decision-making theory and theory of mind.

The modeling approach taken in this dissertation is inspired by many seemingly separated domains. First, I apply theory of mind from cognitive science and partially observable Markov decision process to formally model the components of human mind and how they make decisions, building a scaffold for modeling human communication. Second, I derive how humans coordinate and share their mind by applying the concepts of paternalistic helping in developmental psychology and philosophical discussion about empathy. Third, I derived the definition of utility-based relevance as how much a signaler's belief can make a positive difference to its receiver's well-being, utilizing the cooperative assumption of human communication in linguistics and comparative psychology. I conducted simulation and human behavioral experiments to show that relevance-based communication model can model the overloaded and indirect human communication and can predict humans' choices of signals in communication. Artificial intelligence agents that communicate with relevance-based models are more well-received by humans. Finally, I use Markov decision process and partially observable Markov decision process to propose a way of finding the best timing for sparse human communication.