HomeNewsListPh.D candidate Jiantao Jiao from Stanford University visited on November 19

Ph.D candidate Jiantao Jiao from Stanford University visited on November 19

Ph.D candidate Jiantao Jiao from EE Department, Stanford University visited our lab on November 19. He gave a talk on "How to estimate mutual information with insufficient sampling?" in Room 10-206, Rohm Building at 10:00.

Jiantao Jiao obtained his Bachelor degree from the Department of Electronic Engineering, Tsinghua University in 2012, and Master degree from the Department of Electrical Engineering at Stanford University in 2014. He is currently pursuing the Ph.D. degree at Stanford EE. His research interests span over information theory, statistics, learning theory, algorithms, and optimization. He is a recipient of the Highest honor from Tsinghua University (2011), the Stanford Graduate Fellowship (2012), and was invited to give a semi-plenary talk at the IEEE International Symposium on Information Theory (ISIT) in Hong Kong, 2015.

 

Title:

How to estimate mutual information with insufficient sampling?

Abstract:

Mutual information emerged in Shannon's 1948 masterpiece as the answer to the most fundamental questions of compression and communication. Since that time, however, it has been adopted and widely used in a variety of other disciplines. In particular, its estimation has emerged as a key component in fields such as machine learning, computer vision, systems biology, medical imaging, neuroscience, genomics, economics, ecology, and physics. In particular applications, the underlying distribution is usually unknown, so it is of utmost importance to obtain accurate mutual information estimates from empirical data for inference.

We discuss a new approach to the estimation of mutual information between random objects with distributions residing in high-dimensional spaces (e.g. large alphabets), as is the case in increasingly many applications. We will discuss the shortcomings of traditional estimators, and suggest a new estimator achieving essentially optimum worst-case performance under a large class of error criteria. We apply this new estimator in various applications, including the Chow--Liu algorithm and the Tree-Augmented Naive Bayes (TAN) classifier. Experiments with these and other algorithms show that replacing the empirical mutual information by the proposed estimator results in consistent and substantial performance boosts on a wide variety of datasets.