Invited Talk: A Fast Approximate AIB Algorithm for Distributional Word Clustering

Lei Wang, Ph.D

Senior Lecturer, Fellow, Faculty of Informatics

School of Computer Science and Software Engineering

University of Wollongong, New South Wales, Australia

hosted by Jieping Ye

Wednesday, July 3rd 2013, 10:30 am – 12:00 pm

Biodesign A 250

A Fast Approximate AIB Algorithm for Distributional Word Clustering

Abstract: Distributional word clustering merges the words having similar probability distributions to attain reliable parameter estimation, compact classification models and even better classification performance. Agglomerative Information Bottleneck (AIB) is one of the typical word clustering algorithms and has been applied to both traditional text classification and recent image recognition. Although enjoying theoretical elegance, AIB has one main issue on its computational efficiency, especially when clustering a large number of words. Different from existing solutions to this issue, we analyze the characteristics of its objective function — the loss of mutual information, and show that by merely using the ratio of word-class joint probabilities of each word, good candidate word pairs for merging can be easily identified. Based on this finding, we propose a fast approximate AIB algorithm and show that it can significantly improve the computational efficiency of AIB while well maintaining or even slightly increasing its classification performance. Experimental study on both text and image classification benchmark data sets shows that our algorithm can achieve more than 100 times speedup on large real data sets over the state-of-the-art method.

BIO: Dr. Lei Wang received his Ph.D. from Nanyang Technological University, Singapore in 2004. Now he is with Faculty of Informatics of University of Wollongong as Senior Lecturer. He was awarded the Australian Postdoctoral Fellowship by the Australian Research Council and the Early Career Researcher Award by Australian Academy of Science. His research interest lies at machine learning, pattern recognition and computer vision. For machine learning and pattern recognition, he is interested in feature selection, model selection, and kernel-based learning methods. For computer vision, he is interested in content-based image retrieval and generic image categorization.