Knowledge Transfer via Multiple Model Local Structure Mapping

The e®ectiveness of knowledge transfer using classi¯cation algorithms depends on the di®erence between the distribu- tion that generates the training examples and the one from which test examples are to be drawn. The task can be es- pecially di±cult when the training examples are from one or several domains di®erent from the test domain. In this paper, we propose a locally weighted ensemble framework to combine multiple models for transfer learning, where the weights are dynamically assigned according to a model's pre- dictive power on each test example. It can integrate the advantages of various learning algorithms and the labeled information from multiple training domains into one uni¯ed classi¯cation model, which can then be applied on a di®erent domain. Importantly, di®erent from many previously pro- posed methods, none of the base learning method is required to be speci¯cally designed for transfer learning. We show the optimality of a locally weighted ensemble framework as a general approach to combine multiple models for domain transfer. We then propose an implementation of the local weight assignments by mapping the structures of a model onto the structures of the test domain, and then weight- ing each model locally according to its consistency with the neighborhood structure around the test example. Experi- mental results on text classi¯cation, spam ¯ltering and in- trusion detection data sets demonstrate signi¯cant improve- ments in classi¯cation accuracy gained by the framework. On a transfer learning task of newsgroup message catego- rization, the proposed locally weighted ensemble framework achieves 97% accuracy when the best single model predicts correctly only on 73% of the test examples. In summary, the improvement in accuracy is over 10% and up to 30% across di®erent problems.
Date: August 02, 2008
Book Title: ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining (KDD'08)
Type: InProceedings
Edition: Proc 2008
Address: Las Vegas, NV
Downloads: 491

Has 1 soft copy

remote link


  author = "Jing Gao and Wei Fan and Jing Jiang and Jiawei Han",
  title = "{Knowledge Transfer via Multiple Model Local Structure Mapping}",
  month = "August",
  year = "2008",
  edition = "Proc 2008",
  address = ", Las Vegas, NV, ",
  booktitle = "ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining (KDD'08)",