Knowledge Transfer via Multiple Model Local Structure Mapping

The e®ectiveness of knowledge transfer using classi¯cation algorithms depends on the di®erence between the distribu- tion that generates the training examples and the one from which test examples are to be drawn. The task can be es- pecially di±cult when the training examples are from one or several domains di®erent from the test domain. In this paper, we propose a locally weighted ensemble framework to combine multiple models for transfer learning, where the weights are dynamically assigned according to a model's pre- dictive power on each test example. It can integrate the advantages of various learning algorithms and the labeled information from multiple training domains into one uni¯ed classi¯cation model, which can then be applied on a di®erent domain. Importantly, di®erent from many previously pro- posed methods, none of the base learning method is required to be speci¯cally designed for transfer learning. We show the optimality of a locally weighted ensemble framework as a general approach to combine multiple models for domain transfer. We then propose an implementation of the local weight assignments by mapping the structures of a model onto the structures of the test domain, and then weight- ing each model locally according to its consistency with the neighborhood structure around the test example. Experi- mental results on text classi¯cation, spam ¯ltering and in- trusion detection data sets demonstrate signi¯cant improve- ments in classi¯cation accuracy gained by the framework. On a transfer learning task of newsgroup message catego- rization, the proposed locally weighted ensemble framework achieves 97% accuracy when the best single model predicts correctly only on 73% of the test examples. In summary, the improvement in accuracy is over 10% and up to 30% across di®erent problems.