{Home} {Publications and Codes} {Research} {Projects} {Professional Activities} {People}

Machine Vision, Image Recognition and Machine Learning

[Visual Knowledge Transfer and Adaptation]

Paper:

Lei Zhang* and David Zhang, Robust Visual Knowledge Transfer via Extreme Learning Machine-Based Domain Adaptation, IEEE Transactions on Image Processing (T-IP), vol. 25, no. 10, pp. 4959-4973, 2016. [paper], [Supplementary Material]

Abstract:

We address the problem of visual knowledge adaptation by leveraging labeled patterns from source domain and a very limited number of labeled instances in target domain to learn a robust classifier for visual categorization. This paper proposes a new extreme learning machine (ELM)-based cross-domain network learning framework, that is called ELM-based Domain Adaptation (EDA). It allows us to learn a category transformation and an ELM classifier with random projection by minimizing the L2,1-norm of the network output weights and the learning error simultaneously. The unlabeled target data, as useful knowledge, is also integrated as a fidelity term to guarantee the stability during cross-domain learning. It minimizes the matching error between the learned classifier and a base classifier, such that many existing classifiers can be readily incorporated as the base classifiers. The network output weights cannot only be analytically determined, but also transferrable. In addition, a manifold regularization with Laplacian graph is incorporated, such that it is beneficial to semisupervised learning. Extensively, we also propose a model of multiple views, referred as MvEDA. Experiments on benchmark visual datasets for video event recognition and object recognition demonstrate that our EDA methods outperform the existing cross-domain learning methods.

Approach:

The data distribution and decision boundaries. (a) linear classifiers learned for a three-class problem on labeled data in source domain. (b) classifiers learned on the source domain do not fit the target domain due to the change of data distribution. (c) domain adaptation with EDA by simultaneously learning new classifier and category transformation matrix. Note that the category transformation denotes the output adaptation with a matrix Theta.

The flowchart of the proposed cross-domain learning framework in multiple views. Specifically, for each domain, the same type of feature is first extracted. Second, the base classifier is trained on the raw feature of source data. Third, the feature mapping (random projection) is conducted on the both features of source and target data. Fourth, the EDA based domain adaptation classifier is learned. Finally, the visual categorization task with domain adaptation is done.

Experiments:

we evaluate our proposed methods EDA and MvEDA on four datasets: 1) the challenging YouTube & Consumer videos (SIFT and ST features), 2) the 3DA Office dataset (SURF feature), 3) the 4DA Extended office dataset (SURF vs. CNN features), 4) the Bing-Caltech dataset (Classeme feature). Notably, EDA is termed for single feature scenarios and MvEDA is termed for multiple features based application scenarios (e.g., YouTube videos).

Convergence of EDA for three datasets: (a, d) Consumer & YouTube Videos; (b, e) 3DA Office dataset: webcam→dslr; (c, f) 4DA Extended Office dataset: amazon→dslr.

References:

[1] G.-B. Huang, “What are extreme learning machines? Filling the gap between frank Rosenblatt’s dream and John von Neumann’s puzzle,” Cognit. Comput., vol. 7, no. 3, pp. 263–278, 2015.

[2] L. Zhang and D. Zhang, “Domain adaptation extreme learning machines for drift compensation in E-nose systems,” IEEE Trans. Instrum. Meas., vol. 64, no. 7, pp. 1790–1801, Jul. 2015.

[3] W. Li, L. Duan, D. Xu, and I. W. Tsang, “Learning with augmented features for supervised and semi-supervised heterogeneous domain adaptation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 6, pp. 1134–1148, Jun. 2014.

[4] J. Yang, R. Yan, and A. G. Hauptmann, “Cross-domain video concept detection using adaptive SVMs,” in Proc. ACM MM, 2007, pp. 188–197.