Tuning-free Domain Adaptation for Classification and Out-of-Distribution Detection

ABSTRACT

Neural network image classifiers deployed in real tasks must often handle unexpected data from far outside their training sets, especially in safety-sensitive applications where an inappropriate response could cause harm. Out-of-Distribution (OOD) detection codifies this task by augmenting classifiers to detect outliers from unknown classes. While the rise of large pretrained models has improved downstream performance on individual classification tasks, most OOD detection methods rely on a model’s differing responses to in-distribution (ID) data versus OOD data based on trained knowledge of the ID classes. This reliance presents a challenge when attempting to adapt models to new problem domains outside of their trained capabilities without expensive finetuning, while retaining the ability to distinguish outliers.

This talk presents a novel prototypical distance-based refinement to classification and out-of-distribution (OOD) detection using pretrained models without finetuning. Embeddings of the target dataset are used to fit closed-form distribution statistics in the model’s latent space; the resulting distribution can classify in-distribution samples and detect OOD samples, all without specialized training or prior knowledge of the OOD data. This allows large models to be more safely deployed without the high cost of retraining by enhancing their ability to identify and handle out-of-distribution data

Analysis of the distribution properties of large pretrained models when processing new datasets leads to a simple modification to Mahalanobis Distance, which adapts models’ latent space distributions to new domains by removing unused features without the finetuning or hyperparameter searches required by other adaptation procedures. Testing on benchmark datasets demonstrates the ability of this method to adapt existing large pretrained image embedding models to new classification domains outside their trained capabilities.

BIOGRAPHY

Elijah Bolluyt is a postdoctoral fellow in the Department of Electrical and Computer Engineering at Stevens Institute of Technology, advised by Professor Cristina Comaniciu. He received his PhD in Computer Engineering from Stevens, with research focused on dynamic architecture in neural networks. His current research explores outlier robustness and domain adaptation in neural network architectures. His work has appeared in IEEE conferences and Transactions journals investigating topics including replicator dynamics, neural network architectural structure, and biomedical imaging.