Optimizing Active Learning in Vision-Language Models via Parameter-Efficient Uncertainty Calibration

This repository will host the code for the paper titled "Optimizing Active Learning in Vision-Language Models via Parameter-Efficient Uncertainty Calibration"

Stay tuned! The code will be released soon.

Abstract

Active Learning (AL) has emerged as a powerful approach for minimizing labeling costs by selectively sampling the most informative data for neural network model development. Effective AL for large-scale vision-language models necessitates addressing challenges in uncertainty estimation and efficient sampling given the vast number of parameters involved. In this work, we introduce a novel parameter-efficient learning methodology that incorporates uncertainty calibration loss within the AL framework. We propose a differentiable loss function that promotes uncertainty calibration for effectively selecting fewer and most informative data samples for fine-tuning. Through extensive experiments across several datasets and vision backbones, we demonstrate that our solution can match and exceed the performance of complex feature-based sampling techniques while being computationally very efficient. Additionally, we investigate the efficacy of Prompt learning versus Low-rank adaptation (LoRA) in sample selection, providing a detailed comparative analysis of these methods in the context of efficient AL.

Installation

# Clone this repository and enter it
git clone https://github.com/IntelLabs/C_PEAL.git C_PEAL
cd C_PEAL

# Create and activate environment
conda create -n cpeal python=3.10
conda activate cpeal

# Install Python dependencies
pip install -r requirements.txt

# Fetch Dassl and place it at src/dassl
git clone https://github.com/KaiyangZhou/Dassl.pytorch.git src/dassl

If src/dassl already exists and you want to replace it with a fresh clone:

rm -rf src/dassl
git clone https://github.com/KaiyangZhou/Dassl.pytorch.git src/dassl

Credits

This code is built on the CoOp repository. This code is built on the Dassl. This code is built on the Active Prompt Learning.

Citation

If you find this work useful, please consider citing our previous works:

@article{narayanan2024parameter,
  title={Parameter-Efficient Active Learning for Foundational models},
  author={Narayanan, Athmanarayanan Lakshmi and Krishnan, Ranganath and Machireddy, Amrutha and Subedar, Mahesh},
  journal={arXiv preprint arXiv:2406.09296},
  year={2024}
}

License

Details about the license will be provided upon release.