MobileCLIP2 is the upgraded generation of the MobileCLIP family, designed for mobile and low-latency scenarios. With only 50–150M parameters and 3–15ms inference latency, it achieves state-of-the-art zero-shot performance. Compared to its predecessor, MobileCLIP2 enhances multi-modal reinforced training through three improvements: (1) stronger CLIP teacher ensembles trained on the DFN dataset for more effective knowledge distillation; (2) improved captioner teachers fine-tuned on diverse high-quality image-text datasets to increase caption diversity and coverage; and (3) the integration of synthetic captions generated by multiple models to further boost robustness. Experimental results show that MobileCLIP2-B improves ImageNet-1k zero-shot accuracy by 2.2% over MobileCLIP-B. Moreover, MobileCLIP2-S4 matches the zero-shot accuracy of SigLIP-SO400M/14 while being 2× smaller and running with lower latency, and it surpasses DFN ViT-L/14 with 2.5× speedup. We release pretrained models and scalable data generation code to enable the community to extend, reproduce, and build upon our work.

Source model

• Input shape: 1x3x256x256, 1x77
• Number of parameters: 125.1M, 123.6M
• Model size: 482.24M, 476.48M
• Output shape: 1x768, 1x768

Source model repository: mobileclip2

Model Farm provides optimized model resources and test code, which can be obtained through the following two methods:

• Obtain via Model Farm page: Click Models & Test Code in the Performance Reference section on the right to obtain model resources and code packages.

• Obtain via command line (Recommand): Users with APLUX development boards can obtain model resources and code packages through the built-in MMS tool.

For MMS usage, please refer to: MMS Usage & Access to Preview Models

When the user has fine-tuned the source model, the model conversion process must be performed again.

Users can refer to either of the following two methods to complete the model conversion:

• Using AIMO for model conversion: Click Model Conversion Reference in the Performance Reference section on the right to view the conversion steps.

• Using Qualcomm QNN for model conversion: Please refer to the Qualcomm QNN Documentation.

The model performance benchmarks and example code provided by Model Farm are all implemented based on the APLUX AidLite SDK.

For models in .bin format, you can use either of the following two inference engines to run inference on Qualcomm chips:

Inference using APLUX AidLite: please refer to the APLUX AidLite Developer Documentation

Inference using Qualcomm QNN: Please refer to the Qualcomm QNN Documentation

Inference Example Code

The inference example code is implemented using the AidLite SDK.

Click Model & Code to download the model files and the inference code package. The file structure is as follows:

/model_farm_{model_name}_aidlite
    
    |__ models # folder where model files are stored

    |__ python # aidlite python model inference example

    |__ cpp # aidlite cpp model inference example

    |__ README.md