Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. They are text-to-text, decoder-only large language models, available in English, with open weights for both pre-trained variants and instruction-tuned variants. Gemma models are well-suited for a variety of text generation tasks, including question answering, summarization, and reasoning. Their relatively small size makes it possible to deploy them in environments with limited resources such as a laptop, desktop or your own cloud infrastructure, democratizing access to state of the art AI models and helping foster innovation for everyone.
Model Page: Gemma
Authors: Google
Model Data
Data used for model training and how the data was processed.
Training Dataset
These models were trained on a dataset of text data that includes a wide variety of sources. The 27B model was trained with 13 trillion tokens, the 9B model was trained with 8 trillion tokens, and 2B model was trained with 2 trillion tokens. Here are the key components:
- Web Documents: A diverse collection of web text ensures the model is exposed to a broad range of linguistic styles, topics, and vocabulary. Primarily English-language content.
- Code: Exposing the model to code helps it to learn the syntax and patterns of programming languages, which improves its ability to generate code or understand code-related questions.
- Mathematics: Training on mathematical text helps the model learn logical reasoning, symbolic representation, and to address mathematical queries.
The combination of these diverse data sources is crucial for training a powerful language model that can handle a wide variety of different tasks and text formats.
Note: This table showed source model instead of quantized model evaluation. Source Model Evaluation refer to Gemma-2-2B-it Evaluation Result
| Benchmark | Metric | Gemma 2 IT 2B | Gemma 2 IT 9B | Gemma 2 IT 27B |
|---|---|---|---|---|
| RealToxicity | average | 8.16 | 8.25 | 8.84 |
| CrowS-Pairs | top-1 | 37.67 | 37.47 | 36.67 |
| BBQ Ambig | 1-shot, top-1 | 83.20 | 88.58 | 85.99 |
| BBQ Disambig | top-1 | 69.31 | 82.67 | 86.94 |
| Winogender | top-1 | 52.91 | 79.17 | 77.22 |
| TruthfulQA | 43.72 | 50.27 | 51.60 | |
| Winobias 1_2 | 59.28 | 78.09 | 81.94 | |
| Winobias 2_2 | 88.57 | 95.32 | 97.22 | |
| Toxigen | 48.32 | 39.30 | 38.42 |
Users can run large language models on Qualcomm chips using either of the following methods:
Run large models with APLUX AidGen: Please refer to the APLUX AidGen Developer Documentation
Run large models with Qualcomm Genie: Please refer to the Qualcomm Genie Documentation