Granite-3.1-8b-Instruct
Text Generation
W4A16
post
Granite-3.1-8b-Instruct

Granite-3.1-8B-Instruct is a 8B parameter long-context instruct model finetuned from Granite-3.1-8B-Base using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets tailored for solving long context problems. This model is developed using a diverse set of techniques with a structured chat format, including supervised finetuning, model alignment using reinforcement learning, and model merging.

Performance Reference
Device
Backend
Precision
TTFT
Prefill
Decode
Context Size
File Size
Model Details

Supported Languages: English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese. Users may finetune Granite 3.1 models for languages beyond these 12 languages.

Source Model Evaluation

Note: This table showed source model instead of quantized model evaluation. Source Model Evaluation refer to Granite-3.1-8b-Instruct Evaluation Result

HuggingFace Open LLM Leaderboard V1
Models ARC-Challenge Hellaswag MMLU TruthfulQA Winogrande GSM8K Avg
Granite-3.1-8B-Instruct 62.62 84.48 65.34 66.23 75.37 73.84 71.31
Granite-3.1-2B-Instruct 54.61 75.14 55.31 59.42 67.48 52.76 60.79
Granite-3.1-3B-A800M-Instruct 50.42 73.01 52.19 49.71 64.87 48.97 56.53
Granite-3.1-1B-A400M-Instruct 42.66 65.97 26.13 46.77 62.35 33.88 46.29
HuggingFace Open LLM Leaderboard V2
Models IFEval BBH MATH Lvl 5 GPQA MUSR MMLU-Pro Avg
Granite-3.1-8B-Instruct 72.08 34.09 21.68 8.28 19.01 28.19 30.55
Granite-3.1-2B-Instruct 62.86 21.82 11.33 5.26 4.87 20.21 21.06
Granite-3.1-3B-A800M-Instruct 55.16 16.69 10.35 5.15 2.51 12.75 17.1
Granite-3.1-1B-A400M-Instruct 46.86 6.18 4.08 0 0.78 2.41 10.05
Model Inference

Users can run large language models on Qualcomm chips using either of the following methods:

License
Source Model:APACHE-2.0
Deployable Model:APLUX-MODEL-FARM-LICENSE
Performance Reference
Device
Backend
Precision
TTFT
Prefill
Decode
Context Size
File Size