We’re rolling out AfriqueLLM — a suite of open Large Language Models adapted to 20 African languages using continued pre-training (CPT) on 26B tokens. This work comes out of Montreal and pushes African language support beyond demos into real performance.
🔗 Models on Hugging Face
Languages covered
Afrikaans, Amharic, Egyptian Arabic, Moroccan Arabic, Tunisian Arabic, Hausa, Igbo, Kinyarwanda, Plateau Malagasy, Nyanja, Oromo, Shona, Southern Sotho, Somali, Swahili, Tigrinya, Tswana, Xhosa, Yoruba, Zulu.
High-resource additions: English, French, Arabic, Portuguese.
What we actually tested
Base models: Llama 3.1 (8B), Gemma 3 (4B & 12B), Qwen 3 (8B & 14B)
Training: 16 nodes / 64× NVIDIA H100
Evaluation: AfroBench-Lite, few-shot
Key findings (no fluff)
Data mixture matters most. Monolingual + code + math + synthetic data wins.
Strong base models beat “multilingual by default.”
Qwen 3 dominates after CPT — AfriqueQwen-14B (63.79) beats Gemma 3 27B (60.33) at less than half the size.
Synthetic data scales low-resource languages.
AfriqueQwen-14B surpasses 54B NLLB-MoE on translation.
Bottom line: African language performance isn’t blocked by scale alone — it’s blocked by data composition and base model strength. AfriqueLLM proves that with the right CPT strategy, smaller models can win.
If you’re building for African users, this is the baseline you should be benchmarking against.
