NVIDIA and AI4Bharat Release 2.8 Trillion Token Indic Language Dataset with Open Data Pipeline
A joint effort brings curated synthetic pretraining data across 22 Indian languages to open-source, addressing a critical gap in non-English language model training.