Dashboard

NVIDIA Indic pretraining dataset: 2.8T tokens, 22 languages

NVIDIA announced a 2.8 trillion token Indic pretraining dataset across 22 languages, plus a translation pipeline. What it is, what it is not, who benefits.

Cecilia Iona
Cecilia Iona
Senior Editor, AI & Product
17 October 20261 min read

NVIDIA has announced Nemotron-pretraining-Indic-v1, a pretraining dataset of about 2.8 trillion tokens across 22 Indian languages, together with an open pipeline for generating more. The announcement is dated 16 October 2026. The dataset itself is described as "releasing soon", so as of today you can read about it but probably cannot download it.

This summary is based on NVIDIA's post on the Hugging Face blog. Figures are quoted from it, not independently checked.

What the NVIDIA Indic pretraining dataset contains

Three separate things are bundled in the post:

  • The corpus. About 2.8 trillion tokens built with AI4Bharat at IIT Madras. 2,601.0 billion are native-script tokens and 224.6 billion are romanized, meaning Indian languages written in the Latin alphabet. All 22 languages have native-script data and twelve also have romanized data, which NVIDIA counts as 34 language-script configurations. The counts use the Gemma-4 tokenizer and are inventory counts, not a training budget.

  • The pipeline. An open, GPU-accelerated recipe built on NeMo Data Designer, published as the Nemotron Bharat translation plugin on GitHub. It covers translation, quality assessment and repair.

  • A model. Nemotron 3.5 Super VL was continued-pretrained on 10 billion Indic tokens drawn from this corpus and from natural Indic text.

How the data was made

Most of the corpus is not written in Indian languages from the start. NVIDIA took documents from 15 Nemotron-CC-v2 subsets, including web text, question and answer pairs, maths and code, then translated them. Gemma-4-31B-it handled translation and assessment for twelve languages, and an 8B model called Indic Translate from Bodhan AI handled the other ten.

Each output was scored and kept only if it reached at least 75 out of 100. Rewrites that recovered failed translations account for 438.4 billion of the tokens. This is the synthetic data approach in practice, and what synthetic data is explains why the quality filter matters more than the volume.

The one benchmark number

NVIDIA reports that continued pretraining on the Indic data improved results by 4.98 to 9.82 percentage points across five Indic benchmarks. The gain is given as a range, with no starting scores in the announcement text, so what it means in plain terms is unclear. It is also a gain for one NVIDIA model, not a promise for yours.

What it does not do

NVIDIA is open about the limits, and they are worth repeating:

  • Translation does not replace text originally written in Indian languages. Idiom, local references and register tend to survive badly.

  • The automated quality checks are only partly backed by human review.

  • An evaluation suite, Indic-LLM-Eval-Suite, is still being developed.

  • No licence is stated in the post. It tells users to read the dataset card and licence before training, so do that before building anything on it.

That caveat connects to a practical point for anyone shipping to Indian users: a model can score well on a benchmark and still produce stiff or wrong output. How to verify an AI translation you cannot read is the checking routine that applies whichever model you pick.

Who this matters to

Most readers of this blog will not train a model, so the realistic effects are indirect.

  • If you build apps for Indian users, better open models for these languages should arrive over the next months. Meanwhile, how prompting changes when you write in your own language and adding multi-language support to an AI-built app are the things you can do today.

  • If you fine-tune models, the pipeline is the useful part. The post says you can load configurations such as hi-Deva-IN with Hugging Face Datasets and stream samples to inspect them without a full download, once the data is out. Pin a dataset revision so results are reproducible.

  • If you only call hosted APIs, nothing changes this week.

For the rest of the week's releases, see how to keep up with AI news.

How did this land?

About the author

Cecilia Iona
Cecilia Iona

Senior Editor, AI & Product

Cecilia leads the Swarmz editorial desk. She has spent a decade turning complex AI and product topics into writing people actually finish, and she owns the blog's quality bar.

Share

Get the next post in your inbox

One email a month. Product updates, engineering posts, and the best of Built with Swarmz.

I agree to receive emails about AI building tips and Swarmz product news. Unsubscribe any time.