Nanotron Research

The Nanotron team focus on sharing open knowledge and developping open-source libraries for efficient distributed training of large-scale AI models.

Some of its contributions are:

• the Nanotron library
• the Picotron library
• the Ultrascale-Playbook, a comprehensive book covering all distributed/parallelisation and low-level techniques that can be used to efficiently train models at the largest scales.