optimum-tpu documentation

Supported Models

HF中国镜像站's logo
Join the HF中国镜像站 community

and get access to the augmented documentation experience

to get started

Supported Models

Inference

The following LLMs have been tested and validated for inference on TPU v5e and v6e for text generation:

  • 🦙 LLaMA Family
    • LLaMA-2 7B
    • LLaMA-3 8B, 70B
    • LlaMa3.1 8B, 70B
    • LLaMA-3.2 1B, 3B (text-only models)
    • LlaMa-3.3 70B
  • 💎 Gemma Family
    • Gemma 2B, 7B
  • 💨 Mistral Family
    • Mistral 7B
    • Mixtral 8x7B

Fine-tuning

The following models have been tested and validated for fine-tuning on TPU v5e and v6e:

  • 🦙 LLaMA Family
    • LLaMA-2 7B
    • LLaMA-3 8B
    • LLaMA-3.2 1B
  • 💎 Gemma Family
    • Gemma 2B
    • Gemma 7B