Self-Hosted AI with Hugging Face TGI

Text Generation Inference (TGI) is Hugging Face’s open-source server for running open models on your own NVIDIA GPUs, with an OpenAI-compatible API. Browse 63 open models that run on Hugging Face TGI, see how much GPU memory each one needs, and open any model for a step-by-step guide to chat with it in TypingMind.

Showing 31-60 of 63 models
Meta logo

Llama-3.3 via Hugging Face TGI

Meta · 141.1 GB

New state of the art 70B model. Llama 3.3 70B offers similar performance compared to the Llama 3.1 405B model.

View
Alibaba logo

Qwen2.5-Coder via Hugging Face TGI

Alibaba · 6 sizes from 990 MB

The latest series of Code-Specific Qwen models, with significant improvements in code generation, code reasoning, and code fixing.

32K context
View
Cohere logo

Aya-Expanse via Hugging Face TGI

Cohere · 2 sizes from 16.1 GB

Cohere For AI's language models trained to perform well across 23 different languages.

8K context
View
NVIDIA logo

Llama-3.1-Nemotron via Hugging Face TGI

NVIDIA · 141.1 GB

Llama-3.1-Nemotron-70B-Instruct is a large language model customized by NVIDIA to improve the helpfulness of LLM generated responses to user queries.

128K context
View
NVIDIA logo

Mistral-NeMo-Minitron via Hugging Face TGI

NVIDIA · 16.8 GB

Mistral-NeMo-Minitron-8B-Instruct is a model for generating responses for various text-generation tasks including roleplaying, retrieval augmented generation, and function calling.

8K context
View
Alibaba logo

Qwen2.5-Math via Hugging Face TGI

Alibaba · 3 sizes from 3.1 GB

In August 2024, we released the first series of mathematical LLMs - Qwen2-Math - of our Qwen family.

4K context
View
Meta logo

Llama-3.2-Vision via Hugging Face TGI

Meta · 21.3 GB

Llama 3.2 Vision is a collection of instruction-tuned image reasoning generative models in 11B and 90B sizes.

View
Meta logo

Llama-3.2 via Hugging Face TGI

Meta · 2 sizes from 2.5 GB

Meta's Llama 3.2 goes small with 1B and 3B models.

View
Alibaba logo

Qwen2-VL via Hugging Face TGI

Alibaba · 3 sizes from 4.4 GB

We're excited to unveil Qwen2-VL, the latest iteration of our Qwen-VL model, representing nearly a year of innovation.

32K context
View
01.AI logo

Yi-Coder via Hugging Face TGI

01.AI · 2 sizes from 3 GB

Yi-Coder is a series of open-source code language models that delivers state-of-the-art coding performance with fewer than 10 billion parameters.

128K context
View
Microsoft logo

Phi-3.5-Mini via Hugging Face TGI

Microsoft · 7.6 GB

Phi-3.5-mini is a lightweight, state-of-the-art open model built upon datasets used for Phi-3 - synthetic data and filtered publicly available websites - with a focus on very high-quality, reasoning dense data. The model belongs to the Phi-3 model family and supports 128K token context length.

128K context
View
Meta logo

Llama-3.1 via Hugging Face TGI

Meta · 3 sizes from 16.1 GB

Llama 3.1 is a new state-of-the-art model from Meta available in 8B, 70B and 405B parameter sizes.

View
DeepSeek logo

DeepSeek-V2 via Hugging Face TGI

DeepSeek · 2 sizes from 471.5 GB

A strong, economical, and efficient Mixture-of-Experts language model.

160K context
View
Hugging Face logo

SmolLM via Hugging Face TGI

Hugging Face · 3 sizes from 270 MB

🪐 A family of small models with 135M, 360M, and 1.7B parameters, trained on a new high-quality dataset.

2K context
View
IBM logo

Granite-Code via Hugging Face TGI

IBM · 4 sizes from 7 GB

A family of open foundation models by IBM for Code Intelligence

125K context
View
DeepSeek logo

DeepSeek-Coder-V2-Lite via Hugging Face TGI

DeepSeek · 31.4 GB

In standard benchmark evaluations, DeepSeek-Coder-V2 achieves superior performance compared to closed-source models such as GPT4-Turbo, Claude 3 Opus, and Gemini 1.5 Pro in coding and math benchmarks. The list of supported programming languages can be found here.

160K context
View
Alibaba logo

Qwen2 via Hugging Face TGI

Alibaba · 4 sizes from 990 MB

Qwen2 is a new series of large language models from Alibaba group

32K context
View
Cohere logo

Aya-23 via Hugging Face TGI

Cohere · 16.1 GB

Aya 23, released by Cohere, is a new family of state-of-the-art, multilingual models that support 23 languages.

8K context
View
01.AI logo

Yi-1.5 via Hugging Face TGI

01.AI · 5 sizes from 12.1 GB

Yi-1.5 is an upgraded version of Yi. It is continuously pre-trained on Yi with a high-quality corpus of 500B tokens and fine-tuned on 3M diverse fine-tuning samples.

4K context
View
DeepSeek logo

DeepSeek-V2-Lite via Hugging Face TGI

DeepSeek · 31.4 GB

A smaller Mixture-of-Experts version of DeepSeek-V2 with 16B total and 2.4B active parameters.

160K context
View
Technology Innovation Institute logo

Falcon via Hugging Face TGI

Technology Innovation Institute · 2 sizes from 14.4 GB

A large language model built by the Technology Innovation Institute (TII) for use in summarization, text generation, and chat bots.

2K context
View
Microsoft logo

Phi-3-Medium via Hugging Face TGI

Microsoft · 2 sizes from 27.9 GB

The Phi-3-Medium-4K-Instruct is a 14B parameters, lightweight, state-of-the-art open model trained with the Phi-3 datasets that includes both synthetic data and the filtered publicly available websites data with a focus on high-quality and reasoning dense properties.

4K context
View
NVIDIA logo

Llama3-ChatQA-1.5 via Hugging Face TGI

NVIDIA · 16.1 GB

NVIDIA's Llama 3 model tuned for conversational question answering and retrieval-augmented generation (RAG).

8K context
View
Alibaba logo

Qwen1.5 via Hugging Face TGI

Alibaba · 8 sizes from 1.2 GB

For more details, please refer to our blog post and GitHub repo.

32K context
View
Microsoft logo

Phi-3-Mini via Hugging Face TGI

Microsoft · 2 sizes from 7.6 GB

The Phi-3-Mini-4K-Instruct is a 3.8B parameters, lightweight, state-of-the-art open model trained with the Phi-3 datasets that includes both synthetic data and the filtered publicly available websites data with a focus on high-quality and reasoning dense properties.

4K context
View
Meta logo

Llama-3 via Hugging Face TGI

Meta · 2 sizes from 16.1 GB

Meta Llama 3: The most capable openly available LLM to date

8K context
View
Cohere logo

Command-R-V01 via Hugging Face TGI

Cohere · 70 GB

Command R is a Large Language Model optimized for conversational interaction and long context tasks.

View
DeepSeek logo

DeepSeek-Coder via Hugging Face TGI

DeepSeek · 4 sizes from 2.7 GB

DeepSeek Coder is a capable coding model trained on two trillion code and natural language tokens.

4K context
View
Upstage logo

SOLAR via Hugging Face TGI

Upstage · 21.5 GB

A compact, yet powerful 10.7B large language model designed for single-turn conversation.

4K context
View
Mistral AI logo

Mistral via Hugging Face TGI

Mistral AI · 2 sizes from 14.5 GB

The 7B model released by Mistral AI, updated to version 0.3.

32K context
View

TypingMind for Teams

Bring your self-hosted AI models to your team

Running Hugging Face TGI models on your own GPU server keeps your data private. To give your whole company the same private AI, run your models on a company server and connect them to TypingMind Teams. Everyone gets one AI workspace with your self-hosted models next to the cloud models you allow.

  • Add your self-hosted models once for the whole team. Nobody has to install anything.
  • Self-host TypingMind Teams in your own network, so chats never leave your infrastructure.
  • Control who uses which model with SSO, roles, and user groups.
  • Set usage limits and review every admin change in the audit log.
Self-host TypingMind Teams on GitHub, Google Cloud, Docker, MySQL, AWS, or Node.js