How to self-host Cosmos-Reason1 with Hugging Face TGI as private AI

Open nvidia/Cosmos-Reason1-7B on Hugging Face. The model ID is the part after huggingface.co/, and you can copy it with the button next to the name. The text-generation-inference tag means the model works with TGI.

Cosmos-Reason1 is open to everyone, so you don't need a Hugging Face account or token to download it.

On your GPU server (1 × 24 GB GPU or more), run:

docker run -d --rm --name tgi --gpus all --shm-size 1g \
  -p 127.0.0.1:8080:80 \
  -v $PWD/data:/data \
  ghcr.io/huggingface/text-generation-inference:3.3.5 \
  --model-id nvidia/Cosmos-Reason1-7B \
  --num-shard 1 \
  --cors-allow-origin https://www.typingmind.com

• -d keeps TGI running after you close the terminal.
• -p 127.0.0.1:8080:80 serves TGI on port 8080 of your server, for connections from the server itself only. The SSH tunnel in step III is one of them.
• -v $PWD/data:/data keeps the downloaded model, so the next start is fast.
• --num-shard 1 keeps the model on one GPU, even if your server has more.
• --cors-allow-origin https://www.typingmind.com lets only the TypingMind web app call TGI from a browser, not other websites you visit.

The first start downloads 16.6 GB of model weights, so it can take a while. Follow along with:

docker logs -f tgi

TGI is ready when the log prints Connected. Press Ctrl+C to stop following the log, then check it with:

curl http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model": "tgi", "messages": [{"role": "user", "content": "Say hi"}]}'

You should get a JSON reply with the model's answer.

TypingMind runs on https://, so your browser blocks calls to a plain http:// server, unless the server is at localhost. An SSH tunnel makes TGI show up at localhost:8080 on your computer. On your computer, run:

ssh -N -L 8080:localhost:8080 user@your-gpu-server

Replace user@your-gpu-server with how you log in to the server, and keep this terminal open while you chat. Skip this step if TGI runs on the computer you use for TypingMind.

Open www.typingmind.com in your browser. Use the web version: the TypingMind Mac app and the Setapp version block plain http:// addresses like TGI's.

Click Models in the left sidebar, then Add custom model.

Fill in the form:

Cosmos-Reason1-7B (TGI)nvidia/Cosmos-Reason1-7B32768http://localhost:8080/v1/chat/completions

Open the Capabilities tab and turn off Auto check capabilities on save, so TypingMind keeps the features you set below.

Then set the features for Cosmos-Reason1:

• Support System Role: on
• Support Streaming Output: on
• Support Image Input: on
• Support Plugins: off
• Support PDF Input: off
• Support External Image URLs: off
• Support Thinking: off
• Support Prompt Caching: off

Click Test & Save. If your browser asks to let www.typingmind.com access other apps and services on this device, click Allow. This lets the page reach TGI at localhost.

Pick Cosmos-Reason1-7B (TGI) in the model menu under the chat box and say hi. Replies come straight from your own GPU server.

Cosmos-Reason1-7B (TGI)

Send images to Cosmos-Reason1

Cosmos-Reason1 can read images. Click the paperclip under the chat box, attach a photo or screenshot, and ask about it. The image stays on your own GPU server.

You can now use your private AI with TypingMind's chat features: prompts, AI agents, projects, and switching between your self-hosted model and cloud models in the same chat.