Model inference tools

Build image, video, audio and 3D generation workflows by connecting blocks on a graph; for creators who want control, on Windows, Linux or macOS.

Get text answers from the huge Kimi K3 AI model using one CPU and no graphics card.

Run big open-weight AI models on your own gaming PC and chat with them.

llama.cpp is a C/C++ program with no external dependencies for running language models, including models that read images, on hardware such as laptop CPUs, GPUs and phones.

The Python library that defines how open models are written down, covering text, images, audio, video and combinations of them, for both running a model and training one.

An inference engine written in plain C that tries to run very large models — hundreds of billions to trillions of parameters — on hardware that could not normally hold them.

Load a big open language model from Hugging Face and generate text on a GPU with only a few GB of memory.

Ollama lets you choose an AI model and chat with it on your own computer.

Run supported language models and serve their answers to applications with a model-serving library for developers.

A cross-encoder — a model that reads two pieces of text together and judges their relationship — built by further training Qwen3.5.

Get answers from a 27-billion-parameter reasoning model that spends less time thinking to itself before replying.

Give one model text, images, video or audio, with up to a million tokens of context such as a whole code repository or a long tool log, and get text back for coding and agent work.