> ## Documentation Index > Fetch the complete documentation index at: /llms.txt > Use this file to discover all available pages before exploring further. # AI Inference Operations > How AI inference works on the Livepeer network – pipeline types, the batch vs live-video distinction, hardware requirements, and how jobs flow from application to your GPU node. Livepeer's AI subnet launched in Q3 2024 and has grown into a major source of new fee revenue for Orchestrators. It turns GPU nodes into open, composable inference infrastructure that serves image generation, live-video effects, and large language model completions. AI workloads reach your node through Gateway routing, capability advertisement, and container-based inference. The core operator distinction is between batch inference and live-video inference because the hardware profile and routing logic differ. > [accordion] Already running video? > > AI pipelines sit beside an existing video Orchestrator. The usual delta is: > > * keep the existing go-livepeer service and video pricing flags > * add the AI worker and `aiModels.json` > * budget VRAM for the models you want warm > * expose only the pipelines you keep healthy under load > > See [Dual Mode Configuration](/v2/Orchestrators/guides/deployment-details/dual-mode-configuration.md) for the exact operational delta and [Diffusion Pipeline Setup](/v2/Orchestrators/guides/ai-and-job-workloads/diffusion-pipeline-setup.md) for the first AI worker path. > [!NOTE] > > **Low-LPT entry path:** AI inference is often a better starting point than solo video orchestration when stake is limited. Capability, pricing, and latency matter more than active-set position for many AI jobs. ## How the network routes AI jobs Applications never communicate with Orchestrators directly. Every request flows through a Gateway, which handles authentication, pricing negotiation, and routing to qualified nodes. ```icon="terminal" theme={null} Application / User ↓ Gateway (routing, auth, pricing, QoS) ↓ Orchestrator (go-livepeer + AI Worker) ↓ AI Runner container (your GPU + model) ↓ Result returned through gateway ``` Your Orchestrator advertises **capabilities** – which pipelines it supports and at what price – and Gateways route matching jobs to it. You never build a marketplace, billing system, or authentication layer. You run excellent inference infrastructure. When a Gateway selects your Orchestrator, it is because your combination of capability, pricing, latency, and uptime made you the best option for that specific request. ### How Gateway selection actually works Gateways discover Orchestrators through the `OrchestratorInfo` structure, which your node broadcasts and updates on-chain. The key fields that determine whether you receive AI jobs are: Field What it controls Operator implication `capabilities` Which pipelines you support Declare every pipeline in `aiModels.json` `capabilities_prices` Your price per pipeline and model Must be below the Gateway's `-maxPricePerCapability` `hardware` GPU specs advertised to the Gateway Auto-populated by go-livepeer `price_info` Base transcoding price Controls transcoding job eligibility
**Gateway pricing is a hard gate.** Gateways configure a maximum price they will pay per capability using the `-maxPricePerCapability` JSON flag. A pipeline priced above that maximum receives no jobs from that Gateway, regardless of hardware quality. Before setting prices in `aiModels.json`, check what prices the major Gateways are using. See [Models and VRAM Reference](/v2/Orchestrators/guides/ai-and-job-workloads/model-demand-reference.md) for a pricing reference table and [Gateway Orchestrator Offerings](/v2/Gateways/resources/reference/technical/Orchestrator-offerings) for the full capability discovery protocol documentation. For the complete list of supported pipelines and their model architectures, see [AI Model Support](/v2/developers/concepts/ai-on-Livepeer.md) in the Developers section. ## The two workload types The most important distinction for operators is between **batch AI** and **live-video AI**. These are different job types with different hardware profiles, different runtime architectures, and different operational characteristics. > [card] [Batch AI](/v2/orchestrators/guides/ai-and-job-workloads/diffusion-pipeline-setup.md) > > Request-response inference. An application sends a prompt or media file, your node processes it and returns the result. Includes text-to-image, audio-to-text, image-to-video, LLM completions, and more. > [card] [Cascade live-video AI](/v2/orchestrators/guides/ai-and-job-workloads/realtime-ai-setup.md) > > Continuous frame-by-frame video transformation. Live video streams in, processed video streams out with sub-100ms latency. Used for live AI effects, generative video overlays, and streaming AI agents. ### Comparison Batch AI Live-video AI **Latency target** Seconds per request \<100ms per frame **Input type** Text, image, audio, video files Live video stream (WebRTC) **Output type** Image, text, audio, video Processed live video stream **Runtime** `livepeer/ai-runner` containers `livepeer/ai-runner:live-base` + ComfyStream **Min VRAM** 4–24 GB depending on pipeline 24 GB recommended **Good for** High-throughput, varied workloads Streaming AI effects, Cascade pipeline **Entry hardware** RTX 2060 (8 GB) for LLM; RTX 3090/4090 for diffusion RTX 4090 (24 GB) or better **Pricing unit** Per pixel, per token, or per ms Per frame
## AI pipeline types Livepeer's AI worker supports ten pipeline types. Each pipeline handles a specific class of inference task, with its own model format, VRAM floor, and pricing unit. > [accordion] text-to-image — Generate images from text prompts > > The most widely used batch AI pipeline on the network. Takes a text prompt and sampling parameters, returns a generated image. > > **Minimum VRAM:** 24 GB > **Pricing unit:** Per output pixel > **Recommended model:** `SG161222/RealVisXL_V4.0_Lightning` > **Typical hardware:** RTX 3090, RTX 4090, A5000 > > Diffusion models (Stable Diffusion, SDXL variants) run natively on the managed `livepeer/ai-runner` container. The Lightning and Turbo variants reduce step count to deliver results in under 2 seconds on an RTX 4090. > > **Source:** [SG161222/RealVisXL\_V4.0\_Lightning on HuggingFace](https://huggingface.co/SG161222/RealVisXL_V4.0_Lightning) > [accordion] image-to-image — Style transfer and transformation > > Takes an input image and applies diffusion-based transformation, style transfer, or enhancement. Used for artistic style application, image enhancement, and controlled generation. > > **Minimum VRAM:** 24 GB > **Pricing unit:** Per output pixel > **Recommended model:** SDXL variants, `ByteDance/SDXL-Lightning` > **Typical hardware:** RTX 3090, RTX 4090 > [accordion] image-to-video — Animate a still image > > Generates a short video clip from a single input image. Significantly more VRAM and compute-intensive than image-to-image. > > **Minimum VRAM:** 24 GB > **Pricing unit:** Per output pixel > **Typical hardware:** RTX 4090, A100 > [accordion] image-to-text — Vision-language captioning > > Takes an image and returns a text description. Lower VRAM floor makes this accessible to operators with older consumer cards. > > **Minimum VRAM:** 4 GB > **Pricing unit:** Per input pixel > **Recommended model:** `Salesforce/blip-image-captioning-large` > **Typical hardware:** RTX 2060, GTX 1080 (as secondary pipeline) > [accordion] audio-to-text — Speech recognition and transcription > > Runs Whisper-class speech recognition with timestamps. Widely used for transcription, captioning, and audio search. > > **Minimum VRAM:** 12 GB > **Pricing unit:** Per millisecond of audio > **Recommended model:** `openai/whisper-large-v3` > **Typical hardware:** RTX 3060 12 GB, RTX 3080 10 GB > > **Source:** [openai/whisper-large-v3 on HuggingFace](https://huggingface.co/openai/whisper-large-v3) > [accordion] segment-anything-2 — Promptable segmentation > > Pixel-level object segmentation using SAM2. Takes a prompt (point, box, or mask) and returns a segmentation mask over the input image or video frame. > > **Recommended model:** SAM2 variants > **Source:** [facebookresearch/segment-anything-2 on GitHub](https://github.com/facebookresearch/segment-anything-2) > [accordion] text-to-speech — Natural speech synthesis > > Converts text to natural speech audio. Growing use case for AI-generated video narration and interactive media. > > **Pricing unit:** Per character / per millisecond of output audio > [accordion] upscale — Resolution enhancement > > Upscales low-resolution input to high resolution using diffusion-based super-resolution. > > **Recommended model:** `stabilityai/stable-diffusion-x4-upscaler` > **Pricing unit:** Per input pixel > [accordion] llm — Large language model inference > > OpenAI-compatible text completion endpoint backed by an Ollama-based runner. Runs quantised LLMs with as little as 8 GB VRAM, making it accessible to operators with older consumer GPUs that are unsuitable for diffusion pipelines. > > **Minimum VRAM:** 8 GB > **Pricing unit:** Per custom unit (typically per million tokens) > **Recommended model:** `meta-llama/Meta-Llama-3.1-8B-Instruct` (via Ollama) > **Typical hardware:** GTX 1070 Ti, GTX 1080, RTX 2060 > > The LLM pipeline uses a separate runner architecture from the standard `livepeer/ai-runner` image. See [Batch AI Setup](/v2/Orchestrators/guides/ai-and-job-workloads/diffusion-pipeline-setup.md) for the Ollama deployment guide. > > **Source:** [Cloud SPE Ollama runner blog post](https://www.livepeer.cloud/self-hosting-livepeers-llm-pipeline-deploying-an-ollama-based-gpu-runner-for-ai-orchestrators/) > [accordion] live-video-to-video — Cascade streaming AI > > Continuous frame-by-frame transformation of live video streams. This pipeline takes a WebRTC stream as input and returns a transformed WebRTC stream with sub-100ms per-frame latency. > > **Minimum VRAM:** 24 GB recommended > **Pricing unit:** Per frame > **Runtime:** `livepeer/ai-runner:live-base` + ComfyStream > **Typical hardware:** RTX 4090, A100, H100 > > This pipeline powers the **Cascade** architecture – Livepeer's live-video AI system. It supports live AI effects, live style transfer, and streaming AI agents. > > **Source:** [ComfyStream on GitHub](https://github.com/livepeer/comfystream) ## Hardware by workload type > [!NOTE] > > These are minimum requirements. Running at the minimum will result in longer cold-start times and reduced job competitiveness. The figures below reflect production-ready recommendations. Pipeline Min VRAM Recommended card Notes `llm` 8 GB GTX 1080, RTX 2060 Quantised LLM via Ollama `image-to-text` 4 GB Any NVIDIA GPU Very low VRAM floor `audio-to-text` 12 GB RTX 3060 12 GB Whisper-large-v3 `segment-anything-2` 12 GB RTX 3060 12 GB SAM2 base models `upscale` 16 GB RTX 3080 Diffusion upscaler `text-to-image` 24 GB RTX 3090, RTX 4090 SDXL-class models `image-to-image` 24 GB RTX 3090, RTX 4090 SDXL-class models `image-to-video` 24 GB RTX 4090, A100 Higher compute demand `live-video-to-video` 24 GB RTX 4090, A100 Live-video latency requirement
For detailed VRAM planning, warm model strategy, and multi-pipeline configuration, see [Model Hosting and VRAM Planning](/v2/Orchestrators/guides/ai-and-job-workloads/model-demand-reference.md). ## What you build and what the network supplies The Livepeer Protocol handles the hard parts of running an inference marketplace. : **You do need to:** * Run and maintain GPU infrastructure * Configure `aiModels.json` with your supported pipelines and pricing * Keep your primary models warm and your node performant * Stay competitive on latency and pricing **The network already supplies:** * Build a marketplace or API * Implement authentication or billing * Handle service discovery * Build brand recognition Gateways provide all of that. Your competitive advantage is performance: lower latency, better-tuned models, higher uptime, specialised capabilities. ## Network participation To verify your pipelines are visible to the network and check live capability coverage: * **Network capabilities:** [tools.Livepeer.cloud/ai/network-capabilities](https://tools.livepeer.cloud/ai/network-capabilities) * **Orchestrator performance:** [explorer.livepeer.org](https://explorer.livepeer.org) The network capabilities tool shows all registered Orchestrators and their advertised pipelines currently visible on the network. Before your Orchestrator receives AI jobs, it must appear there under at least one pipeline. ## Watch: AI on Livepeer [Live Video AI on Livepeer - Encode Club Bootcamp](https://www.youtube.com/embed/UKWdvJBqrNw) > [card] [Encode Club Live Video AI Bootcamp](https://www.youtube.com/watch?v=UKWdvJBqrNw) > > Full session from the Q1 2025 bootcamp covering ComfyStream, live AI video pipelines, and Orchestrator setup for Cascade workloads. > [card] [ComfyStream Demo](https://www.youtube.com/watch?v=7a6kBrL0RYg) > > Live demonstration of ComfyStream running live-video AI effects through a Livepeer Orchestrator. ## Next steps > [card] [Batch AI Setup](/v2/orchestrators/guides/ai-and-job-workloads/diffusion-pipeline-setup.md) > > Configure pipelines, aiModels.json, the Ollama LLM runner, and BYOC external containers. > [card] [Cascade Setup](/v2/orchestrators/guides/ai-and-job-workloads/realtime-ai-setup.md) > > Deploy the live-video-to-video pipeline with ComfyStream for live-video AI effects. > [card] [Model Hosting and VRAM](/v2/orchestrators/guides/ai-and-job-workloads/model-demand-reference.md) > > VRAM planning, warm model strategy, pricing, and aiModels.json reference. > [card] [Batch AI Setup](/v2/orchestrators/guides/ai-and-job-workloads/diffusion-pipeline-setup.md) > > Upgrade path for existing transcoding Orchestrators adding AI pipelines. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.