Skip to main content

Ctrl+K

Getting Started

Overview & Concepts
Quickstart
AIBrix Container Images
Installation
Advanced Kubernetes Examples
FAQ

Architecture

AIBrix Architecture
AIBrix Gateway
AIBrix Engine Runtime
AIBrix Autoscaler
AIBrix KVCache Offloading Framework
AIBrix StormService
Gateway Model List Discovery Health

Request Routing

Gateway Routing
Prefix-Cache Aware Routing
Prefill-Decode Disaggregation (PD)
Multi-turn Conversation & Agentic Routing
vLLM Semantic Router
vLLM-Omni Multi-Modal Serving
External Replica Routing

Model Management

Multi-Engine Support
Multi-Node Inference
Heterogeneous GPU Inference
Lora Dynamic Loading
AI Engine Runtime
Model Claim
Model Warmup

Scaling & Efficiency

Autoscaling
KVCache Offloading
KV Cache Events Synchronization

Batch Inference

Batch API
Batch Model Deployment Templates
Batch Resource Manager for Neoclouds

Benchmark

Brixbench
Benchmark and Workload Generator

Development

Development
Release

Production Readiness

Deploying Gateway
Production Model Deployments
AIBrix Console
Observability

Community

Community
Contribution
Research Collaboration

Search

Ctrl+K

By the AIBrix Authors

© Copyright 2024, AIBrix Team.