Skip to main content

Ctrl+K

Getting Started

Quickstart
AIBrix Container Images
Installation
FAQ

Architecture

AIBrix Architecture
AIBrix Router
AIBrix Engine Runtime
AIBrix Autoscaler
AIBrix KVCache Offloading Framework
AIBrix StormService

Gateway & Routing

Gateway Routing
Prefill-Decode Disaggregation (PD)
vLLM Semantic Router

Model Serving

Lora Dynamic Loading
Multi-Node Inference
Multi-Engine Support
Heterogeneous GPU Inference (Experimental)
AI Engine Runtime

Scaling & Performance

Autoscaling
- Metric-based Autoscaling
- Optimizer-based Autoscaler
KVCache Offloading
KV Cache Events Synchronization

Batch Inference

Batch API
Batch Model Deployment Templates
Batch Resource Manager for Neoclouds

Benchmark

Brixbench
Benchmark and Workload Generator

Development

Development
Release

Production Readiness

Deploying Gateway
Production Model Deployments
AIBrix Console
Observability

Community

Community
Contribution
Research Collaboration

Index

By the AIBrix Authors

© Copyright 2024, AIBrix Team.