Embedding inference at scale
for RAG, search, recommendations
// official site: github.com ↗
AI / LLM · PRO TIER
Infinity Embeddingpro
Infinity Embedding is a high-throughput embedding inference server — REST API serving text and image embeddings via models like BGE, E5, Jina, Cohere, and more. OpenAI-compatible /v1/embeddings endpoint makes it a drop-in replacement for the OpenAI embeddings API.
// What it is
A closer look.
Infinity Embedding is a high-throughput embedding inference server — REST API serving text and image embeddings via models like BGE, E5, Jina, Cohere, and more. OpenAI-compatible /v1/embeddings endpoint makes it a drop-in replacement for the OpenAI embeddings API.
5-20× faster than HuggingFace Inference for embeddings, designed for production RAG pipelines.
// Use cases
What it's for.
Concrete scenarios where teams pick Infinity Embedding over the SaaS alternative.
OpenAI-compatible API
drop-in replacement for OpenAI embeddings
Multi-model serving
multiple embedding models in one container
High throughput
batching + tensor parallelism
Long-document embeddings
Jina v3 supports 8k+ tokens
Multilingual embeddings
BGE-M3, multilingual-e5
// Who it's for
Built for these teams.
If your team profile matches one of these, Infinity Embedding is a strong fit out of the box.
RAG pipeline builders
needing embeddings at scale
Search teams
building semantic search
AI app developers
integrating embeddings in their stack
AI agencies
offering embedding services to clients
Hosting providers
selling embedding API tier
// Differentiators
Why teams pick Infinity Embedding.
When evaluating self-hosted options for this category, here are the dimensions on which Infinity Embedding consistently lands above the alternatives.
- ✓MIT license — fully open
- ✓5-20× faster — than HuggingFace Inference
- ✓OpenAI-compatible — works with LangChain, LlamaIndex, etc.
- ✓Multi-model — serve multiple embedding models simultaneously
- ✓Active development — Michael Feil maintains
- ✓Production-tested — used by AI startups in prod
- ✓GPU + CPU — gracefully degrades to CPU
// Integrations
Connects to.
The stack you'll plug Infinity Embedding into — services, protocols, and adjacent apps in the BluixApps catalog.
◇
OpenAI v1
/v1/embeddings endpoint
◈
Reranker support
rerank documents post-retrieval
◆
Pair with
Qdrant / Weaviate / Chroma (vector stores)
▣
Pair with
vLLM / Ollama (RAG completion)
▦
Pair with
LangChain / LlamaIndex (orchestration)
▩
Swagger UI
at /docs
// Adoption & deployment
Notable users & community
- 2k+ GitHub stars (newer but rapidly growing)
- Michael Feil + contributors
- Featured in production RAG roundups
- Active community feedback + integrations
- Multiple AI startups in production
What we ship
- Docker (michaelf34/infinity:latest)
- Default model: BAAI/bge-large-en-v1.5 (configurable via /opt/infinity/.env)
- Persistent volume: HF cache (~1-2 GB per model)
- Port 7884 (Infinity default 7997)
- Swagger UI at
/docs - Install report at
/root/bluixapps/infinity.txt - Recommended model list by use case
- Multi-model serving guide
- Infinity vs alternatives comparison
- Use case examples (BluixApps catalog search, RAG pipelines)
- Pairing suggestions (Qdrant + vLLM + LangChain)
- HF_TOKEN environment variable for gated models
- GPU pre-flight check via
bluixapps_ensure_nvidia_runtime - Backup hook covers HF cache
// Tips & operations
Run it properly.
Operational guidance from running this in production — what to lock down, what surprises people.
// PERFORMANCE
Recommended models by use case
// SECURITY
Multi-model
start with --model-id A --model-id B for parallel
// OPERATIONS
VRAM
4 GB minimum for distilled; 8 GB for large; 16 GB for jina-v3
// RELIABILITY
Speed
5-20× higher throughput than vanilla HF
// DEPLOYMENT
vs OpenAI API
free + private + no rate limit + multi-model
// SCALING
vs sentence-transformers
10× faster batch processing
// MAINTENANCE
Production
reverse proxy + auth + monitoring (Prometheus metrics)
8192
// min ram (MB)
12
// min disk (GB)
7884
// access port
http
// protocol
pro
// bluixapps tier
// Alternatives in AI / LLM
Compare with
Project resources
Official sitegithub.com ↗