Switch LlamaIndex to abliteration.ai

Definition of Switch LlamaIndex to abliteration.ai

Switching LlamaIndex's base URL means configuring the OpenAI LLM to route requests through an alternative OpenAI-compatible provider.

Why Switch LlamaIndex to abliteration.ai matters

• Keep your existing LlamaIndex indexes and query engines unchanged.
• No changes to document ingestion or retrieval logic.
• Test alternative models for RAG without rebuilding pipelines.
• Works with all LlamaIndex query engines and chat engines.

How it works

• Import OpenAI from <code>llama_index.llms.openai</code>.
• Set <code>api_base="https://api.abliteration.ai/v1"</code> in the constructor.
• Replace your OpenAI key with your abliteration.ai API key.
• Use a supported model id like <code>abliterated-model</code>.

LlamaIndex base URL switch

from llama_index.llms.openai import OpenAI
from llama_index.core import Settings

# Before: OpenAI default
# llm = OpenAI(model="gpt-4")

# After: abliteration.ai
llm = OpenAI(
    model="abliterated-model",
    api_base="https://api.abliteration.ai/v1",
    api_key="YOUR_ABLIT_KEY",
)

# Set as default LLM for all LlamaIndex operations
Settings.llm = llm

# Your query engines work unchanged
response = llm.complete("Summarize this document.")
print(response.text)

# Works with RAG pipelines
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query("What is the main topic?")