Build a RAG pipeline on Velqa.dev — embeddings, pgvector and reranking

A RAG pipeline answers questions from your own documents. Velqa serves the three model calls — embedding, reranking, chat — behind a single API key.

Velqa does not host your vectors. There is no index service to provision, no vector database to rent, no copy of your documents on our side. Your data stays in your own Postgres (or any vector store you already run) and you call Velqa only for the model steps. One key, one bill.

The pipeline

All three models are included in every plan (Starter, Dev, Pro, Boost top-up).

Prepare the database

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE chunks (
  id        bigserial PRIMARY KEY,
  source    text NOT NULL,
  content   text NOT NULL,
  embedding vector(1024) NOT NULL   -- bge-m3 returns 1024 dimensions
);

CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);

The dimension is fixed when the table is created. Moving from bge-m3 (1024) to qwen3-embedding-8b (up to 4096) means a new column and a full re-index of your documents.

1. Index

import psycopg
from openai import OpenAI
from pgvector.psycopg import register_vector

client = OpenAI(base_url="https://api.velqa.dev/v1", api_key="VELQA_API_KEY")
conn = psycopg.connect("postgresql://localhost/rag")
register_vector(conn)

def chunk(text, size=1200, overlap=150):
    step = size - overlap
    return [c for i in range(0, len(text), step) if (c := text[i:i + size]).strip()]

def index(source, text):
    parts = chunk(text)
    # the API takes a list: one call per batch, not one call per chunk
    vectors = client.embeddings.create(model="bge-m3", input=parts).data
    with conn.cursor() as cur:
        cur.executemany(
            "INSERT INTO chunks (source, content, embedding) VALUES (%s, %s, %s)",
            [(source, p, v.embedding) for p, v in zip(parts, vectors)],
        )
    conn.commit()

Send chunks in batches of 100 to 500 depending on their size rather than one per call: same price per token, far fewer round trips and far less rate-limit pressure.

2. Search

def search(question, k=30):
    q = client.embeddings.create(model="bge-m3", input=[question]).data[0].embedding
    with conn.cursor() as cur:
        cur.execute(
            "SELECT source, content FROM chunks ORDER BY embedding <=> %s LIMIT %s",
            (q, k),
        )
        return cur.fetchall()

<=> is pgvector's cosine distance — the same operator declared in the HNSW index above. A different operator here silently falls back to a full table scan.

Cast a wide net at this stage (30 to 50 candidates). Vector search is fast but approximate; the reranker is what decides.

3. Rerank

import requests

def rerank(question, rows, top=5):
    resp = requests.post(
        "https://api.velqa.dev/v1/rerank",
        headers={"Authorization": "Bearer VELQA_API_KEY"},
        json={
            "model": "qwen3-reranker-8b",
            "query": question,
            "documents": [content for _, content in rows],
        },
        timeout=60,
    )
    resp.raise_for_status()
    ranked = sorted(resp.json()["results"], key=lambda r: -r["relevance_score"])
    return [rows[r["index"]] for r in ranked[:top]]

The reranker reads every (question, document) pair instead of comparing two vectors: it costs more per token than the embedding, which is why 30 candidates become 5. This is the step that does the most for answer quality — see reranking.

4. Answer

def answer(question):
    top = rerank(question, search(question))
    context = "\n\n".join(
        f"[{i}] ({src})\n{txt}" for i, (src, txt) in enumerate(top, 1)
    )
    resp = client.chat.completions.create(
        model="glm-4.7",
        messages=[
            {
                "role": "system",
                "content": (
                    "Answer only from the provided context. "
                    "Cite your sources in square brackets. "
                    "If the context is not enough, say so instead of guessing."
                ),
            },
            {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {question}"},
        ],
    )
    return resp.choices[0].message.content

Any chat model in the catalog works here. glm-4.7 is a good default; deepseek-v4-flash is cheaper on large contexts.

What it costs

Three line items, all billed per token:

• Indexing is a one-off cost per document, proportional to its size. You only re-index when the document changes or when you switch embedding model.
• Each question pays for one embedding (a few dozen tokens, negligible) then a rerank over the retrieved candidates — the dominant retrieval cost, and it grows with k.
• The answer pays for the context as input and the reply as output, like any chat call.

Per-model prices are on the model catalog page.

Common pitfalls

• Chunks too large: the context dilutes and the reranker scores poorly. Too small and the sentence loses its meaning. 800 to 1500 characters with overlap is a good starting point.
• Sending all 30 candidates to the chat model instead of the 5 reranked ones: you pay 6× the context for a worse answer.
• Forgetting to re-index after switching embedding model. Vectors from two different models are not comparable, even at the same dimension.
• French, Arabic, Darija: bge-m3 and qwen3-reranker-8b are multilingual. Do not translate your documents before indexing, and do not worry about questions asked in a different language from the documents.
• Rate limits during bulk indexing: see rate limits, and space your batches out rather than fanning them out aggressively.

See also

• Embeddings — the two available models and their dimensions.
• Reranking — the /v1/rerank API in detail.
• Model catalog — per-model prices.