Don't use a vector database for code, embeddings are slow and bad for code. Code likes bm25+trigram, that gets better results while keeping search responses snappy.
The repo includes also plpgsql_bm25rrf.sql : PL/pgSQL function for hybrid search ( plpgsql_bm25 + pgvector ) with Reciprocal Rank Fusion; and Jupyter notebook examples.
I agree. Someone here posted a drop-in for grep that added the ability to do hybrid text/vector search but the constant need to re-index files was annoying and a drag. Moreover, vector search can add a ton of noise if the model isn't meant for code search and if you're not using a re-ranker.
For all intents and purposes, running gpt-oss 20B in a while loop with access to ripgrep works pretty dang well. gpt-oss is a tool calling god compared to everything else i've tried, and fast.
static embedding models im finding quite fast
lee101/gobed https://github.com/lee101/gobed is 1ms on gpu :) would need to be trained for code though the bigger code llm embeddings can be high quality too so its just yea about where is ideal on the pareto fronteir really , often yea though your right it tends to be bm25 or rg even for code but yea more complex solutions are kind of possible too if its really important the search is high quality
The Nextcloud MCP Server [0] supports Qdrant as a vectordb to store embeddings and provide semantic search across your personal documents. This enables any LLM & MCP client (e.g. claude code) into a RAG system that you can use to chat with your files.
For local deployments, Qdrant supports storing embeddings in memory as well as in a local directory (similar to sqlite) - for larger deployments Qdrant supports running as a standalone service/sidecar and can be made available over the network.
lee101/gobed https://github.com/lee101/gobed static embedding models so they are embedded in milliseconds and on gpu search with a cagra style on gpu index with a few things for speed like int8 quantization on the embeddings and fused embedding and search in the same kernel as the embedding really is just a trained map of embeddings per token/averaging
I thought that context building via tooling was shown to be more effective than rag in practically every way?
Question being: WHY would I be doing RAG locally?
Don't use a vector database for code, embeddings are slow and bad for code. Code likes bm25+trigram, that gets better results while keeping search responses snappy.
You can do hybrid search in Postgres.
Shameless plug: https://github.com/jankovicsandras/plpgsql_bm25 BM25 search implemented in PL/pgSQL ( Unlicense / Public domain )
The repo includes also plpgsql_bm25rrf.sql : PL/pgSQL function for hybrid search ( plpgsql_bm25 + pgvector ) with Reciprocal Rank Fusion; and Jupyter notebook examples.
I agree. Someone here posted a drop-in for grep that added the ability to do hybrid text/vector search but the constant need to re-index files was annoying and a drag. Moreover, vector search can add a ton of noise if the model isn't meant for code search and if you're not using a re-ranker.
For all intents and purposes, running gpt-oss 20B in a while loop with access to ripgrep works pretty dang well. gpt-oss is a tool calling god compared to everything else i've tried, and fast.
Anybody know of a good service / docker that will do BM25 + vector lookup without spinning up half a dozen microservices?
I've gotten great results applying it to file paths + signatures. Even better if you also fuse those results with BM25.
With AI needing more access to documentation, WDYT about using RAG for documentation retrieval?
static embedding models im finding quite fast lee101/gobed https://github.com/lee101/gobed is 1ms on gpu :) would need to be trained for code though the bigger code llm embeddings can be high quality too so its just yea about where is ideal on the pareto fronteir really , often yea though your right it tends to be bm25 or rg even for code but yea more complex solutions are kind of possible too if its really important the search is high quality
https://duckdb.org/2024/05/03/vector-similarity-search-vss
https://github.com/ggozad/haiku.rag/ - the embedded lancedb is convenient and has benchmarks; uses docling. qwen3-embedding:4b, 2560 w/ gpt-oss:20b.
The Nextcloud MCP Server [0] supports Qdrant as a vectordb to store embeddings and provide semantic search across your personal documents. This enables any LLM & MCP client (e.g. claude code) into a RAG system that you can use to chat with your files.
For local deployments, Qdrant supports storing embeddings in memory as well as in a local directory (similar to sqlite) - for larger deployments Qdrant supports running as a standalone service/sidecar and can be made available over the network.
[0] https://github.com/cbcoutinho/nextcloud-mcp-server
I have done some experiments with nomic embedding through Ollama and ChromaDB.
Works well, but I didn't tested on larger scale
Embedded usearch vector database. https://github.com/unum-cloud/USearch
I built a lib for myself https://pypi.org/project/piragi/
That looks great! Is there a way to store / cache the embeddings?
If your data aren't too large, you can use faiss-cpu and pickle
https://pypi.org/project/faiss-cpu/
For the uneducated, how large is too large? Curious.
FAISS runs in RAM. If your dataset can't fit into ram, FAISS is not the right tool.
Shoud it be:
If the total size of your data isn't loo large...?
Data being a plural gets me.
You might have small datums but a lot of kilobytes!
Any suggestion what to use as embeddings model runtime and semantic search in C++?
lee101/gobed https://github.com/lee101/gobed static embedding models so they are embedded in milliseconds and on gpu search with a cagra style on gpu index with a few things for speed like int8 quantization on the embeddings and fused embedding and search in the same kernel as the embedding really is just a trained map of embeddings per token/averaging
Local LibreChat which bundles a vector db for docs.
LightRAG, Archestra as a UI with LightRAG mcp
Sqlite-vec
Anythingllm is promising
A little BM25 can get you quite a way with an LLM.
try out chroma or better yet as opus to!
simple lil setup with qdrant
sqlite's bm25
SQLite with FTS5
Undergrowth.io
A new account, named after the thinking you're linking just looks like spam.
Also I've got no idea what this product does, this is just a generic page of topical ai buzzwords
Don't tell me what it is, /show me why/ you built it. Then go back and keep that reasoning in, show me why I should care
undergrowth.io