Ask Bhittai
A retrieval-augmented search engine over Shah Jo Risalo, the 18th-century Sindhi Sufi poetry of Shah Abdul Latif Bhittai.
- Year
- 2026
- Role
- Solo: data engineering, retrieval, evaluation, frontend
- Status
- Live
- Context
- Research build
Results
2,519
verse documents indexed
FAISS IndexFlatL2, 384-d multilingual embeddings
51
query benchmark
Hand-built retrieval evaluation set
27.45%
Recall@10
R@1 9.80% · R@5 23.53%: honest numbers on a hard corpus
Built on my own after an Advanced NLP course left me wanting to actually understand retrieval. The corpus came from AMBILE: 4,767 raw verses across all 30 Surs, cleaned by hand and by script (whitespace, missing data, language filtering) into a normalized JSON corpus of 4,730 entries. The 2,519 verses that carry an explanation are the ones indexed.
Roman Sindhi has no spelling rules, so the same word arrives five different ways. A preprocessing layer fixes that before retrieval: dictionary and regex normalization for common variants, and query expansion for characters and places, because embeddings don’t treat “Noori” as a proper noun. A query about Noori is expanded with Jam Tamachi and Keenjhar before it ever reaches the index.
Verses are embedded with paraphrase-multilingual-MiniLM-L12-v2 (384 dimensions), so Sindhi and English queries share one vector space, and indexed in FAISS (IndexFlatL2, exact search). The retrieval helper returns structured context (the original verse, its translation, the explanation and the distance score) rather than a generated answer, and Gemini Flash answers on top of it with a custom system prompt. The Next.js frontend is deployed on Vercel.
I also built a 51-query retrieval benchmark and measured Recall@1, @5 and @10. The numbers are modest, and that is the point: they are measured, and they show exactly where the next improvement has to come from. Same RAG architecture, different chaos.