A Streamlit lab for comparing chunking, retrieval, and reranking strategies side by side, scored with standard IR metrics on a built-in labelled set.
RAG quality is decided before the model sees anything: by how text is chunked, embedded, retrieved, and reranked. Those choices are usually made by habit, and the effect of changing one is rarely measured.
01
Every strategy registers itself with a decorator, and a duplicate name raises an error. Adding a technique is one new file, and it shows up in every comparison for its stage.
02
Uploads are converted to Markdown with LiteParse, or cleaned with Chonkie's text, Markdown, and table preprocessors.
03
Character, recursive, token, semantic, embedding-semantic, Markdown, table, and tree-sitter code splitting, plus three Chonkie chunkers, each with its boundaries drawn on the text.
04
Keyword, boolean, TF-IDF, BM25, dense, hybrid, and MMR rank the same pasted documents. Hybrid fuses BM25 and dense results with reciprocal rank fusion.
05
Six rerankers run on the candidates, and MRR and NDCG are compared before and after reranking on the built-in labelled set.
06
Embeddings get intrinsic evals (Spearman correlation, analogies, clustering ARI); retrieval gets P@k, R@k, MRR, MAP, and NDCG. The Metrics Explorer names the best strategy per metric, and every run is logged with its latency.
The lab runs with no model downloads, which means two components are approximations, said plainly:
Each page compares strategies for one stage at a time. Retrieval works on pasted documents rather than the chunker's output, so the lab answers 'which retriever is best here', not 'which full pipeline is best'.
Python · Streamlit · Chonkie · LiteParse · sentence-transformers · BM25 · tree-sitter · pytest