Compare RankLLM vs. TILDE

RankLLM is a comprehensive Python toolkit designed to enhance reproducibility in information retrieval research, particularly focusing on listwise reranking techniques. This toolkit provides an extensive array of rerankers, including pointwise models such as MonoT5, pairwise models like DuoT5, and listwise models that work seamlessly with platforms like vLLM, SGLang, or TensorRT-LLM. Furthermore, it features specialized variants like RankGPT and RankGemini, which are proprietary listwise rerankers tailored for enhanced performance. The toolkit comprises essential modules for retrieval, reranking, evaluation, and response analysis, thereby enabling streamlined end-to-end workflows. RankLLM's integration with Pyserini allows for efficient retrieval processes and ensures integrated evaluation for complex multi-stage pipelines. Additionally, it offers a dedicated module for in-depth analysis of input prompts and LLM responses, which mitigates reliability issues associated with LLM APIs and the unpredictable nature of Mixture-of-Experts (MoE) models. Supporting a variety of backends, including SGLang and TensorRT-LLM, it ensures compatibility with an extensive range of LLMs, making it a versatile choice for researchers in the field. This flexibility allows researchers to experiment with different model configurations and methodologies, ultimately advancing the capabilities of information retrieval systems.

TILDE (Term Independent Likelihood moDEl) serves as a framework for passage re-ranking and expansion, utilizing BERT to boost retrieval effectiveness by merging sparse term matching with advanced contextual representations. The initial version of TILDE calculates term weights across the full BERT vocabulary, which can result in significantly large index sizes. To optimize this, TILDEv2 offers a more streamlined method by determining term weights solely for words found in expanded passages, leading to indexes that are 99% smaller compared to those generated by the original TILDE. This increased efficiency is made possible by employing TILDE as a model for passage expansion, where passages are augmented with top-k terms (such as the top 200) to enhance their overall content. Additionally, it includes scripts that facilitate the indexing of collections, the re-ranking of BM25 results, and the training of models on datasets like MS MARCO, thereby providing a comprehensive toolkit for improving information retrieval tasks. Ultimately, TILDEv2 represents a significant advancement in managing and optimizing passage retrieval systems.