Why I stopped using semantic embeddings for tool selection and switched back to BM25 [D]
A developer shares production experience building an agent with 140 MCP tools, finding that semantic embeddings for tool selection gave only 64% top-1 accuracy and were confidently wrong. BM25 over tool metadata achieved 81% accuracy, outperforming a hybrid approach that scored 78%. The key insight is that tool descriptions are short and keyword-dependent, making BM25 more effective than embeddings. Indexing schema fields like property names further improved performance. The author recommends testing specific corpora rather than assuming document-RAG defaults transfer to tool selection.