TRIAD finds multimodal retrieval struggles with complete, evolving requests
TRIAD evaluates 17 systems across six multimodal retrieval subsets, each containing 1,000 queries and 1,000 candidates, using multi-positive mAP. Performance generally declines with hierarchical distance and negative constraints; multi-turn retrieval scores lower in 39 of 40 comparisons with paired final intent, though candidate pools highly overlap without being identical. Evaluations are zero-shot and descriptive across public checkpoints; downstream agentic task completion was not tested.
Why read it
It separates similarity ranking from satisfying hierarchy, exclusions, preserved conditions, and revised dialogue state.
Read the original: TRIAD: Multimodal Retrieval across Abstraction, Constraints, and State