GraphRag paper

links

2404.16130

https://www.microsoft.com/en-us/research/blog/graphrag-new-tool-for-complex-data-discovery-now-on-github/

https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/

Figure 2: Head-to-head win rate percentages of (row condition) over (column condition) across two datasets, four metrics, and 125 questions per comparison (each repeated five times and averaged). The overall winner per dataset and metric is shown in bold. Self-win rates were not computed but are shown as the expected 50% for reference. All Graph RAG conditions outperformed na¨ıve RAG on comprehensiveness and diversity. Conditions C1-C3 also showed slight improvements in answer comprehensiveness and diversity over TS (global text summarization without a graph index).

1. Comprehensiveness. How much detail does the answer provide to cover all aspects and details of the question?
2. Diversity. How varied and rich is the answer in providing different perspectives and insights on the question?
3. Empowerment. How well does the answer help the reader understand and make informed judgments about the topic?
4. Directness: “How specifically and clearly does the answer address the question?”