Browse by author
Lookup NU author(s): Dr Huizhi Liang
This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).
Recommender systems (RSs) are designed to provide personalized recommendations to users. Recently, knowledge graphs (KGs) have been widely introduced in RSs to improve recommendation accuracy. In this study, however, we demonstrate that RSs do not necessarily perform worse even if the KG is downgraded to the user-item interaction graph only (or removed). We propose an evaluation framework KG4RecEval to systematically evaluate how much a KG contributes to the recommendation accuracy of a KG-based RS, using our defined metric KG utilization efficiency in recommendation (KGER). We consider the scenarios where knowledge in a KG gets completely removed, randomly distorted and decreased, and also where recommendations are for cold-start users. Our extensive experiments on four commonly used datasets and a number of state-of-the-art KG-based RSs reveal that: to remove, randomly distort or decrease knowledge does not necessarily decrease recommendation accuracy, even for cold-start users. These findings inspire us to rethink how to better utilize knowledge from existing KGs, whereby we discuss and provide insights into what characteristics of datasets and KG-based RSs may help improve KG utilization efficiency. The code and supplementary material of this article are available at: https://github.com/HotBento/KG4RecEval.
Author(s): Zhang H, Wang D, Sun Z, Li Y, Sun Y, Liang H, Wang W
Publication type: Article
Publication status: Published
Journal: ACM Transactions on Information Systems
Year: 2025
Volume: 43
Issue: 3
Pages: 1-36
Print publication date: 25/02/2025
Online publication date: 21/01/2025
Acceptance date: 16/01/2025
Date deposited: 24/01/2025
ISSN (print): 1046-8188
ISSN (electronic): 1558-2868
Publisher: Association for Computing Machinery
URL: https://doi.org/10.1145/3713071
DOI: 10.1145/3713071
ePrints DOI: 10.57711/1c6m-r488
Altmetrics provided by Altmetric
