W-Index: An Index for Evaluating Link Prediction considering Only the Role of Wins
With the emergence of numerous link prediction methods, how to accurately evaluate them and select the appropriate one has become a key problem that cannot be ignored. Since AUC was first used for link prediction evaluation in 2008, it is arguably the most preferred metric because it well balances t...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2020-01-01
|
| Series: | Complexity |
| Online Access: | http://dx.doi.org/10.1155/2020/7307058 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832560202034970624 |
|---|---|
| author | Yun Yuan Jingwei Wang Yunlong Ma Min Liu |
| author_facet | Yun Yuan Jingwei Wang Yunlong Ma Min Liu |
| author_sort | Yun Yuan |
| collection | DOAJ |
| description | With the emergence of numerous link prediction methods, how to accurately evaluate them and select the appropriate one has become a key problem that cannot be ignored. Since AUC was first used for link prediction evaluation in 2008, it is arguably the most preferred metric because it well balances the role of wins (the testing link has a higher score than the unobserved link) and the role of draws (they have the same score). However, in many cases, AUC does not show enough discrimination when evaluating link prediction methods, especially those based on local similarity. Hence, we propose a new metric, called W-index, which considers only the effect of wins rather than draws. Our extensive experiments on various networks show that the W-index makes the accuracy scores of link prediction methods more distinguishable, and it can not only widen the local gap of these methods but also enlarge their global distance. We further show the reliability of the W-index by ranking change analysis and correlation analysis. In particular, some community-based approaches, which have been deemed effective, do not show any advantages after our reevaluation. Our results suggest that the W-index is a promising metric for link prediction evaluation, capable of offering convincing discrimination. |
| format | Article |
| id | doaj-art-aece0db467c64457a5aff8b300981e99 |
| institution | Kabale University |
| issn | 1076-2787 1099-0526 |
| language | English |
| publishDate | 2020-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | Complexity |
| spelling | doaj-art-aece0db467c64457a5aff8b300981e992025-02-03T01:28:10ZengWileyComplexity1076-27871099-05262020-01-01202010.1155/2020/73070587307058W-Index: An Index for Evaluating Link Prediction considering Only the Role of WinsYun Yuan0Jingwei Wang1Yunlong Ma2Min Liu3School of Electronic and Information Engineering, Tongji University, Shanghai 201804, ChinaSchool of Electronic and Information Engineering, Tongji University, Shanghai 201804, ChinaSchool of Electronic and Information Engineering, Tongji University, Shanghai 201804, ChinaSchool of Electronic and Information Engineering, Tongji University, Shanghai 201804, ChinaWith the emergence of numerous link prediction methods, how to accurately evaluate them and select the appropriate one has become a key problem that cannot be ignored. Since AUC was first used for link prediction evaluation in 2008, it is arguably the most preferred metric because it well balances the role of wins (the testing link has a higher score than the unobserved link) and the role of draws (they have the same score). However, in many cases, AUC does not show enough discrimination when evaluating link prediction methods, especially those based on local similarity. Hence, we propose a new metric, called W-index, which considers only the effect of wins rather than draws. Our extensive experiments on various networks show that the W-index makes the accuracy scores of link prediction methods more distinguishable, and it can not only widen the local gap of these methods but also enlarge their global distance. We further show the reliability of the W-index by ranking change analysis and correlation analysis. In particular, some community-based approaches, which have been deemed effective, do not show any advantages after our reevaluation. Our results suggest that the W-index is a promising metric for link prediction evaluation, capable of offering convincing discrimination.http://dx.doi.org/10.1155/2020/7307058 |
| spellingShingle | Yun Yuan Jingwei Wang Yunlong Ma Min Liu W-Index: An Index for Evaluating Link Prediction considering Only the Role of Wins Complexity |
| title | W-Index: An Index for Evaluating Link Prediction considering Only the Role of Wins |
| title_full | W-Index: An Index for Evaluating Link Prediction considering Only the Role of Wins |
| title_fullStr | W-Index: An Index for Evaluating Link Prediction considering Only the Role of Wins |
| title_full_unstemmed | W-Index: An Index for Evaluating Link Prediction considering Only the Role of Wins |
| title_short | W-Index: An Index for Evaluating Link Prediction considering Only the Role of Wins |
| title_sort | w index an index for evaluating link prediction considering only the role of wins |
| url | http://dx.doi.org/10.1155/2020/7307058 |
| work_keys_str_mv | AT yunyuan windexanindexforevaluatinglinkpredictionconsideringonlytheroleofwins AT jingweiwang windexanindexforevaluatinglinkpredictionconsideringonlytheroleofwins AT yunlongma windexanindexforevaluatinglinkpredictionconsideringonlytheroleofwins AT minliu windexanindexforevaluatinglinkpredictionconsideringonlytheroleofwins |